[Geowanking] Proposal: Improving on the ESRI Personal Geodatabase?

Landon Blake lblake at ksninc.com
Thu Nov 10 12:27:25 PST 2005


I had the same idea a couple of months ago when I was trying to devise a
solution for OpenJUMP's memory problem. (OpenJUMP reads all features
into memory, and crashes when working with very large data files.)

I had tossed around a couple of different solutions, but settled on one
that would store features and geometries in a database like structure.
Once I had decided on this solution, I realized that it could be used to
improve OpenJUMP's design in many other ways.

I have begun work on the intial design of the database, which I have
tentatively named the OJGeoDatabase. I haven't written any code, but
have been concentrating on the API the database will present. My hope is
that a well designed API will allow others to experiment with different
back-end implementations, which will be invisible to OpenJUMP.

The database will use XML for persistent storage, and will include
binary tree indexes stored in XML format. My design will concentrate on
low memory use and will incorporate marshalling/unmarshalling of
database objects using an extension of an XML pull parser.. It will be
able to store features, geometries, queries and relationships. I am
seeking a modular design to allow for easier code maintenance and
upgrades. The main modules of the database are:

[1] Feature Database
[2] Geometry Database
[3] Spatial Index
[4] Relationship Database
[5] Query Manager

My final implementation of the OJGeoDatabase API will be threaded, and
modeled on a client server architecture, so it can be accessed by
multiple users over a network. I am many months from a first release,
but all of the code will be open source, and I hope to have the code
well tested using the JUnit Framework, and well documented for other
programmers.

I hope to make the OJGeoDatabase easily extendable, and it will not be
tied to any particular feature model or geometry library.

I would love to get some input on the design, or to get other open
source GIS projects involved. A common geodatabase format that could be
used among different open source applications would be a truly great
thing.

Please let me know if you are interested, and I can discuss it with you
more.

The Sunburned Surveyor

-----Original Message-----
From: geowanking-bounces at lists.burri.to
[mailto:geowanking-bounces at lists.burri.to] On Behalf Of Alex Willmer
Sent: Thursday, November 10, 2005 5:27 AM
To: geowanking at lists.burri.to
Subject: [Geowanking] Proposal: Improving on the ESRI Personal
Geodatabase?

In my day job I support an ArcSDE/ArcIMS/ArcGIS infrastructure. The GIS
specialists frequently use Personal Geodatabases for offline editing,
temporary storage & project archival.

For those unfamiliar a Personal Geodatabase (PGDB) is an MS Access
database (.mdb) file with some predefined tables and table structures. A
PGDB can store multiple vector and raster layers with attributes, upto a
total size of 2 GiB. An overview can be found here:

http://www.esri.com/software/arcgis/geodatabase/about/personal.html

Recently I've been pondering that a similar concept would be of use to
opensource GIS systems. It could provide a format featuring:
- a single file able to hold multiple datatypes, and layers with indexed
access
- a zero-admin, portable implementation of Simple Features for SQL
- a common transport for moving geodata between disparate databases
- an alternative to shapefiles with fewer limitations on attributes,
types, identifiers etc

I don't suggest reverse engineering the PGDB format (although this would
also be of great use). I think something based on SQLite would fit the
bill. A 

The SQLite library is already popular as an embedded database in
opensource software. It smashes the PGDB limit of 2GiB (SQLite 3 can
potentially address files of 2 TiB). Finally, unlike mdb, SQLite has no
scripting built in - so virus scanners are less likely to interfere with
any file format based upon it (ie mail filters should let it through and
on-access scanners should ignore the potentially large files it would
support.)

Hoping for comments, suggestions & questions.

Regards

Alex

_______________________________________________
Geowanking mailing list
Geowanking at lists.burri.to
http://lists.burri.to/mailman/listinfo/geowanking




More information about the Geowanking mailing list