Friday, March 22, 2013

Loading .osm into BaseX

Using Open Street Map .osm Data and BaseX with REST services
  • Copy the BaseX77.war into your prefered WebContainer (e.g. JBoss7)
  • Modify the web.xml
  • Download .osm data and load the the .osm data as parts (e.g. split them with osmosis)
  • Use XPath or XQuery over REST for your information needs
Add the following to the web.xml:

  org.basex.indexsplitsize
  1000000

Add in your command line (or script):
curl -i -X PUT http://admin:admin@127.0.0.1:8080/BaseX77/rest/osmdb

curl -i -X PUT -T "austria.part1.osm" http://admin:admin@127.0.0.1:8080/BaseX77/rest/osmdb/austria.part1.osm
curl -i -X PUT -T "austria.part2.osm" http://admin:admin@127.0.0.1:8080/BaseX77/rest/osmdb/austria.part2.osm

curl -i -X GET http://127.0.0.1:8080/BaseX77/rest/osmdb?command=FLUSH

curl -i -X GET http://127.0.0.1:8080/BaseX77-20130321.085136/rest/osmdb?command=OPTIMIZE
Worked fine with the paramters -Xms512m -Xmx 1024m for the following dabase size:
Database Properties:
 Name: osmdb
 Size: 7407 MB
 Nodes: 355247222
 Documents: 588
...
For testing query your oms-xml database by entering an URL to your browser returning the waypoins of a nice place:
for 
 $way in /osm/way, 
 $node in /osm/node
where 
 $way/tag[@k="name"][@v="Michaelerplatz"] and
 $node/@id=$way/nd/@ref
return $node
This query is back in about 3 minutes, the below optimized query is back in about 8 milisecons.
for 
 $way in /osm/way, 
 $node in /osm/node
where 
 $way/tag/@v="Michaelerplatz" and
 $node/@id=$way/nd/@ref
return $node|$way
This will be optimized by BaseX to
for $way in db:attribute("osmdb", "Michaelerplatz")/self::v/parent::tag/parent::*:way 
for $node in db:attribute("osmdb", $way/nd/@ref)/self::id/parent::*:node 
return (($node union $way))