Re: Openstreetmap XML

[Geoff Leyland <geoff_leyland@...>]

On 27/11/2013, at 8:54 pm, steve donovan <steve.j.donovan@gmail.com> wrote:

> On Wed, Nov 27, 2013 at 9:49 AM, Geoff Leyland
> <geoff_leyland@fastmail.fm> wrote:
>> The files are large enough that I suspect it’s not practical to read the whole file into a string and then generate a table
> 
> How big are the files?  It's amazing what you can get away with these
> days with modern computers!

To be honest, I don’t actually know, since I haven’t downloaded a whole country yet.  I just know that the data I have for a small area is as big as data for the whole country in more compact formats (admittedly I live in a small country).  I would expect several GB.  I tend to use LuaJIT, so there’s a hard limit on how big a table can be, as well as a practical limit imposed by how much you want the GC to manage.

> If it's too big, then a stream-based LuaExpat way would be the
> (standard) way to go.

I was just going to fiddle with Roberto’s parser to make it a bit more streamy, but I’ve just read there’s an alternative format for OSM data anyway.