Skip to content

Empty changeset diffs #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
pnorman opened this issue Sep 26, 2015 · 4 comments
Open

Empty changeset diffs #2

pnorman opened this issue Sep 26, 2015 · 4 comments

Comments

@pnorman
Copy link

pnorman commented Sep 26, 2015

http://planet.openstreetmap.org/replication/changesets/001/507/867.osm.gz is a zero-length file

This is the first one I've encountered. If it corresponded to a time with zero changeset differences, I'd have expected at least <osm />

Cross-ref ToeBee/ChangesetMD#17

@ToeBee
Copy link

ToeBee commented Nov 23, 2015

Generating an invalid gzip file on the server is definitely a problem. Makes it impossible to tell if it was just a bad download or an empty diff file when trying to consume the diffs.

@zerebubuth
Copy link
Owner

This has me very confused - I can't see where in the code it's possible for it to create an invalid or zero-size file, so I've added some sanity checking in commit b6450a6 and hopefully that will provide a clue to what's going wrong.

@ToeBee
Copy link

ToeBee commented Nov 24, 2015

Well since this has only ever happened once, as far as we can tell at least, maybe it was just glitch. Wasn't there a server problem with replication in general a little while ago? Maybe the process got killed mid-stream or something.

@zerebubuth
Copy link
Owner

It's unusual, sure. The reason I'd like to track it down is that I thought I'd written it in such a way as to make zero length files impossible.

According to my mental model, what should be happening is:

  1. The changeset XML is generated in memory from the database. Any failure here should result in an exception and no output.
  2. The XML and state files are written to /tmp. Any failure here should result in an exception leaving garbage in /tmp, but no visible changes on planet.osm.org.
  3. The XML and state files are moved into place, which should be an atomic operation. Any failure here might leave an unwanted file, but it shouldn't be empty or garbage.

Clearly I'm getting at least one assumption wrong... the question is: which one(s)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants