The anatomy of an import

I've spent a good part of the last week improving importing to make sure that the nightmare import that happened last week won't repeat itself. The amount of data had to be processed has been doubled since last time, and still the live import took only 30 minutes. To be honest it took 10 minutes, the rest was the purging of the expired disk cache. So, what changed?

The import of Friday the 13th

The import process is still running, as it appears I royally underestimated the time needed for the database server to get through the last batch. About 4.5GB of raw data has been pre-processed into a 700MB SQL that is still being imported as I write this. Many small mistakes escalated into the current situation, here is what happened:

Where the streets have no name

Apparently there are small pocket zones in the game that don't belong to a valid zone. What I've seen so far are small spaceships that you need to visit to slap someone for a quest (not shiphand missions, those are implemented properly). Targeting creatures in these Limbo zones threw an addon error, the latest update fixes this issue.