As you’re probably aware, we’ve had a series of problems with the wiki lately. I’d like to share a status report as to what those are and where we stand.
Failing wiki extensions
There’s an issue that’s causing extensions that add features to the wiki to fail to start up. This is a problem caused by a bug in the wiki software that’s not properly handling the scenario in which multiple hosts are restarted at the same time. Due to a lack of proper mutual exclusion during wiki startup, the launch of extensions can fail fairly spectacularly, resulting in the loss of the configuration of the extensions, so that future attempts to start them are guaranteed to fail.
This didn’t happen on every restart (oh, the joy of mutual exclusion bugs), but happened pretty often. Since the wiki periodically restarts itself automatically to clean up cruftiness, the result was that this bug would randomly crop up fairly frequently.
Late last week, MindTouch’s support team gave us a script that detects when this has happened and repairs the lost configuration and restarts the extensions. This was tested over the weekend on three of our most commonly-used extensions, and seems to have worked very well. They’ll be adding the rest of our extensions into the mix tomorrow.
That’s a short-term, hacky, workaround for the problem.
MindTouch’s engineers are working on a patch that will correct the underlying bug. They’ve implemented the fix on their trunk codebase, and are testing it there now. It’s a substantial revision to how extensions are loaded, and they want to get it thoroughly tested. Once they’re sure the patch works, they’ll backport it to the release we’re running and test it there. Then, finally, we’ll get it installed, and we should be in good shape.
This situation is being tracked in bug 715341 if you’d like to follow along.
The Great DOM Reference Kerfluffle
While experimenting with some ideas for the DOM Reference, Jean-Yves inadvertently initiated a move of the entire DOM Reference to a different part of the wiki. That was bad enough, but could have been easily fixed. Unfortunately, the site crashed partway through the move, resulting in things being left in a state where it wasn’t a simple “move the subtree back where it belongs” operation. Instead, he had to manually move every page back where it belonged one at a time. This took several days, but is now finished. If you notice any issues with the DOM Reference, please let us know!