Apr 142010

So we installed the MindTouch software upgrade last night, and as many of you already know, it didn’t go well. The site became incredibly unstable, despite the update working very well on our staging server. While MindTouch investigates why this happened, we’ve rolled MDC back to MindTouch 9.08.3.

As an unfortunate side effect, we had to roll back our database to its state as of about 7 AM PDT yesterday, so most edits applied yesterday were lost. We did back up the database as it stood when we took the site down to do the rollback, so we can retrieve any major edits on request, but you may want to check on the status of any pages you edited yesterday. If you made major edits that were lost, please let me know and I’ll try to get them pulled from the backup database. But this is non-trivial, so please only ask for sizable edits.

Obviously I’m really disappointed that this happened, especially after how carefully — and lengthily — we tested and debugged the upgrade process before applying it to the live site. Clearly there’s some key difference between the staging server and the live MDC that we aren’t seeing, but is causing the software to behave radically differently.

It’s possible that it’s simply the usage. Simulated load doesn’t necessarily precisely duplicate real usage. Analysis is underway already. I’ll blog further details as they’re available.

 Posted by at 1:45 PM

  2 Responses to “MDC upgrade situation”

  1. DekiWiki was clearly the superior choice over the more mature, user, administrator and server friendly alternatives. Definitely worth all the wasted man-hours waiting for pages to load and performing rollback after rollback because MindTouch can’t scale their way out of a wet paper bag.

  2. Not sure what alternative fits all of “more mature”, “user friendly”, “administrator friendly”, and “server friendly”. As a user, writer, and editor, I’ve always hated MediaWiki with a pretty intense passion. Wikitext is for geeks, not for people who spend their entire day writing. While it worked nicely for a select subset of users, it wasn’t ideal.

    We had other issues with MediaWiki as well, which is why we switched to something else. We had plenty of headaches with MediaWiki, including having to hack into the code to implement features we needed, which broke every time there was a new MediaWiki version.

    This isn’t a defense of the problems we’re having with MindTouch, but a reminder that we had major problems before we switched, too. MindTouch is IMHO much more writer-friendly despite a few UI quirks, and is vastly easier to extend and enhance without hacking its code directly.

    That said, obviously these major problems we’re having with scalability and stability are not acceptable, and our patience is not without limit.