Sep 212011
 

Here are today’s Wiki Wednesday articles! If you know about these topics, please try to find a few minutes to look over these articles that are marked as needing technical intervention and see if you can fix them up. You can do so either by logging into the wiki and editing the articles directly, or by emailing your notes, sample code, or feedback to mdnwiki@mozilla.org.

Contributors to Wiki Wednesday will get recognition in next week’s Wiki Wednesday announcement. Thanks in advance for your help!

JavaScript

SpiderMonkey

Developing Mozilla

Extensions

XUL

XPCOM

Interfaces

Thanks to Neil Rashbrook for his contributions last week.

Plugins

CSS

Thanks to McGurk and Jean-Yves Perrier for their contributions last week.

SVG

HTML

Thanks to John Dyer and McGurk for contributing last week.

DOM

Thanks to Neil Rashbrook for his contributions last week.

 Posted by at 2:36 PM
Sep 142011
 

Why, it’s just fine, thank you! Since our trial fix for the crashing and memory abuse problems was implemented last week (on the 9th), we haven’t had any restarts or crashes. This is insanely good news! Work is now underway on preparing to deploy the longer-term solution for the problem. I don’t have a timeframe for when that will be done, but it’s coming.

I’m extremely excited that, for the first time in ages, the devmo wiki is behaving stably and reliably. It’s been a long time coming!

 Posted by at 5:23 PM
Sep 142011
 

Here are today’s Wiki Wednesday articles! If you know about these topics, please try to find a few minutes to look over these articles that are marked as needing technical intervention and see if you can fix them up. You can do so either by logging into the wiki and editing the articles directly, or by emailing your notes, sample code, or feedback to mdnwiki@mozilla.org.

Contributors to Wiki Wednesday will get recognition in next week’s Wiki Wednesday announcement. Thanks in advance for your help!

JavaScript

SpiderMonkey

Developing Mozilla

Extensions

XUL

XPCOM

Interfaces

Plugins

CSS

Thanks to McGurk for contributing last week.

SVG

HTML

DOM

 Posted by at 5:05 PM
Sep 092011
 

Yesterday, we made substantial progress on fixing the problems we’ve been having with the Mozilla Developer Network wiki. By “we,” of course, I mean our support rep at MindTouch, Brian, and the IT guy at Mozilla whose misfortune it is to have to deal with my constant whining, Jake Maul.

Problem #1: Syntax Errors

First off, Brian figured out why the extensions were failing to have the correct preferences at startup. It turns out that there was a file permissions problem preventing this from working properly. That has been fixed, so we should no longer see the syntax errors in pages that was caused when extensions weren’t running properly. This was most commonly seen with the syntax highlighter extension, but happened with several others that relied on preferences being configured in order to work properly (but those weren’t being used nearly as often).

Problem #2: Stability

Secondly, and arguably more exciting, is that Brian figured out why the site has been crashing and requiring frequent restarts in order to keep running. In order to understand, let’s look at how the site has been configured to date.

In order to handle all our traffic, devmo runs on three nodes, each running its own copy of the MindTouch software. That includes the Lucene indexer software, which is used to build search indexes of the database. Now, in order to keep everything synchronized, all file data was being kept on one NFS share.

And therein lay the problem: we had three MindTouch instances all trying to work with the same Lucene index at the same time. The result: contention. Lots of it. Especially when they were all trying to make changes to the index at the same time. This resulted in collisions that caused delays, which resulted in requests piling up and eventually the software would either crash or memory use would build to the point that the processes would be killed and restarted just to clear the bottleneck.

A Trial Fix

So last night, Brian and Jake worked out a trial fix to see what would happen: they took one of the three hosts out of the load balancer’s rotation, leaving it to do indexing of the wiki’s content but not getting any web traffic. Then they turned off the indexing process on the other two hosts. This, in essence, turned one of the hosts into a dedicated indexing box, while the other two machines’ Lucene requests would access the same files on the NFS share that were being updated by the dedicated indexing machine.

This was deployed yesterday evening. Since then, there have been no crashes or automated restarts due to memory usage. Previously, we had frequent restarts due to memory use exceeding 700 MB; now, memory usage hasn’t been exceeding 400 MB at any time.

This is fantastic news, and bodes well for the long term fix I will describe momentarily.

A Next Step

Over the next few minutes, as I write this post, Jake and Brian are adding the third machine back to the load balancer pool, so that it will receive web traffic in addition to handling all Lucene indexing work. This should improve responsiveness slightly beyond where we are right now by distributing load further. This should be done by the time you read this post; in fact, as I wrote this paragraph, Jake said “Bringing it back up in the LB now.”

Some additional work is being done to verify the current state of the site to ensure that it will keep working smoothly while we work on the long term solution, which I will now describe.

The Long Term Fix

The long-term solution to this problem: a dedicated Lucene server. This machine will be running its own MindTouch instance, but will get no web traffic. Instead, it will handle all API requests for Lucene activity for all three of the web hosts, and will host the Lucene data files locally on its local disk instead of on an NFS share. This will have multiple benefits for us:

  1. Indexing of the data will be done by a single machine, preventing contention (this is temporarily solved by our current fix).
  2. Indexing will not be getting done by any of the web hosts, reducing load on those machines (this is partially fixed by our current solution, but one of our hosts is still doing indexing).
  3. All Lucene requests will be directed to a dedicated box, reducing Lucene related load on our web hosts (this is not addressed at all by the current solution).
  4. By having Lucene processing all handled by a dedicated box, there will be no contention at all for the Lucene data files (this is only addressed for writing by the current solution, and not at all for reading).

There are probably other benefits to this change, but these are the ones I’ve heard bandied about the most.

In Summary

The take-away from all of this: devmo is currently more stable than it’s been in a long time, and is likely to improve both in terms of stability and performance as we continue to work on the long-term solution. In addition, we have other things we are likely to be able to do to improve performance that we’ll look at going forward.

 Posted by at 2:41 PM
Sep 072011
 

Here are today’s Wiki Wednesday articles! If you know about these topics, please try to find a few minutes to look over these articles that are marked as needing technical intervention and see if you can fix them up. You can do so either by logging into the wiki and editing the articles directly, or by emailing your notes, sample code, or feedback to mdnwiki@mozilla.org.

Contributors to Wiki Wednesday will get recognition in next week’s Wiki Wednesday announcement. Thanks in advance for your help!

JavaScript

Thanks to Panagiotis Tsalaportas for contributing last week.

SpiderMonkey

Developing Mozilla

Thanks to Neil Rashbrook for contributing last week.

Extensions

XUL

XPCOM

Thanks to Neil Rashbrook for contributing last week.

Interfaces

Thanks to Neil Rashbrook for contributing last week.

Plugins

CSS

Thanks to McGurk for contributing last week.

SVG

HTML

Thanks to McGurk for contributing last week.

DOM

Thanks to BYK and Matt Brubeck for their contributions last week.

 Posted by at 4:38 PM
Sep 032011
 

I just wanted to post a quick status update for how things stand now with the MDN documentation wiki.

The site is generally working well, aside from the cosmetic issues I posted about previously. There is one technical issue: the site is periodically crashing. We don’t yet know why, but an active investigation is underway. A script is running every few minutes to check the status of the server and restart the wiki if it’s crashed. This is not a solution, of course, but it will at least keep the site usable until we figure out what’s going on.

It sounds more than a little ridiculous to say, but I will anyway: other than the occasional crash, the site is working much better, I think, than it was prior to the upgrade. Once we get this crash sorted out, I think we’re going to be in good shape compared to where we were a week ago.

 Posted by at 1:24 PM
Sep 012011
 

We’ve finished fixing up the major outstanding issues with the MDN wiki, and I thought I’d take a few minutes to write down what’s new and what still needs to be fixed. Let’s start with the bad news: the handful of remaining issues we need to fix. All of them are, fortunately, entirely cosmetic. Yay!

Things that need fixing

  • Currently, there’s a bug in the skin on the site that causes the toolbar to wander down toward the bottom of the window as you scroll. We know what’s wrong there and hopefully it will be fixed very soon.
  • The stylesheet used while editing doesn’t match the one used while viewing content, so the text doesn’t look the way it ought to.
  • The <title> block on user pages is not currently set correctly, so titles of user pages are not informative.
  • Column layout in certain search result and tag listing pages is not correct, resulting in truncated or badly overlapping titles.
  • The tag list has a stray pair of parentheses after it at the bottom of the page.

Stuff we got in this upgrade

This was a pretty major upgrade; we jumped all the way from 9.2.3 through 10.0 to 10.1.1. As such, we have a lot of exciting improvements to enjoy. I’ll touch on the ones that I find most interesting for our usage:

  • The editor is now CKEditor 3.0, which is faster and more flexible. Among other things, we now have:
    • Improved performance.
    • Support for viewing the blocks in the editor.
    • A preview button, although it doesn’t render out templates, so it’s not as useful as it could be for our needs.
    • Local auto-saving of drafts! Your work gets periodically saved locally on your computer, and if you lose power or accidentally quit, you can resume where you left off.
    • A much improved toolbar, with a Code button — complete with a Ctrl-O key equivalent — and individual buttons for H1 through H3. Check it out!
    • Ctrl-S toggles source view mode.
    • While I was at it, I got rid of some cruft we don’t use out of the toolbar.
  • Editing of titles has been improved; it’s no longer part of the page content, but a separate editing box. You can also decouple (or re-couple) the title and the URL path of articles.
  • When you go to your user page, you now see a dashboard showing your recent activity. Your custom content is in a separate tab on that page.
  • The tagging user interface is improved — and is much faster.
  • The Move Page dialog box has been reworded to be less confusing to use.
  • Overall performance seems noticeably better.
  • If you try to edit a page on an iPad, you now can (although only in source mode). Still, in a pinch, it’s better than nothing.
  • Infrastructure exists for page rankings and comments; we have these features disabled for now but may enable them later.
  • Similarly, the built-in search system has been vastly improved but we’re currently not using it. We may soon.

Cross your fingers

Here’s hoping this upgrade helps with some of the problems folks have been having. It took us a couple of days to get things working, but I think it will be worth it. Let me know, or file bugs, if you run into any problems not mentioned above.

 Posted by at 3:13 PM
Sep 012011
 

Here are today’s Wiki Wednesday articles! If you know about these topics, please try to find a few minutes to look over these articles that are marked as needing technical intervention and see if you can fix them up. You can do so either by logging into the wiki and editing the articles directly, or by emailing your notes, sample code, or feedback to mdnwiki@mozilla.org.

Contributors to Wiki Wednesday will get recognition in next week’s Wiki Wednesday announcement. Thanks in advance for your help!

JavaScript

SpiderMonkey

Developing Mozilla

Extensions

XUL

XPCOM

Interfaces

Plugins

Thanks to Panagiotis Tsalaportas for contributing last week.

CSS

Thanks to Panagiotis Tsalaportas for contributing last week.

SVG

HTML

DOM

 Posted by at 9:04 AM