Apr 052018
 

Our fourth and final SEO experiment for MDN, to optimize internal links within the open web documentation, is now finished. Optimizing internal links involves ensuring that each page (in particular, the ones we want to improve search engine results page (SERP) positions for, are easy to find.

This is done by ensuring that each page is linked to from as many topically relevant pages as possible. In addition, it should in turn link outward to other relevant pages. The more quality links we have among related pages, the better our position is likely to be. The object, from a user’s perspective, is to ensure that even if the first page they find doesn’t answer their question, it will link to a page that does (or at least might help them find the right one).

Creating links on MDN is technically pretty easy. There are several ways to do it, including:

  • Selecting the text to turn into a link and using the toolbar’s “add link” button
  • Using the “add link” hotkey (Ctrl-K or Cmd-K)
  • Any one of a large number of macros that generate properly-formatted links automatically, such as the domxref macro, which creates a link to a page within the MDN API reference; for example: {{domxref(“RTCPeerConnection.createOffer()”)}} creates a link to https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/createOffer, which looks like this: RTCPeerConnection.createOffer(). Many of the macros offer customization options, but the default is usually acceptable and is almost always better than trying to hand-make the links.

Our style guide talks about where links should be used. We even have a guide to creating links on MDN that covers the most common ways to do so. Start with these guidelines.

The content updates

10 pages were selected for the internal link optimization experiment.

Changes made to the selected pages

In general, the changes made were only to add links to pages; sometimes content had to be added to accomplish this but ideally the changes were relatively small.

  • Existing text that should have been a link but was not, such as mentions of terms that need definition, concepts for which pages exist that should have been linked to, were turned into links.
  • The uses of API names, element names, attributes, CSS properties, SVG element names, and so forth were turned into links when either first used in a section or if a number of paragraphs had elapsed since they were last a link. While repeated links to the same page don’t count, this is good practice for usability purposes.
  • Any phrase for which a more in-depth explanation is available was turned into a link.
  • Links to related concepts or topics were added where appropriate; for example, on the article about HTMLFormElement.elements, a note is provided with a link to the related Document.forms property.
  • Links to related functions or HTML elements or whatever were added.
  • The “See also” section was reviewed and updated to include appropriate related content.

Changes made to targeted pages

Targeted pages—pages to which links were added—in some cases got smaller changes made, such as the addition of a link back to the original page, and in some cases new links were added to other relevant content if the pages were particularly in need of help.

Pages to be updated

The pages selected to be updated for this experiment:

The results

The initial data was taken during the four weeks from December 29, 2017 through January 25th, 2018. The “after” data was taken just over a month after the work was completed, covering the period of March 6 through April 2, 2018.

The results from this experiment were fascinating. Of all of the SEO experiments we’ve done, the results of this one were the most consistently positive.

Landing Page Unique Pageviews Organic Searches Entrances Hits
/en-us/docs/web/api/document_object_model/locating_dom_elements_using_selectors +46.38% +23.68% +29.33% +59.35%
/en-us/docs/web/api/htmlcollection/item +52.89% +35.71% +53.60% +38.56%
/en-us/docs/web/api/htmlformelement/elements +69.14% +57.83% +69.30% +70.74%
/en-us/docs/web/api/mediadevices/enumeratedevices +23.55% +14.53% +16.71% +15.67%
/en-us/docs/web/api/xmlhttprequest/html_in_xmlhttprequest +49.93% -3.50% +24.67% +59.63%
/en-us/docs/web/api/xmlserializer +36.24% +46.94% +31.50% +37.66%
/en-us/docs/web/css/all +46.15% +16.52% +23.51% +48.28%
/en-us/docs/web/css/inherit +22.55% +27.16% +20.58% +17.12%
/en-us/docs/web/css/object-position +102.78% +119.01% +105.56% +405.52%
/en-us/docs/web/css/unset +24.60% +18.45% +19.20% +35.01%

These results are amazing. Every single value is up, with the sole exception of a small decline in organic search views (that is, appearances in Google search result lists) for the article “HTML in XMLHttpRequest.” Most values are up substantially, with many being impressively improved.

Note: The data in the table above was updated on April 12, 2018 after realizing the “before” data set was inadvertently one day shorter than the “after” set. This reduced the improvements marginally, but did not affect the overall results.

Uncertainties

Due to the implementation of the experiment and certain timing issues, there are uncertainties surrounding these results. Those include:

  • Ideally, much more time would have elapsed between completing the changes and collecting final data.
  • The experiment began during the winter holiday season, when overall site usage is at a low point.
  • There was overall site growth of MDN traffic over the time this experiment was underway.

Decisions

Certain conclusions can be reached:

  1. The degree to which internal link improvements benefited traffic to these pages can’t be ignored, even after factoring in the uncertainties. This is easily the most benefit we got from any experiment, and on top of that, the amount of work required was often much lower. This should be a high priority portion of our SEO plans.
  2. The MDN meta-documentation will be further improved to enhance recommendations around linking among pages on MDN.
  3. We should consider enhancements to macros used to build links to make them easier to use, especially in cases where we commonly have to override default behavior to get the desired output. Simplifying the use of macros to create links will make linking faster and easier and therefore more common.
  4. We’ll re-evaluate the data after more time has passed to ensure the results are correct.

Discussion?

If you’d like to comment on this, or ask questions about the results or the work involved in making these changes, please feel free to follow up or comment on the thread I’ve created on our Discourse forum.

 Posted by at 10:08 AM
Mar 232018
 

The next SEO experiment I’d like to discuss results for is the MDN “Competitive Content Analysis” experiment. In this experiment, performed through December into early January, involved selecting two of the top search terms that resulted in MDN being included in search results—one of them where MDN is highly-placed but not at #1, and one where MDN is listed far down in the search results despite having good content available.

The result is a comparison of the quality of our content and our SEO against other sites that document these technology areas. With that information in hand, we can look at the competition’s content and make decisions as to what changes to make to MDN to help bring us up in the search rankings.

The two keywords we selected:

  • “tr”: For this term, we were the #2 result on Google, behind w3schools.
  • html colors”: For this keyword, we were in 27th place. That’s terrible!

These are terms we should be highly placed for. We have a number of pages that should serve as good destinations for these keywords. The job of this experiment: to try to make that be the case.

The content updates

For each of the two keywords, the goal of the experiment was to improve our page rank for the keywords in question; at least one MDN page should be near or at the top of the results list. This means that for each keyword, we need to choose a preferred “optimum” destination as well as any other pages that might make sense for that keyword (especially if it’s combined with other keywords).

To accomplish that involves updating the content of each of those pages to make sure they’re as good as possible, but also to improve the content of pages that link to the pages that should show up on search results. The goal is to improve the relevant pages’ visibility to search as well as their content quality, in order to improve page position in the search results.

Things to look for

So, for each page that should be linked to the target pages, as well as the target pages themselves, these things need to be evaluated and improved as needed:

  • Add appropriate links back and forth between each page and the target pages.
  • Is the content clear and thorough?
  • Make sure there’s interactive content, such as new interactive examples.
  • Ensure the page’s layout and content hierarchy is up-to-date with our current content structure guidelines.
  • Examine web analytics data to determine what improvements the data suggest should be done beyond these items.

Pages reviewed and/or updated for “tr”

The primary page, obviously, is this one in the HTML element reference:

These pages are closely related and were also reviewed and in most cases updated (sometimes extensively) as part of the experiment:

A secondary group of pages which I felt to be a lower priority to change but still wound up reviewing and in many cases updating:

Pages reviewed and/or updated for “html colors”

This one is interesting in that “html colors” doesn’t directly correlate to a specific page as easily. However, we selected the following pages to update and test:

The problem with this keyword, “html colors”, is that generally what people really want is CSS colors, so you have to try to encourage Google to route people to stuff in the CSS documentation instead of elsewhere. This involves ensuring that you refer to HTML specifically in each page in appropriate ways.

I’ve opted in general to consider the CSS <color> value page to be the destination for this, for reference purposes, with the article “Applying color” being a new one I created to use as a landing page for all things color related to route people to useful guide pages.

The results

As was the case with previous experiments, we only allowed about 60 days for the Google to pick up and fully internalize the changes as well as for user reactions to affect the outcome, despite the fact that 90 days is usually the minimum time you run these tests for, with six months being preferred. However, we have had to compress our schedule for the experiments. We will, as before, continue to watch the results over time.

Results for the “tr” keyword

The pages updated to improve their placement when the “tr” keyword is used in Google search, as well as the amount of change over time seen for each, is shown in the table below. These were the pages which were updated and which appeared in search results analysis for the selected period of time.

Change (%)

Address

Impressions

Clicks Position CTR
HTML/Element/tr -43.22% 124.57% 2.58% 285.71%
HTML/Element/table 26.68% 27.02% -2.90% 0.00%
HTML/Element/template 27.02% 9.21% -15.45% -14.05%
API/HTMLTableRowElement
API/HTMLTableRowElement/insertCell -2.78% -23.91% -2.16% -21.77%
API/HTMLTableRowElement/rowIndex
HTML/Element/thead 38.82% 19.70% 0.00% -13.67%
HTML/Element/tbody 42.72% 100.52% 14.19% 40.68%
HTML/Element/tfoot 8.90% 11.29% 2.64% 2.18%
HTML/Element/th -50.32% 3.43% 0.39% 106.25%
HTML/Element/td 20.05% 40.27% -8.04% 17.01%
API/HTMLTableElement/rows

The data is interesting. Impression counts are generally up, as are clicks and search engine results page (SERP) position. Interestingly, the main <tr> page, the primary page for this keyword, has lost impressions yet gained clicks, with the CTR skyrocketing by a sizeable 285%. This means that people are seeing better search results when searching just for “tr”, and getting right to that page more often than before we began.

Results for the “html colors” keyword

The table below shows the pages updated for the “html colors” keyword and the amount of change seen in the Google activity for each page.

Page Change Impressions (%) Change Clicks (%) Change Position (Absolute) Change Position (%) Change CTR (%)
https://developer.mozilla.org/en-US/docs/Learn/Accessibility/CSS_and_JavaScript +28.61% +19.88% -0.99 -4.54% -6.78%
https://developer.mozilla.org/en-US/docs/Learn/CSS/Styling_boxes/Backgrounds -43.88% -38.17% -2.71 -21.20% +10.17%
https://developer.mozilla.org/en-US/docs/Learn/CSS/Styling_boxes/Borders +51.59% +33.33% +3.28 +48.62% -12.04%
https://developer.mozilla.org/en-US/docs/Learn/CSS/Styling_text/Fundamentals +34.87% +29.55% -1.35 -11.34% -3.94%
https://developer.mozilla.org/en-US/docs/Web/CSS/background-color +9.03% +19.26% -0.17 -2.46% +9.38%
https://developer.mozilla.org/en-US/docs/Web/CSS/border-color +36.02% +36.98% -0.09 -1.38% +0.71%
https://developer.mozilla.org/en-US/docs/Web/CSS/color +23.04% +23.42% +0.03 +0.34% +0.31%
https://developer.mozilla.org/en-US/docs/Web/CSS/color_value +14.95% +34.09% -1.21 -10.26% +16.65%
https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Colors/Color_picker_tool -10.78% +6.68% +1.76 +24.78% +19.56%
https://developer.mozilla.org/en-US/docs/Web/CSS/outline-color +830.70% +773.91% -0.97 -12.42% -6.10%
https://developer.mozilla.org/en-US/docs/Web/CSS/text-decoration-color +3254.57% +3429.41% -1.45 -21.98% +5.21%
https://developer.mozilla.org/en-US/docs/Web/HTML/Applying_color +50.32% +45.21% -0.56 -4.83% -3.40%
https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/color +31.15% +25.57% -0.44 -4.44% -4.25%

These results are also quite promising, especially since time did not permit me to make as many changes to this content as I’d have liked. The changes for the color value type page are good; nearly 15% increase in impressions and a very good 34% rise in clicks means a health boost to CTR. Ironically, though, our position in search results dropped by nearly 1.25 points., or 10%.

The approximate 23% increase in both impressions and clicks on the CSS color attribute is quite good, and I’m pleased by the 10% gain in CTR for the learning area article on styling box backgrounds.

Almost every page sees significant gains in both impressions and clicks (take a look at text-decoration-color, in particular, with over 3000% growth!).

The sea of red is worrisome at first glance, but I think what’s happening here is that because of the improvements in impression counts (that is, how often users see these pages on Google), they are prone to reaching the page they really want more quickly. Note which pages are the ones with the positive click-through rate (CTR), which is the ratio of clicks divided by impressions. This is in order of highest change in CTR to lowest:

  1. https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Colors/Color_picker_tool
  2. https://developer.mozilla.org/en-US/docs/Web/CSS/color_value
  3. https://developer.mozilla.org/en-US/docs/Learn/CSS/Styling_boxes/Backgrounds
  4. https://developer.mozilla.org/en-US/docs/Web/CSS/background-color
  5. https://developer.mozilla.org/en-US/docs/Web/CSS/text-decoration-color
  6. https://developer.mozilla.org/en-US/docs/Web/CSS/border-color
  7. https://developer.mozilla.org/en-US/docs/Web/CSS/color

What I believe we’re seeing is this: due to the improvements to SEO (and potentially other factors), all of the color-related pages are getting more traffic. However, the ones in the list above are the ones seeing the most benefit; they’re less prone to showing up at inappropriate times and more likely to be clicked when they are presented to the user. This is a good sign.

Over time, I would hope to improve the SEO further to help bring the search results positions up for these pages, but that takes a lot more time than we’ve given these pages so far.

Uncertainties

For this experiment, the known uncertainties (an oxymoron, but we’ll go with that term anyway) include:

  • As before, the elapsed time was far too share to get viable data for this experiment. We will examine the data again in a few months to see how things are progressing.
  • This project had additional time constraints that led me not to make as many changes as I might have preferred, especially for the “html colors” keyword. The results may have been significantly different had more time been available, but that’s going to be common in real-world work anyway.
  • Overall site growth during the time we ran this experiment also likely inflated the results somewhat.

Decisions

After sharing these results with Kadir and Chris, we came to the following initial conclusions:

  • This is promising, and should be pursued for pages which already have low-to-moderate traffic.
  • Regardless of when we begin general work to perform and make changes as a result of competitive content analysis, we should immediately update MDN’s contributor guides to incorporate recommended changes.
  • The results suggest that content analysis should be a high-priority part of our SEO toolbox. Increasing our internal link coverage and making documents relate to each other creates a better environment for search engine crawlers to accumulate good data.
  • We’ll re-evaluate the results in a few months after more data has accumulated.

If you have questions or comments about this experiment or its results, please feel free to post to this topic on Mozilla’s Discourse forum.

 Posted by at 4:46 PM
Mar 212018
 

Following in the footsteps of MDN’s “Thin Pages” SEO experiment done in the autumn of 2017, we completed a study to test the effectiveness and process behind making changes to correct cases in which pages are perceived as “duplicates” by search engines. In SEO parlance, “duplicate” is a fuzzy thing. It doesn’t mean the pages are identical—this is actually pretty rare on MDN in particular—but that the pages are similar enough that they are not easily differentiated by the search engine’s crawling technology.

This can happen when two pages are relatively short but are about a similar topic, or on reference pages which are structurally and content-wise quite similar to one another. From a search engine’s standpoint, if you create articles about the background-position and background-origin CSS attributes, you have two pages based on the same template (a CSS reference page) whose contents are easily prone to being identical. This is especially true given how common it is to start a new page by copying content from another, similar, page and making the needed changes, and sometimes even only the barest minimum changes needed.

All that means that from an SEO perspective, we actually have tons of so-called “duplicate” pages on MDN. As before, we selected a number of pages that were identified as duplicates by our SEO contractor and made appropriate changes to them per their recommendations, which we’ll get into briefly in a moment.

Once the changes were made, we waited a while, then compared the before and after analytics to see what the effects were.

The content updates

We wound up choosing nine pages to update for this experiment. We actually started with ten but one of them wound up being removed from the set after the fact when I realized it wasn’t a usable candidate. Unsurprisingly, essentially all duplicate pages were found in reference documentation. Guides and tutorials were with almost no exceptions immune to the phenomenon.

The pages were scattered through various areas of the open web reference documentation under https://developer.mozilla.org/docs/Web.

Things to fix or improve

Each page was altered as much as possible in an attempt to differentiate it from the pages which were found to be similar. Tactics to accomplish this included:

  • Make sure the page is complete. If it’s a stub, write the page’s contents up completely, including all relevant information, sections, etc.
  • Ensure all content on the page is accurate. Look out for places where copy-and-pasted information wasn’t corrected as well as any other errors.
  • Make sure the page has appropriate in-content (as opposed to sidebar or “See also”) links to other pages on MDN. Feel free to also include links in the “See also” section, but in-content links are a must. At least the first mention of any term, API, element, property, attribute, technology, or idea should have a link to relevant documentation (or to an entry in MDN’s Glossary). Sidebar links are not generally counted when indexing web sites, so relying on them entirely for navigation will wreak utter havoc on SEO value.
  • If there are places in which the content can be expanded upon in a sensible way, do so. Are there details not mentioned or not covered in as much depth as they could be? Think about the content from the perspective of the first-time reader, or a newcomer to the technology in question. Will they be able to answer all possible questions they might have by reading this page or pages that the page links to through direct, in-content links?
  • What does the article assume the reader knows that they might not yet? Add the missing information. Anything that’s skimmed over, leaving out details and assuming the reader can figure it out from context needs to be fleshed out with more information.
  • Ensure that all of the API’s features are covered by examples. Ideally, every attribute on an element, keyword for a CSS property’s value, parameter for a method, and so forth should be used in at least one example.
  • Ensure that each example is accompanied by descriptive text. There should be an introduction explaining what the example does and what feature or features of the technology or API are demonstrated as well as any text that explains how the example works. For example, on an HTML element reference page, simply listing the properties then providing an example that only uses a subset of those properties isn’t enough.  Add more examples that cover the other properties, or at least the ones that are likely to be used by anyone who isn’t a deeply advanced user.
  • Do not simply add repetitive or unhelpful text. Beefing up the word count just to try to differentiate the page from other content is actually going to do more harm than good.
  • It’s frequently helpful to also change the pages which are considered to be duplicate pages. By making different changes to each of the similar pages, you can create more variations than by changing one page alone.

Pages to be updated

The pages we selected to update:

  • https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/fillText
  • https://developer.mozilla.org/en-US/docs/Web/API/Window/pageYOffset
  • https://developer.mozilla.org/en-US/docs/Web/API/MouseEvent/pageX
  • https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/getUniformLocation
  • https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/enableVertexAttribArray
  • https://developer.mozilla.org/en-US/docs/Web/API/GlobalEventHandlers/onpointerdown
  • https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/x1
  • https://developer.mozilla.org/en-US/docs/Web/CSS/max-block-size
  • https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackSupportedConstraints/frameRate

In most if not all of these cases, the pages which are similar are obvious.

The results

After making appropriate changes to the pages listed above as well as certain other pages which were similar to them, we allowed 60 days to pass. This is less than the ideal 90 days, or, better, six months, but time was short. We will check the data again in a few months to see how things change given more time.

The changes made were not always as extensive as would normally be preferred, again due to time constraints created by the one-person experimental model. When we do this work on a larger scale, it will be done by a larger community of contributors. In addition, much of this work will be done from the beginning of the writing process as we will be revising our contributor guide to incorporate the recommendations as appropriate after these experiments are complete.

Pre-experiment unvisited pages

As was the case with the “thin pages” experiment, pages which were getting literally zero—or even close to zero— traffic before the changes continued to not get much traffic. Those pages were:

  • https://developer.mozilla.org/en-US/docs/Web/API/GlobalEventHandlers/onpointerdown
  • https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/x1
  • https://developer.mozilla.org/en-US/docs/Web/CSS/max-block-size
  • https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackSupportedConstraints/frameRate

For what it’s worth, we did learn from this eventually and experiments that were begun after the “thin pages” results were in no longer included pages which got no traffic during the period leading up to the start of the experiment, but this experiment had already begun running by then.

Post-experiment unvisited pages

There were also two pages which had a small amount of traffic before the changes were made but no traffic afterward. This is likely a statistical anomaly or fluctuation, so we’re discounting these pages for now:

  • https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/getUniformLocation
  • https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/enableVertexAttribArray

The outcome

The remaining three pages had useful data. This is a small data set but is what we have, so let’s take a look.

 Page URL Sept. 16-Oct. 16 Nov. 18 – Dec 18
Clicks Impressions Clicks Impressions Clicks Chg. % Impressions Chg. %
https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/fillText 678 8,751 862 13,435 27.14% 34.86%
https://developer.mozilla.org/en-US/docs/Web/API/Window/pageYOffset 1,193 3,724 1,395 7,700 14.48% 51.64%
https://developer.mozilla.org/en-US/docs/Web/API/MouseEvent/pageX 447 2,961 706 7,906 36.69% 62.55%

For each of these three pages, the results are promising. Both click and impression counts are up for each of them, with click counts increasing by anywhere from 14% to 36% and impression counts increasing between 34% and 62% (yes, I rounded down for each of these values, since this is pretty rough data anyway). We’ll check the results again soon and see if the results changed further.

Uncertainties

Because of certain implementation specifics of this experiment, there are obviously some uncertainties in the results:

  • We didn’t allow as much time as is recommended for the changes to fully take effect in search data before measuring the results. This was due to time constraints for the experiment being performed, but, as previously mentioned, we’ll look at the data again later to double-check our results.
  • The set of pages with useful results was very small, and even the original set of pages was fairly small.
  • There was substantial overall site growth during this period, so it’s likely the results are affected by this. However, the size of the improvements seen here suggests that even with that in mind, the outcome was significant.

Decisions

After a team review of these results, we came to some conclusions. We’ll revisit them later, of course, if we decide that a review of the data later suggests changes be made.

  1. The amount of improvement seen strongly suggests we should prioritize fixing duplicate pages, at least in cases where at least one of the pages which are considered duplicates of one another is getting at least a low-to-moderate amount of traffic.
  2. The MDN meta-documentation, in particular the writer’s guide and the guides to writing each of the types of reference content, will be updated to incorporate the recommendations into the general guidelines for contributing to MDN. Additionally, the article on MDN about writing SEO-friendly content will be updated to include this information.
  3. It turns out that many of the changes needed to fix “thin” pages also applies when fixing “duplicate” pages.
  4. We’ll re-evaluate prioritization after reviewing the latest data after more time has passed.

The most interesting thing I’m learning about SEO, I think, is that it’s really about making great content. If the content is really top-notch, SEO practically attends to itself. It’s all about having thorough, accurate content with solid internal and external links.

Discussion?

If you have comments or questions about this experiment or the changes we’ll be making, please feel free to follow-up or comment on this thread on Mozilla’s Discourse site.

 Posted by at 7:02 PM
Nov 132017
 

Last week, I wrote about the results of our “thin pages” (that is, pages too short to be properly cataloged by search engines) SEO experiment, in which we found that while there appear to be gains in some cases when improving the pages considered to be too short, there was too much uncertainty and too few cases in which gains seemed to occur at all, to justify making a full-fledged effort to fix every thin page on MDN.

However, we do want to try to avoid thin pages going forward! Having content that people can actually find is obviously important. In addition, we encourage contributors working on articles for other reasons who find that they’re too short to go ahead and update them.

I’ve already updated our meta-documentation (that is, our documentation about writing documentation) to incorporate most of the recommendations for avoiding thin content. These changes are primarily in the writing style guide. I’ve also written the initial portions of a separate guide to writing for SEO on MDN.

For fun, let’s review the basics here today!

What’s a thin page?

A thin page is a page that’s too short for search engines to properly catalog and differentiate from other pages. Pages that are shorter than 250-300 words of content text do not provide enough context for search algorithms to reliably comprehend what the article is about, which means the page winds up not in the right place in search results.

For the purposes of computing the length of an article, the article’s length is the number of words of body content—that is, content that isn’t in headers, footers, sidebars, or similar constructs—plus the number of words located in alt text on <img> elements.

How to avoid thin pages

These tips are taken straight from the guidelines on MDN:

  • Keep an eye on the convenient word counter located in the top-right corner of the editor’s toolbar on MDN.
  • Obviously, if the article is a stub, or is missing material, add it. We try to avoid outright “stub” pages on MDN, although they do exist, but there are plenty of pages that are missing large portions of their content while not technically being a “stub.”
  • Generally review the page to ensure that it’s structured properly for the type of page it is. Be sure every section that it should have is present and has appropriate content.
  • Make sure every section is complete and up-to-date, with no information missing. Are all parameters listed and explained?
  • Be sure everything is fully fleshed-out. It’s easy to give a quick explanation of something, but make sure that all the nuances are covered. Are there special cases? Known restrictions that the reader might need to know about?
  • There should be examples covering all parameters or at least common sets of parameters. Each example should be preceded with an overview of what the example will do, what additional knowledge might be needed to understand it, and so forth. After the example (or interspersed among pieces of the example) should be text explaining how the code works. Don’t skimp on the details and the handling of errors in examples; readers will copy and paste your example to use in their own projects, and your code will wind up used on production sites! See Code examples and our Code sample guidelines for more useful information.
  • If there are particularly common use cases for the feature being described, talk about them! Instead of assuming the reader will figure out that the method being documented can be used to solve a common development problem, actually add a section with an example and text explaining how the example works.
  • Include proper alt text on all images and diagrams; this text counts, as do captions on tables and other figures.
  • Do not descend into adding repetitive, unhelpful material or blobs of keywords, in an attempt to improve the page’s size and search ranking. This does more harm than good, both to content readability and to our search results.

Reviewing the above guidelines and suggestions (some of which are admittedly pretty obvious) when confronted with pages that are just too short may help kick-start your creativity so you can write the needed material to ensure that MDN’s content drifts to the top of the turbid sea of web documentation and other content to be found on the Internet.

 Posted by at 12:50 PM
Nov 082017
 

The MDN team has been working on a number of experiments designed to make decisions around prioritization of various forms of SEO problems we should strive to resolve. In this document, we will examine the results of our first such experiment, the “Thin Pages” experiment.

The goal of this experiment was to select a number of pages on MDN which are considered “thin”—that is, too short to be usefully analyzed—and update them using guidelines provided by our SEO contractor.

Once the changes were made and a suitable time had passed, we re-evaluated the pages’ statistics to determine whether or not the changes had an appreciable effect. With that information in hand, we then made a determination as to whether or not prioritizing this work makes sense.

The content updates

We selected 20 pages, choosing from across the /Web and /Learn areas of MDN and across the spectrum of usage levels. Some pages had little to no traffic at the outset, while others were heavily trafficked. Then, I went through these pages and updated them substantially, adding new content and modernizing their layouts in order to bring them up to a more useful size.

The changes were mostly common-sense ones:

  • Pages that aren’t necessary were deleted (as it turns out, none of the pages we selected were in this category).
  • Ensuring each page had all of the sections they’re meant to have.
  • Ensuring that every aspect of the topic is covered fully.
  • Ensuring that examples are included and cover an appropriate set of cases.
  • Ensuring that all examples include complete explanations of what they’re doing and how they work.
  • Ensuring that pages include not only the standard tags, but additional tags that may add useful keywords to the page.
  • Fleshing out ideas that aren’t fully covered.

The pages we chose to update are:

The results

After making the changes we were able to make in a reasonable amount of time, we then allowed the pages some time to percolate through Google’s crawler and such. Then we re-measured the impression and click counts, and the results were not quite what we expected.

First, of all the pages involved, only a few actually got any search traffic at all. The following pages were not seen by users searching on Google at all during either/or the starting or ending analysis:

The remaining pages did generally see measurable gains, some of them quite high, but none are clearly outside the range of growth expected giving MDN’s ongoing growth:

June 1-30 Sept. 24 – Oct. 23
Page URL Clicks Impressions Clicks Impressions Clicks Chg. % Impressions Chg. %
https://developer.mozilla.org/en-US/docs/Web/CSS/Media_Queries 15 112 111 2600 640.00% 2221.43%
https://developer.mozilla.org/en-US/docs/Web/CSS/transform-function/translateZ 1789 6331 1866 9004 4.30% 42.22%
https://developer.mozilla.org/en-US/docs/Web/HTML/Inline_elements 3151 60793 4729 100457 50.08% 65.24%

This is unfortunately not a very large data set, but we can draw some crude results from it. We’ll also continue to watch these pages over the next few months to see if there’s any further change.

The number of impressions went up, in some cases dramatically. But there’s just not enough here to be sure this was related to the thin page revisions or related to other factors, such as the large-scale improvements to the HTML docs recently made.

Uncertainties

There are, as mentioned already, some uncertainties around these results:

  • The number of pages that had useful results was smaller than we would have preferred.
  • We had substantial overall site growth during the same time period, and certain areas of the site were heavily overhauled. Both of these facts may have impacted the results.
  • We only gave the pages a couple of months after making the changes before measuring the results. We were advised that six months is a more helpful time period to monitor (so we’ll look again in a few months).

Decisions

After reviewing these results, and weighing the lack of solid data at this stage, we did come to some initial conclusions, which are open to review if the numbers change going forward:

  1. We won’t launch a full-scale formal project around fixing thin pages. It’s just not worth it given the dodginess of the numbers we have thus far.
  2. We will, however, update the meta-documentation to incorporate the recommendations around thin pages.That means providing advice about the kinds of content to include, reminding people to be thorough, reminding writers to include plenty of samples that cover a variety of use cases and situations, and so forth. We will also add a new “SEO” area to the meta docs that covers these recommendations more explicitly in terms of the SEO impact.
  3. We will check these numbers again in a couple of months to see if there’s been further improvement. The recommendation was to wait six months for results, but we did not have that kind of time.

Discussion?

For discussion of this experiment, and of the work updating MDN that will come from it, I encourage you to follow-up or comment in this thread on the Mozilla Discourse site.

 Posted by at 11:22 AM
May 122017
 

I’ve been writing developer documentation for 20 years now, 11 of those years at Mozilla. For most of those years, documentation work was largely unmanaged. That is to say, we had management, and we had goals, but how we reached those goals was entirely up to us. This worked well for me in particular. My brain is like a simple maze bot in some respects, following all the left turns until it reaches a dead end, then backing up to where it made the last turn and taking the next path to the right, and repeating until the goal has been reached.

This is how I wrote for a good 14 or 15 years of my career. I’d start writing about a topic, linking to APIs, functions, other guides and tutorials, and so forth along the way—whether they already existed or not. Then I’d go back through the page and click the first link on the page I just created, and I’d make sure that that page was solid. Any material on that page that needed to be fixed for my new work to be 100% understood, I’d update. If there were any broken links, I’d fix them, creating and writing new pages as needed, and so forth.

How my mind wants to do it

Let’s imagine that the standards gurus have spoken and have decided to add to a new <dial> element to HTML, providing support for creating knobs and speedometer-style feedback displays. My job is to document this element.

I start by creating the main article in the HTML reference for <dial>, and I write that material, starting with a summary (which may include references to <progress>, <input>, and other elements and pages). It may also include links to articles I plan to create, such as “Using dial elements” and “Displaying information in HTML” as well as articles on forms.

As I continue, I may wind up with links to subpages which need to be created; I’ll also wind up with a link to the documentation for the HTMLDialElement interface, which obviously hasn’t been written yet. I also will have links to subpages of that, as well as perhaps for other elements’ attributes and methods.

Having finished the document for <dial>, I save it, review it and clean it up, then I start following all the links on the page. Any links that take me to a page that needs to be written, I write it. Any links that take me to a page that needs content added because of the new element, I expand them. Any links that take me to a page that is just horribly unusably bad, I update or rewrite as needed. And I continue to follow those left-hand turns, writing or updating article after article, until eventually I wind up back where I started.

If one of those pages is missing an example, odds are good it’ll be hard to resist creating one, although if it will take more than a few minutes, this is where I’m likely to reluctantly flag it for someone else to do later, unless it’s really interesting and I am just that intrigued.

By the time I’m done documenting <dial>, I may also have updated badly out of date documentation for three other elements and their interfaces, written pages about how to decide on the best way to represent your data, added documentation for another undocumented element that has nothing to do with anything but it was a dead link I saw along the way, updated another element’s documentation because that page was where I happened to go to look at the correct way to structure something, and I saw it had layout problems…

You get the idea.

How I have to do it now

Unfortunately, I can’t realistically do that anymore. We have adopted a system of sprints with planned work for each sprint. Failing to complete the work in the expected amount of time tends to get you dirty looks from more and more people the longer it goes on. Even though I’m getting a ton accomplished, it doesn’t count if it’s not on the sprint plan.

So I try to force myself to work on only the stuff directly related to the sprint we’re doing. But sometimes the line is hard to find. If I add documentation for an interface, but the documentation for its parent interface is terrible, it seems to me that updating that parent interface is a fairly obvious part of my job for the sprint. But it wasn’t budgeted into the time available, so if I do it, I’m not going to finish in time.

The conundrum

That leaves me in a bind: do strictly what I’m supposed to do, leaving behind docs that are only partly usable, or follow at least some of those links into pages that need help before the new content is truly usable and “complete,” but risk missing my expected schedule.

I almost always choose the latter, going in knowing I’m going to be late because of it. I try to control my tendency to keep making left turns, but sometimes I lose myself in the work and write stuff I am not expected to be doing right now.

Worse, though, is that the effort of restraining myself to just writing what’s expected is unnatural to me. My brain rebels a bit, and I’m quite sure my overall throughput is somewhat lower because of it. As a result: a less enjoyable writing experience for me, less overall content created, and unmet goals.

I wonder, sometimes, how my work results would look if I were able to cut loose and just go again. I know I have other issues slowing me down (see my earlier blog post Peripheral neuropathy and me), but I can’t help wondering if I could be more productive by working how I think, instead of doing what doesn’t come naturally: work on a single thing from A to Z without any deviation at all for any reason.

 Posted by at 10:30 AM
Nov 132015
 

I’m going to highlight a meeting for you today. This is the point where you yawn politely, look at the time, and and try to escape without my noticing. But I see you over there! Get back here. This is important!

Each Thursday, the MDN content team holds its weekly API documentation meeting at 8 AM Pacific time in Mozilla’s Open the DevEngage Vidyo room. This meeting is for discussions about ongoing and upcoming work on documentation for all Web APIs. This includes the classic DOM as well as all newer APIs, from Ambient Light to Speech Synthesis and beyond. It even includes Firefox OS-specific APIs. We don’t even discriminate against non-standard APIs, as long as they’re exposed to browser content.

That’s a lot of stuff to cover! Everything needs to be understood, written about, sample code located or created (and tested!), and all tied together and reviewed until it makes sense and is as accurate as we can make it.

That’s why we have been holding these meetings in collaboration with the API development team for a long. A few months ago, the technical evangelism team also started sending a representative to each meeting. This tripartite meeting lets each team share recent accomplishments and what they’ll be doing next. This has multiple benefits:

  • The writers learn what new technologies are being implemented, what improvements are in the works, and when things are likely to ship. We also learn when special events are coming that would benefit from having documentation ready.
  • The technical evangelists get details on what new APIs are coming up, and can discuss plans for spreading the word with the developers creating the APIs and the writers documenting them, to coordinate plans and schedules.
  • The technical evangelists can relay user sentiment information in a more personal way to both the development teams and the writers; this kind of feedback is incredibly helpful!
  • The development team can let the writers and evangelists know what the status is on current API work, and we can discuss this status in a team setting instead of only reading about it in a formal note or bug comments.
  • The developers can share information about what problem points they see or expect to exist in understanding and working with technologies, in order to help guide future work in samples, demos, and documentation.

There are intangible benefits, too. Over the two-plus years we’ve been holding this weekly meeting, we’ve developed an increasingly close working relationship between the developer documentation and the API engineering teams. This has enormous benefits not just for these two teams, but for the Web we serve.

If you have a passion for creating APIs for the Web or for teaching others how to use them, please consider joining our meeting. Even if you only drop in once in a while, you’ll find it a great way to stay informed and to help guide the future of our content and evangelism efforts.

 Posted by at 6:07 PM
Jun 142014
 

I’m happy with how my talk at the Open Help conference went this morning. I gave a talk about how Mozilla builds, fosters, and provides useful information to our documentation community. I enjoyed giving it, and the questions were good ones.

I saw a lot of note-taking and nodding heads, and the discussions over the rest of the day definitely gave me the sense that people appreciated my thoughts on documentation community-building and processes. I hope there will be video available at some point; certainly it was recorded.

Our documentation status pages and other ways we present data about tasks that need to be performed really got a lot of attention, and Florian’s graph of our contributor growth really opened some eyes — especially when I pointed to the sharp uptick since we deployed our in-house grown, open source, Kuma documentation platform.

Also, I think our development team will be pleased that I heard a lot of “I wish we were using Kuma for our docs” today, and at least one person saying they’re going to look at setting up to try building it themselves.

Three or four people said that if it were separated from the Mozilla-specific bits, they’d be pushing to switch to Kuma. That made me smile; hopefully they will still feel that way if/when we manage to ever make the platform separate from the MDN-specific parts.

I’m with my people here; it’s really wonderful to just talk about documentation tactics without all the other stuff. It’s a great feeling, especially when it seems you’re able to give your peers ideas to take home and try for themselves.

My slides

View my slides below, or download them as a PDF.

 Posted by at 10:10 PM
Jun 112014
 

I’ll be attending the Open Help Conference & Sprints event this weekend, arriving in Cincinnati on Friday afternoon and returning home on Tuesday morning after the first day of documentation sprinting. Mozilla is a proud sponsor of this event, which features a number of key influencers in the open source documentation arena. The mission: to exchange notes and ideas about how to improve the quality and quantity of good open source documentation, and to hold documentation sprints to get some writing projects done.

I’ll be giving a talk on Saturday morning (at 10 AM—the first presentation of the event) entitled “Help people so they can help you.” I’ll be covering the things the MDN team does to build, foster, and support its growing community of contributors. I’m looking forward to this, even though I’m nowhere near prepared yet. I enjoy sharing ideas about how to create and maintain excellent documentation. It’s my career’s mission, and as a Mozillian, I’m pleased to be able to share. Hopefully, too, I’ll get new ideas this weekend as well!

As an aside, if you have any thoughts on things I should be sure to mention, don’t hesitate to leave a comment or drop me an email!

If you’re going to be at Open Help, or are simply in the Cincinnati area, I hope to run into you!

 Posted by at 9:51 AM
Dec 092013
 

Today we launched the new design for the Mozilla Developer Network Web site. We’ve been working on this for a long time, with discussions starting as much as a year ago, and actual coding starting this past summer. It was a huge effort involving a ton of brilliant people, and I’m thrilled with the results! Not only is the new design more attractive, but it has a number of entirely new features that will help you find what you want—and read it—more easily, and in more places, than ever before.

Updated home page

The new home page has a completely revamped organization, with a large search box front and center. Below that are links to key areas of the MDN Web site, including much of our most-used documentation, such as the JavaScript, CSS, and HTML docs, content about Firefox OS and developer tools, and the brand-new Mozilla Developer Program, whose goal is to help Web developers learn more and more quickly.

MDN home page screenshot

Zoning in

You may notice the new concept we call “zones” in the screenshot above. A “zone” is a new concept we’ve added, wherein we can construct a special topic area that can accumulate documentation and samples from across MDN to cover that topic. For example, the Firefox OS zone provides documentation about Firefox OS, which involves not only Firefox OS-specific content, but information about HTML, CSS, and so forth.

Screenshot of the Firefox OS zone landing page

Here you see our new zone navigation bar along the left side of the screen, promotional boxes for specific content, and lists of key articles.

Search and ye shall find

Search has been significantly improved; we now have an on-site search powered by Elastic Search rather than using Google, which lets us customize the search in useful ways. One particularly helpful feature is the addition of search filters. Once you’ve done a search, you can narrow the search further by using these filters. For example, here’s a search for “window element”:

Screenshot of search resultsWell. 5186 results is a lot. You can use our new search filters, though, to narrow those results down a bit. You can choose to restrict your search to one or more topic areas, types of document, and/or skill level. Let’s look for articles specifically about the DOM:

Screenshot of search resultsIt’s worth noting here that these filters rely on content being tagged properly, and much of our content is still in the process of being tagged (especially regarding skill level). This is something we can always use help with, so please drop into #mdn on IRC if you’re interested in helping with this quick and easy way to help improve our search quality!

Responsive design

An area in which MDN was sorely lacking in the past was responsive design. This is the concept of designing content to adapt to the device on which it’s being used. The new MDN design makes key adjustments to its layout based on the size of your screen.

 Screenshot of the Firefox zone landing page in "desktop" view.  Screenshot of the Firefox zone landing page in "small desktop" view. Screenshot of the Firefox zone landing page in "mobile" view.
Here’s the Firefox zone’s landing page in standard “desktop” mode. This is what you’ll see browsing to it in a typical desktop browser environment. Here’s what the same page looks like in “small desktop” mode. This is what the page looks like when your browser window is smaller, such as when viewing on a netbook. And here’s the same page in “mobile” view. The page’s layout is adjusted to be friendlier on a mobile device.

Each view mode rearranges the layout, and in some cases removes less important page elements, to improve the page’s utility in that environment.

Quicklinks

The last new feature I’ll point out (although not the last improvement by a long shot!) is the new quicklink feature. We now have the ability to add a collection of quicklinks to pages; this can be done manually by building the list while editing the page, or by using macros.

Here’s a screenshot of the quicklinks area on the page for the CSS background property:

Quicklinks in the CSS reference.

The quicklinks in the CSS reference provide fast access to related properties; when looking at a background-related property, as seen above, you get quick access to all of the background-related properties at once.

There’s also an expandable “CSS Reference” section. Clicking it gives you an alphabetical list of all of the CSS reference pages:

CSS reference page with expanded links to other pages in the reference.

As you see, this lets you quickly navigate through the entire CSS reference without having to backtrack to the CSS landing page. I think this will come as an enormous relief to a lot of MDN users!

To top it off, if you want to have more room for content and don’t need the quicklinks, you can hide them by simply clicking the “Hide sidebar” button at the top of the left column; this results in something like the following:

CSS reference page with the left sidebar closed.The quicklinks feature is rapidly becoming my favorite feature of this new MDN look-and-feel. Not all of our content is making full use of it yet, but we’re rapidly expanding its use. It makes navigating content so much easier, and is easy to work with as a content creator on MDN, too.

Next steps

Our development team and the design and UX teams did a fantastic job building this platform, and our community of writers threw in their share as well: between testing the changes to providing feedback, not to mention contributing and updating enormous amounts of documentation to take advantage of new features and to look right in the new design, I’m enormously proud of everyone involved.

There’s plenty left to do. There are new platform features yet to be built, and the content always needs more work. If you’d like to help, drop into #mdn to talk about content or #mdndev to talk about helping with the platform itself. And feel free to file bugs with your suggestions and input.

See you online!

 Posted by at 3:12 PM