Nov 082017
 

The MDN team has been working on a number of experiments designed to make decisions around prioritization of various forms of SEO problems we should strive to resolve. In this document, we will examine the results of our first such experiment, the “Thin Pages” experiment.

The goal of this experiment was to select a number of pages on MDN which are considered “thin”—that is, too short to be usefully analyzed—and update them using guidelines provided by our SEO contractor.

Once the changes were made and a suitable time had passed, we re-evaluated the pages’ statistics to determine whether or not the changes had an appreciable effect. With that information in hand, we then made a determination as to whether or not prioritizing this work makes sense.

The content updates

We selected 20 pages, choosing from across the /Web and /Learn areas of MDN and across the spectrum of usage levels. Some pages had little to no traffic at the outset, while others were heavily trafficked. Then, I went through these pages and updated them substantially, adding new content and modernizing their layouts in order to bring them up to a more useful size.

The changes were mostly common-sense ones:

  • Pages that aren’t necessary were deleted (as it turns out, none of the pages we selected were in this category).
  • Ensuring each page had all of the sections they’re meant to have.
  • Ensuring that every aspect of the topic is covered fully.
  • Ensuring that examples are included and cover an appropriate set of cases.
  • Ensuring that all examples include complete explanations of what they’re doing and how they work.
  • Ensuring that pages include not only the standard tags, but additional tags that may add useful keywords to the page.
  • Fleshing out ideas that aren’t fully covered.

The pages we chose to update are:

The results

After making the changes we were able to make in a reasonable amount of time, we then allowed the pages some time to percolate through Google’s crawler and such. Then we re-measured the impression and click counts, and the results were not quite what we expected.

First, of all the pages involved, only a few actually got any search traffic at all. The following pages were not seen by users searching on Google at all during either/or the starting or ending analysis:

The remaining pages did generally see measurable gains, some of them quite high, but none are clearly outside the range of growth expected giving MDN’s ongoing growth:

June 1-30 Sept. 24 – Oct. 23
Page URL Clicks Impressions Clicks Impressions Clicks Chg. % Impressions Chg. %
https://developer.mozilla.org/en-US/docs/Web/CSS/Media_Queries 15 112 111 2600 640.00% 2221.43%
https://developer.mozilla.org/en-US/docs/Web/CSS/transform-function/translateZ 1789 6331 1866 9004 4.30% 42.22%
https://developer.mozilla.org/en-US/docs/Web/HTML/Inline_elements 3151 60793 4729 100457 50.08% 65.24%

This is unfortunately not a very large data set, but we can draw some crude results from it. We’ll also continue to watch these pages over the next few months to see if there’s any further change.

The number of impressions went up, in some cases dramatically. But there’s just not enough here to be sure this was related to the thin page revisions or related to other factors, such as the large-scale improvements to the HTML docs recently made.

Uncertainties

There are, as mentioned already, some uncertainties around these results:

  • The number of pages that had useful results was smaller than we would have preferred.
  • We had substantial overall site growth during the same time period, and certain areas of the site were heavily overhauled. Both of these facts may have impacted the results.
  • We only gave the pages a couple of months after making the changes before measuring the results. We were advised that six months is a more helpful time period to monitor (so we’ll look again in a few months).

Decisions

After reviewing these results, and weighing the lack of solid data at this stage, we did come to some initial conclusions, which are open to review if the numbers change going forward:

  1. We won’t launch a full-scale formal project around fixing thin pages. It’s just not worth it given the dodginess of the numbers we have thus far.
  2. We will, however, update the meta-documentation to incorporate the recommendations around thin pages.That means providing advice about the kinds of content to include, reminding people to be thorough, reminding writers to include plenty of samples that cover a variety of use cases and situations, and so forth. We will also add a new “SEO” area to the meta docs that covers these recommendations more explicitly in terms of the SEO impact.
  3. We will check these numbers again in a couple of months to see if there’s been further improvement. The recommendation was to wait six months for results, but we did not have that kind of time.

Discussion?

For discussion of this experiment, and of the work updating MDN that will come from it, I encourage you to follow-up or comment in this thread on the Mozilla Discourse site.

 Posted by at 11:22 AM

Sorry, the comment form is closed at this time.