Crowdsourcing/The Wikipedia way/The drive for quality

Whereas the tagline “the free encyclopedia anyone can edit” suggests anarchy, newcomers can be surprised by how constrained and rule-bound the project is. The tagline is correct, but does not convey the evolutionary process that Wikipedia actually employs, preserving encyclopaedic contributions while resisting promotional, crank, or other undesirable influences.

A lot of the activity on Wikipedia is behind the scenes, not part of the encyclopedia itself. This includes policies, style guidelines, noticeboards, discussion pages about improving individual articles, “wikiprojects” to improve a topic area, and discussions about user behaviour. A lot of this is geared towards assessing articles on a detailed quality scale, including processes of informal and formal review. An article’s quality rating, if it has one, is visible on clicking “Talk” at the top of the page on the desktop version of Wikipedia.

The Wikipedia quality scale

Class	Criteria
FA	The article has attained featured article status. Detailed community review process.
GA	The article has attained good article status. Specific review process.
B	The article is mostly complete and without major issues, but requires some further work to reach good article standards.
C	The article is substantial, but is still missing important content or contains a lot of irrelevant material. The article should have references to reliable sources, but may still have significant issues or require substantial cleanup.
Start	The article has a usable amount of good content but is weak in many areas. Quality of the prose may be distinctly unencyclopedic; but the article should satisfy fundamental content policies such as notability, and provide sources to establish verifiability.
Stub	The article is either a very short article or a rough collection of information that will need much work to become a meaningful article. It is usually very short, but if the material is irrelevant or incomprehensible, an article of any length falls into this category.

Review of a Wikipedia article is different from peer-review of an original research paper for a journal. Someone who “reviews a paper” is also reviewing the research that it reports. It takes expert knowledge to infer, from the paper, how the research was conducted, whether it used the best available methods, and hence whether the research is reliable and significant.

By contrast, a Wikipedia article is not a proxy for something else. The article itself, and its relation to its sources, are the target of the review. The reviewer does not have to infer things that happened away from Wikipedia. The reliability of the article comes from its citations (which is why Open Access research, verifiable by a wider audience, is of more use in improving Wikipedia than paywalled research). So while it is useful for reviewers to be aware of the relevant literature and able to use relevant research tools, it is not essential that all reviewing is done by experts.

Whereas a poorly-written paper reporting significant research might be accepted by a journal and then copy-edited, the reviews on Wikipedia examine the wording and layout in fine detail. Every sentence and citation can potentially be challenged. On the way to Featured Article status, an article typically goes through three review processes, involving at least eight reviewers. While Peer Review and Good Article involve one reviewer (or more if the review is contentious), the Featured Article review is a community process typically involving five or six reviewers. Another contributor reviews the FA discussion to assess whether it has reached consensus.

Whether it involves formal review or not, a quality scale is important in helping contributors work together, rather than against each other. The scale has to be flexible enough to adapt to different subjects, since a summary of knowledge about a battleship is different from a summary of knowledge about an art form. At the same time, the scale has to be cashed out in enough detail to guide individual decisions about progress. Tangible models (other encyclopedias in the case of Wikipedia) make this easier.