Wiki Pedagogy/Evaluation Tools

Wiki worth

The value that wikis may hold for educational contexts, through the efforts of the many over time, has not yet been established. Surowieck has suggested that large groups of people are smarter than an elite few, no matter how brilliant; groups are better at solving problems, fostering innovation, coming to wise decisions and even predicting the future. This "Wisdom of crowds" is evinced in the ever-increasing quantity and quality of WikiProjectEndeavours, both inside and outside of education. See, for example, the WikiMedia Foundation Projects (such as Wiktionary, WikiNews, and WikispecieS), EvoWiki (a reader-built encyclopedia of evolution, biology, and origins), SEEK (Science Environment for Ecological Knowledge), Wikibooks (a collection of open-content textbooks that anyone can edit), Wikiversity (a free, open learning environment and research community), OOPS (An Opensource Opencourseware Prototype System for lecture translation at MIT), and the WikiTravelGuide. The important criteria of project durability remain to be seen. But if the Linux project is any indication of Open Project potential, these typically virtual enactments may prove to be powerful and sustainable creative forces.

Before looking at some specific evaluations of Wiki use in higher education, the following is a brief summary of the literature regarding what Wikis may do best^[1].

Wikis may work best for...

knowledge building "over time" (through versions and groups);
progressive problem-solving (particularly open-ended problems, e.g. Brereton et al, 2003) and even problem redefinition (Scardamalia et al., 1994). For example, Wikis could work well for COP (communities of practice) whose goal is to develop solutions to common problems over time in order to improve practice (Godwin-Jones, 2003);
explaining increasingly diverse and contrary ideas, as well as examining the relatedness of ideas from diverse contexts (Scardamalia et al, 1994);
combining, synthesizing and evaluating definitions and terminology across disciplines (Fountain, 2005c; Scardamalia et al, 1994; Brereton et al., 2003);
questioning underlying causes and principles^[2];
critically reading, and responding in a constructive and public way, to others' work^[3];
learning how to add both nuance and complexity to concepts in a given field, through systematic engagement and analysis with work produced by more advanced students, specialists and experts (Fountain, 2005c; Brereton et al., 2003); and
learning to observe deeply, stereotype less, and avoid premature judgment (Brereton et al., 2003).

Wiki project evaluations

Detailed evaluation of specific wiki use^[4] in educational contexts is embedded in a number of papers:

CoWeb Catalog (description of classroom activities that students and teachers have invented for the CoWeb at Georgia Tech);
Other CoWeb papers;
A pilot project for a biology class;
How wikis didn't work (report on a six-week workshop on visual design in web development for non-experts);
Wikipedia as participatory journalism.

While the evaluation of each of the aforementioned pedagogical use of wikis is intimately tied to specific contexts (domain requirements, course goals, students' confidence and competency levels, Internet access, time and timeline allotments, etc.), there are several general points of evaluation:

Just bringing in a new tool does not change practice. Co-elaboration and cooperation will not simply occur because wikis are introduced to one's practice.
Content knowledge can be improved, but this takes time and does not work where individuality is upheld.
Using new tools in place of other tools works, but it is not the best use of wiki space and potential (i.e. using wikis as overheads).
Quality of language can be maintained — or even improved — if versions ready for quality assessment are identified. Students do not want to think about language questions until they are ready; they want to exchange freely, then present the information well.
Open authoring (autonomy and non-authority) does not necessarily lead to the destruction, modification or copying of others' work.
Co-authorship is not a problem for the students if the guidelines for evaluation are clear.

While the following example is not taken from higher education, it is extremely relevant, and may be indicative of how such "problems" get resolved. Grade six students in an elementary school devised their own publishing protocol in a public environment (a blog). They created their own (French version) language work ethic and icon for "how" they wished to create their texts as well as "when" they were ready for their texts to be evaluated grammatically. It is important to note that this quality of language question ("mistakes-gone-public") was of such significance to both teachers and parents that the entire project almost closed down. It was the students themselves, who took the initiative to create and enforce a language policy so that they could continue to publish online. Some personal wiki evaluations

Though analysis is still underway, I would like to offer some preliminary comments pertaining to the wiki research work conducted in my graduate and undergraduate technology courses at Laval^[5] as well as in our SSRHC project, "Technoscientific literacies via open source enabled virtual research collectives"^[6].

Preliminary analysis across these two research projects indicates that the exclusive use of wiki software (asychronous, text-based technology) to conduct research across digitally-based virtual communities, within which participants did not know each other, complicated student psychosocial relationships. These included frequent misunderstandings due to the lack of virtual and audio clues; the organization of projects was also cumbersome, and synchronous chats were eventually added to address this issue. Yet in spite of these difficulties, students reported that they liked working with wikis. They appreciated the simplicity, the autonomy of access, the ability to share work with others and the ability to view others' work^[7].

Analysis of the following issues is presently underway. If you are interested in being informed about ensuing articles on these research lines, please write to Renee.Fountain@fse.ulaval.ca.

the quality of student articulations^[8] as evinced in students' concept elaborations and "rationality islands"^[9];
the role students attributed to knowledge — deemed technoscientific or otherwise — in their own and others' discourse;
the ways in which students responded to, negotiated and (re)produced knowledge differences; and
their capacities for deliberative, community-inclusive interventions.

End notes

↑ Guidelines for effective pedagogical wiki integration are located at the end of the Pedagogical Potential section.
↑ Collaborative renderings may ease the difficulty of learning to identify assumptions, querying attributions of causality as well as imaging implications and consequences associated therein (Scardamalia et al, 1994).
↑ If the ideas are complex (which is arguably to be hoped for), students could begin by pointing out what is hard to understand and/or the inadequacies of explanations to date (Scardamalia et al., 1994).
↑ Again, is it important to note that given the relatively recent emergence of wiki technology, the following evaluations have not occurred over long time periods.
↑ highly recommend Judith Horman's masters thesis (2005) to those interested in the social interactions that do — and do not — take place in these "forced" digital collectives.
↑ The SSRCH project was conducted by researchers Jacques Daignault (UQAR, Lévis campus), Jacques Désautels, Marie Larochelle, and myself (Laval University). I wish to thank the Social Sciences and Humanities Research Council of Canada for its support in funding this wiki research.
↑ However, the task of publicly critiquing others work proved to be psychologically difficult (afraid to hurt or be hurt) and frustrating (offering, but not receiving quality critique) for many. For a detailed analysis of the psychosocial factors in undergraduate wiki coursework, see Horman (2005).
↑ Questions pertaining to the quantity of text produced — i.e. whether students wrote more and more often — are hard to gauge, as individual contributions to text versions were not noted. However, if individual contributions were important, these could be foregrounded in small collectives wherein each person would write in a specific colour. As was previously noted in the Pedagogy section, chronological version analysis, while rich, is extremely time-consuming, as changes in each version are not indicated. This could, however, be used to enable students to increase their analytical and reading observation skills.
↑ A pedagogical approach used in complex decision-making developed by Fourez (description in English) and Fourez (description in French).

[1] Guidelines for effective pedagogical wiki integration are located at the end of the Pedagogical Potential section.

[2] Collaborative renderings may ease the difficulty of learning to identify assumptions, querying attributions of causality as well as imaging implications and consequences associated therein (Scardamalia et al, 1994).

[3] If the ideas are complex (which is arguably to be hoped for), students could begin by pointing out what is hard to understand and/or the inadequacies of explanations to date (Scardamalia et al., 1994).

[4] Again, is it important to note that given the relatively recent emergence of wiki technology, the following evaluations have not occurred over long time periods.

[5] y recommend Judith Horman's masters thesis (2005) to those interested in the social interactions that do — and do not — take place in these "forced" digital collectives.

[6] The SSRCH project was conducted by researchers Jacques Daignault (UQAR, Lévis campus), Jacques Désautels, Marie Larochelle, and myself (Laval University). I wish to thank the Social Sciences and Humanities Research Council of Canada for its support in funding this wiki research.

[7] However, the task of publicly critiquing others work proved to be psychologically difficult (afraid to hurt or be hurt) and frustrating (offering, but not receiving quality critique) for many. For a detailed analysis of the psychosocial factors in undergraduate wiki coursework, see Horman (2005).

[8] Questions pertaining to the quantity of text produced — i.e. whether students wrote more and more often — are hard to gauge, as individual contributions to text versions were not noted. However, if individual contributions were important, these could be foregrounded in small collectives wherein each person would write in a specific colour. As was previously noted in the Pedagogy section, chronological version analysis, while rich, is extremely time-consuming, as changes in each version are not indicated. This could, however, be used to enable students to increase their analytical and reading observation skills.

[9] A pedagogical approach used in complex decision-making developed by Fourez (description in English) and Fourez (description in French).

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]