7 Things You Need to Know about Turnitin GradeMark and PeerMark
A white paper from Teaching and Learning with Technology at Penn State

At a Glance:

The Turnitin suite of writing assessment tools includes

  • A “Quickmarks” tool to add annotated comments and symbols to an assigment
  • Rubric tool for assignments
  • Automated grammar checker
  • Peer critique tool with options to insert questions for students
  • Originality report

Who to contact

Teaching and Learning with Technology – Elizabeth Pyatt

Usage Scenarios

Any course which includes a writing assignment could use these tools, but courses which focus on teaching writing could benefit from students seeing feedback from these tools multiple times through the semester.

In addition to writing assignments, the tools could be used in the following scenarios.

  • Provide students, particularly non-native speakers of English, feedback on English grammar.
  • Foreign language instructors could use customized feedback comments for non-English writing assignments.
  • The rubric could be applied to items posted online if a URL is provided.
  • If images are embedded in a document, they can be graded and critiqued by fellow students.
  • Documents with links to multimedia documents could also be graded and critiqued with the Turnitin tools.
  • The PeerMark tool could be adapted to allow students to critique group projects and presentations (based on uploaded documents).

References

GradeMark and PeerMark research team

Elizabeth Pyatt, Teaching and Learning with Technology
John Dolan, College of the Liberal Arts
Mary Janzen, Teaching and Learning with Technology
Shannon Kennan, College of Communications
Barb Polka Smith, Teaching and Learning with Technology
Lisa Urban, Teaching and Learning with Technology

Additional input from

John Beale, College of Communications
Nicole Chong, Dickinson School of Law
Stevie Rocco, College of Earth and Mineral Sciences
Heather Hughes, Teaching and Learning with Technology
Brian Young, Teaching and Learning with Technology

1. What is it?

Turnitin (https://www.turnitin.com/), maintained by iParadigms, was originally designed as a plagiarism detection service. Institutions or individuals pay a fee to access a database of papers used to compare student assignments for originality. Results of the scan are given as an “Originality Report” which lists found quotes and paper sources.

Today it has expanded to include a suite of writing instruction tools including:

  1. GradeMark: a paper markup and rubric tool.
  2. e-Rater Grammar Check: an automated grammar and spell-checker tool licensed from ETS Princeton.
  3. PeerMark: a student peer critique tool with tools for instructors to determine review criteria for students.

This paper reports the results of an investigation of these three tools in the Penn State environment.

2. Who is using it?

According to Turnitn.com’s About page, over 10,000 educational institutions, both K-12 and higher education, have licensed Turnitin. Example institutions from higher education include:

3. How does it work?

Within Turnitin, instructors create assignments into which papers are uploaded. When GradeMark options are enabled, instructors have several grading options available. Assignments with the e-Rater Grammar Check tool enabled will see the marks on students assignments. Marks which are not appropriate can be dismissed by the instructor.

In addition, instructors can use the  QuickMarks feature, which allows instructors to drag and drop comments onto student papers. Comments can come from a set of standard comments (e.g., “Awk,” “¶”) or be customized to include comments related to the course.

Document with highlighted sentence, comment Awk. and Grammar comments

Document in GradeMark with Awk. comment and eRater comment P/V to warn against “passive voice.”

Question Mark set including Awk., Citatin Needed, delete, Run-On, Sp. (spelling), Weak Transition, Vague, Support, Commonly Confused.

Set of commonly used Question Marks including Awk., Citation Needed, Missing , and others.

GradeMark also allows an instructor to record voice comments and to include a grading rubric. The rubric can customized for a wide range of scales and criteria and allows instructors to calculate an assignment score.

Wikipedia Rurbic, read following caption for more information

Sample rubric for a Wikipedia article. Criteria include # Sections (scored as 3), #Print Cit (4), #Web Cit (3) and Objectivity (4). Note that Objectivity counts for 40% and other criteria for only 10-20% each.

For peer review, instructors can create a PeerMark assignment based on a previously submitted assignment. For PeerMark assignments, instructors can assign questions to students, and determine the number of reviews students do and whether reviews are anonymous or not. Turnitin also includes a library of potential questions to guide students to thinking critically about their critique.

PeerMark distribution options. See caption following for details.

PeerMark distribution set so that students are required to review one paper automatically assigned by PeerMark. Self-review is not checked.

Sample PeerMark questions for students including Rate this paper's overall readability and Does each paragraph logically progress from the former ones? Why or why not?

Sample PeerMark questions for students to answer in their evaluation.

4. Why is it significant?

One of the most pedagogically exciting but most underused tools, according to Turnitin.com, is the PeerMark tool. Not only does this tool allow students to reflect on the process of writing and gain perspective into what their instructor is looking for, but it allows students to receive feedback from multiple sources.

In addition, the knowledge that an assignment is undergoing peer review can motivate students to put more effort into their assignments. It would be interesting to see if the PeerMark tool could be adapted for other peer review activities, including assessment of teammates in group projects.

The QuickMarks commenting tool is also significant in that it is designed to replicate many of the traditional paper markup tools and symbols in a digital format. Within Turnitin, instructors can highlight text in different colors, add standard comments, including those with special symbols like “¶,” and insert custom comments. Not only does this provide an easy feedback mechanism for students, but it could reduce the need for printouts. Another advantage to the QuickMark commenting system is that instructors can quickly review comments for an entire paper in the comment review window and can add audio comments for an entire paper.

5. What are the downsides?

As with any new tool, there are some pedagogical and technical issues to consider when implementing it in the classroom. In terms of pedagogy, the following items are noted.

  1. Many reviewers were concerned that the e-Rater Grammar Check results were often imperfect. For instance, it skips all capitalized words and is unfamiliar with many technical terms. It could also give false positives for some errors. Unfortunately, there is no way to amend either the dictionary or the rules. The tool is probably best used to give students a sense of their own writing patterns or as a grammar review tool.
  2. If PeerMark is implemented as just a way for students to grade each other, it could be misconstrued as a way for instructors to turn over grading to students. Students would need guidance on what to look for and an understanding of how examining other students’ work will improve their own writing process.
  3. In the Penn State environment, most instructors are able to use Turnitin with minimal training. However, if the new modules are introduced, it will be important to ensure that adequate training and support is available to instructors.
    Note: It is especially important that Turnitin not be mandated across programs or departments unless the proper support is available.
  4. Several use cases involving review and peer critique of images and other non-text files have arisen and Turnitin only supports a relatively small set of text-based file types. Although it is possible to link to or embed images in a file, the twenty-word minimum may still be a barrier for these grading scenarios.
  5. Turnitin does not fully support anonymous grading by instructors as one set of instructors requested. Anonymous grading would allow instructors to grade assignments without bias, but ideally students would be able to see the feedback.

In addition to the pedagogical issues above, it is important to consider these technical issues.

  1. Due to the agreement between Penn State and the National Federation for the Blind, the accessibility of the tool set was assessed. Turnitin has made efforts to enhance accessibility, but there were were key features that neither a blind student nor a student relying on a keyboard would be able to access. In addition, deaf and hearing impaired students would not be able to access recorded comments without a transcript. It would be important to advise instructors how to handle these issues in the event a student needing accommodation were participating in their courses.  
  2. Currently, Turnitin is not integrated into a learning management system, although the option is available to institutions for a fee. It would be recommended that integration into a learning management system be considered in order to streamline issues of grading and student roster management.

6. What are the implications for teaching and learning?

The whole Turnitin suite in general has the potential to provide students with multiple channels of feedback to help them improve their writing, a skill which is always in high demand in the professional workplace. Other tools, particularly the rubric tool, may be replicated elsewhere, but the Turnitin suite integrates these tools together in a single environment.

Further, by expanding the Turnitin tools beyond the originality report, instructors can begin to view Turnitin as a writing education tool which could help students improve their writing rather than a policing tool. An originality report could be seen not just as “proof” of cheating but as a way to show students where to insert citations and what work is needed to improve the quality of their writing to a point beyond parroting what they have read. With this perspective, instructors wishing to use the grading tools may also reconsider the value of the originality report in that context.

7. Where is it going?

The tool has been implemented in a large number of schools and is a mature tool fully supported by the Turnitin service. The tools include a number of pedagogically sound options for instructors, but as with all tools, instructors need to have a good understanding of what those options are. If this tool set is adopted by Penn State, then it will be important to help instructors understand the best methods for using each tool. This can be done by developing usage case studies and explaining best practices for rubrics, peer grading, and the e-Rater Grammar Check tool.

Tagged with →  
Share →
Skip to toolbar