#ocTEL wek four, TEL One: e-assessment

  • Why did/would you choose a particular type of e-assessment? Describe why you think it is effective and how it can help deepen knowledge and understanding.
  • In your experience, what type of approach creates an environment conducive to self-directed learning, peer support and collaborative learning? How might technology help?
  • What opportunities and challenges does this approach present to tutors?

In a large (350+ students) first-year class, we traditionally used in-class oral presentations as one of the assignments. Students presented individually, in their 25-student tutorials, for about 5 minutes each. They were assessed in class by their tutor. Teaching staff went through the assessment criteria in detail with the students before the assignment. The ideal was that students would develop their presentation skills, and that students would learn from each other by listening to each others’ presentations. It also aligned with an institutional policy which stipulates that markers can only be paid to mark two assessment items outside of class time: using an oral presentation marked in class means that a 12-week unit can comprise three, rather than only two, assessment items. A challenge for tutors and coordinators was that because presentations were live, there was no way to return to them to discuss them with the students for feedback or in the event of a contested grade. The reality, as opposed to the earlier ideal stated, was that students felt little motivation to improve their presentation skills because the audience for their presentations was small, and because that audience had no bearing on the mark. The other reality was that students did not learn from each other because they became quickly bored with sitting there listening to presentation after presentation on the same topics; many students did not attend tutorial unless they were presenting in it.

To buy back tutorial time (getting through 25 student presentations in one-hour tutorials usually took at least two tutorials), to increase students’ motivation to produce quality presentations, to teach the students valuable digital literacy skills, and to create an accessible archive of presentations, I changed the format of the presentations from in-class oral to YouTube videos. Assignment instructions were the same as for the original assignment, and students were clearly informed that they would not be assessed on their videography skills. Videos were to be head-to-cam only, with no incorporation of visuals, text, or effects. This was so that students maintained their focus on the assignment content and on their own presentation of the material. The video presentations were essentially the same as the in-class presentations in format. An additional learning session was added to the unit in which students were taught how to make and upload a YouTube video; surprisingly few students knew how to do this. For privacy reasons, students did not have to use their real names when they uploaded videos; they used their students numbers and the unit code as the video title and tag. They submitted the assignment by submitting the YouTube URL through Blackboard.

The shift to video was designed to build students’ digital literacy, to increase their motivation to present well by making their presentations more public, and to build an achive of presentations for reuse as exemplars. The shift also allowed markers to return to individual assignments to discuss them with students. It achieved these results.

However, I don’t think the shift to video presentations adequately deepened students’ learning, peer support, or collaboration. Feedback remained very much “the sole responsibility of the practitioner [tutor].” To further improve the assignment, I’ll build in an activity in class in which students work together to assess existing videos using the assessment criteria. I could also set up a discussion forum in the week before the assignment is due in which students can post their draft videos for peer feedback; students who provide quality feedback will earn extra marks on the assignment. The latter activity will mean more work in marking for the tutors, who will have to go through the forums to mark comments posted there, as well as marking each student’s final video.

Suggestions on how to make this peer commenting activity/process efficient and effective?

Advertisements

#ocTEL Explorer activity 3.5: evaluating a resource in your area

I thought that this was going to be what ocTEL was all about: Finding The Stuff. ocTEL has, however, proved significantly more useful and substantial. TEL isn’t all about The Stuff: it’s about understanding how The Stuff fits into broader teaching and learning strategies, understanding how learning works, understanding technical issues around using The Stuff (Cable Green), and clarifying intended learning outcomes so that choosing TEL applications/platforms (ie “The Stuff”) is a focussed and finite task. That said, this week we turned to the dauntingly vast¬† sea of The Stuff to find “a resource in an area which is important in your teaching.”

The subject we teach in my department–Entertainment producing–is specific and reaonably novel, and quick searches around the subject didn’t turn up much on the provided resource banks. That’s not a problem. My teaching team are great at the content knowledge–they know this material and this subject very well–more useful would be finding resources in areas in which we are not experts, such as, for example, critical reflection. Our final-year students in particular undertake significant critical reflection tasks, but none of us is an expert in the area. Specialised staff time is beter spent engaging students with the core subject material. So I decided to search for critical reflection in the resources, looking for material that could support the students in their reflective practice and assessment activities.

Jorum generated three unique links, however, each of them was focussed on a specific discipline: for example, “This resource is aimed primarily at students in healthcare practice to get them to reflect on their experiences and practice.” While it would be possible for me to embed this material into an Entertainment Producing course, saying to students, “when the site says ‘health care professionals’, mentally replace that with ‘entertainment producers’,” this would dimish the material’s credibility, and possibly in students’ minds, its relevance to them. The site at which I looked is CC BY-NC, so I could select only its pages which refer to critical reflection in a generic mode; I would have considered doing this if the resource had been productively interactive rather than straightforwardly informational. As other ocTEL participants have discussed in the forums in relation to this activity, in embedding external material into a university course, care needs to be taken so that it does not appear to students as if the lecturer is simply cutting and pasting simple material found online into the course. There is a fine perceptual line between lecturer as high-quality curator, and lecturer as lazy copier; understanding where this line is is based on each lecturer’s individual understanding of the teaching and learning context and cohort. Understanding this line is a key part of deciding “whether a resource is worth adapting” (ocTEL activity 3.5 question). For this reason, although Jorum generated a couple of reasonably relevant resources, I wouldn’t use either.

Under “critical reflection,” Merlot only generated a couple of discipline-specific items.

I struck gold with LearnHigher, specifically with its Critical Reflection and Thinking area.  The material is CC BY-NC, so resuable and adaptable. The activity is particularly useful.

A YouTube search for “critical reflection” generated around 2100 items, 69 of which were CC licenced. Of the latter, the majority were videos of students from other disciplines doing critical reflection presentations: not particularly useful, especially because we don’t see what marks the students earned for them. This means they can’t easily be used as exemplars. Because YouTube provides only videos, any material found here is necessarily instructional only, rather than interactive or activity-based. I found a couple of videos that I would consider including in a list of resources about critical reflection for students. Including a number of instructional resources about ‘how to do critical reflection’ could be productive in that providing students with a variety of voices and media improves the odds of connecting with students from a variety of learning styles. This was the first time I’d used YouTube’s search filtering option: it was helpful.

This activity left me with a research project. One of the activity’s questions is “what criteria did you use to evaluate this resource.” One of the criteria is my own subject expertise; another is my understanding of the courses we teach and how the material might articulate into them. However, there is another key criterion, and this is students’ consumer and cultural preferences. By consumer preferences, I mean students’ perception that they’ve invested in coming to this university for the type of education it advertises: at what point do they feel that they’re not experiencing this, and that they’re instead experiencing a cobbled-together hogde-podge of external content? Do students see the lecturer as a skilled curator, or as a locus of knowledge? I recently researched whether students in my faculty preferred online teaching and learning videos featuring their own lecturers, or videos from Lynda.com (publication forthcoming: delayed by ocTEL participation!). The students preferred the in-house material, despite its somewhat basic production values. They preferred hearing Australian accents, and felt that the in-house material was “more relevant” to their studies. Culturally, they feel somewhat resistant to US and UK material. They liked the “more relaxed” style of the in-house material, which connected culturally with Australians’ self-perception as laid-back and classless. This was one piece of research. More research is required into specific cohorts of students’ perceptions of external content.

This ocTEL activity pointed up the fact that the bulk of OER is from the US or the UK: some might see this as an example of “the spectre of pedagogocial imperialism in the virtual classroom.” This is a perceptual, subjective issue. The only way to understand it is to research one’s own students. Using OER activities, particularly if I adapt them myself, seems a safer bet than using vidoes, which necessarily signal their cultural provenance.

Activity key outcomes:

1. conduct further research with my department’s students to gauge their perceptions of external vs in-house videos, learning activities, content.

2. focus my OER searches on teaching and learning activities rather than videos.