#ocTEL wek four, TEL One: e-assessment

  • Why did/would you choose a particular type of e-assessment? Describe why you think it is effective and how it can help deepen knowledge and understanding.
  • In your experience, what type of approach creates an environment conducive to self-directed learning, peer support and collaborative learning? How might technology help?
  • What opportunities and challenges does this approach present to tutors?

In a large (350+ students) first-year class, we traditionally used in-class oral presentations as one of the assignments. Students presented individually, in their 25-student tutorials, for about 5 minutes each. They were assessed in class by their tutor. Teaching staff went through the assessment criteria in detail with the students before the assignment. The ideal was that students would develop their presentation skills, and that students would learn from each other by listening to each others’ presentations. It also aligned with an institutional policy which stipulates that markers can only be paid to mark two assessment items outside of class time: using an oral presentation marked in class means that a 12-week unit can comprise three, rather than only two, assessment items. A challenge for tutors and coordinators was that because presentations were live, there was no way to return to them to discuss them with the students for feedback or in the event of a contested grade. The reality, as opposed to the earlier ideal stated, was that students felt little motivation to improve their presentation skills because the audience for their presentations was small, and because that audience had no bearing on the mark. The other reality was that students did not learn from each other because they became quickly bored with sitting there listening to presentation after presentation on the same topics; many students did not attend tutorial unless they were presenting in it.

To buy back tutorial time (getting through 25 student presentations in one-hour tutorials usually took at least two tutorials), to increase students’ motivation to produce quality presentations, to teach the students valuable digital literacy skills, and to create an accessible archive of presentations, I changed the format of the presentations from in-class oral to YouTube videos. Assignment instructions were the same as for the original assignment, and students were clearly informed that they would not be assessed on their videography skills. Videos were to be head-to-cam only, with no incorporation of visuals, text, or effects. This was so that students maintained their focus on the assignment content and on their own presentation of the material. The video presentations were essentially the same as the in-class presentations in format. An additional learning session was added to the unit in which students were taught how to make and upload a YouTube video; surprisingly few students knew how to do this. For privacy reasons, students did not have to use their real names when they uploaded videos; they used their students numbers and the unit code as the video title and tag. They submitted the assignment by submitting the YouTube URL through Blackboard.

The shift to video was designed to build students’ digital literacy, to increase their motivation to present well by making their presentations more public, and to build an achive of presentations for reuse as exemplars. The shift also allowed markers to return to individual assignments to discuss them with students. It achieved these results.

However, I don’t think the shift to video presentations adequately deepened students’ learning, peer support, or collaboration. Feedback remained very much “the sole responsibility of the practitioner [tutor].” To further improve the assignment, I’ll build in an activity in class in which students work together to assess existing videos using the assessment criteria. I could also set up a discussion forum in the week before the assignment is due in which students can post their draft videos for peer feedback; students who provide quality feedback will earn extra marks on the assignment. The latter activity will mean more work in marking for the tutors, who will have to go through the forums to mark comments posted there, as well as marking each student’s final video.

Suggestions on how to make this peer commenting activity/process efficient and effective?

Advertisements

#ocTEL Explorer activity 3.5: evaluating a resource in your area

I thought that this was going to be what ocTEL was all about: Finding The Stuff. ocTEL has, however, proved significantly more useful and substantial. TEL isn’t all about The Stuff: it’s about understanding how The Stuff fits into broader teaching and learning strategies, understanding how learning works, understanding technical issues around using The Stuff (Cable Green), and clarifying intended learning outcomes so that choosing TEL applications/platforms (ie “The Stuff”) is a focussed and finite task. That said, this week we turned to the dauntingly vast  sea of The Stuff to find “a resource in an area which is important in your teaching.”

The subject we teach in my department–Entertainment producing–is specific and reaonably novel, and quick searches around the subject didn’t turn up much on the provided resource banks. That’s not a problem. My teaching team are great at the content knowledge–they know this material and this subject very well–more useful would be finding resources in areas in which we are not experts, such as, for example, critical reflection. Our final-year students in particular undertake significant critical reflection tasks, but none of us is an expert in the area. Specialised staff time is beter spent engaging students with the core subject material. So I decided to search for critical reflection in the resources, looking for material that could support the students in their reflective practice and assessment activities.

Jorum generated three unique links, however, each of them was focussed on a specific discipline: for example, “This resource is aimed primarily at students in healthcare practice to get them to reflect on their experiences and practice.” While it would be possible for me to embed this material into an Entertainment Producing course, saying to students, “when the site says ‘health care professionals’, mentally replace that with ‘entertainment producers’,” this would dimish the material’s credibility, and possibly in students’ minds, its relevance to them. The site at which I looked is CC BY-NC, so I could select only its pages which refer to critical reflection in a generic mode; I would have considered doing this if the resource had been productively interactive rather than straightforwardly informational. As other ocTEL participants have discussed in the forums in relation to this activity, in embedding external material into a university course, care needs to be taken so that it does not appear to students as if the lecturer is simply cutting and pasting simple material found online into the course. There is a fine perceptual line between lecturer as high-quality curator, and lecturer as lazy copier; understanding where this line is is based on each lecturer’s individual understanding of the teaching and learning context and cohort. Understanding this line is a key part of deciding “whether a resource is worth adapting” (ocTEL activity 3.5 question). For this reason, although Jorum generated a couple of reasonably relevant resources, I wouldn’t use either.

Under “critical reflection,” Merlot only generated a couple of discipline-specific items.

I struck gold with LearnHigher, specifically with its Critical Reflection and Thinking area.  The material is CC BY-NC, so resuable and adaptable. The activity is particularly useful.

A YouTube search for “critical reflection” generated around 2100 items, 69 of which were CC licenced. Of the latter, the majority were videos of students from other disciplines doing critical reflection presentations: not particularly useful, especially because we don’t see what marks the students earned for them. This means they can’t easily be used as exemplars. Because YouTube provides only videos, any material found here is necessarily instructional only, rather than interactive or activity-based. I found a couple of videos that I would consider including in a list of resources about critical reflection for students. Including a number of instructional resources about ‘how to do critical reflection’ could be productive in that providing students with a variety of voices and media improves the odds of connecting with students from a variety of learning styles. This was the first time I’d used YouTube’s search filtering option: it was helpful.

This activity left me with a research project. One of the activity’s questions is “what criteria did you use to evaluate this resource.” One of the criteria is my own subject expertise; another is my understanding of the courses we teach and how the material might articulate into them. However, there is another key criterion, and this is students’ consumer and cultural preferences. By consumer preferences, I mean students’ perception that they’ve invested in coming to this university for the type of education it advertises: at what point do they feel that they’re not experiencing this, and that they’re instead experiencing a cobbled-together hogde-podge of external content? Do students see the lecturer as a skilled curator, or as a locus of knowledge? I recently researched whether students in my faculty preferred online teaching and learning videos featuring their own lecturers, or videos from Lynda.com (publication forthcoming: delayed by ocTEL participation!). The students preferred the in-house material, despite its somewhat basic production values. They preferred hearing Australian accents, and felt that the in-house material was “more relevant” to their studies. Culturally, they feel somewhat resistant to US and UK material. They liked the “more relaxed” style of the in-house material, which connected culturally with Australians’ self-perception as laid-back and classless. This was one piece of research. More research is required into specific cohorts of students’ perceptions of external content.

This ocTEL activity pointed up the fact that the bulk of OER is from the US or the UK: some might see this as an example of “the spectre of pedagogocial imperialism in the virtual classroom.” This is a perceptual, subjective issue. The only way to understand it is to research one’s own students. Using OER activities, particularly if I adapt them myself, seems a safer bet than using vidoes, which necessarily signal their cultural provenance.

Activity key outcomes:

1. conduct further research with my department’s students to gauge their perceptions of external vs in-house videos, learning activities, content.

2. focus my OER searches on teaching and learning activities rather than videos.

 

#ocTEL Activity 2.4(b): design an authentic learning activity

This activity involves designing an online, authentic learning activity. It was only after I designed the activity that I re-read the instructions and saw that it was meant to be a “short” activity; this one is designed to last for a full year. Nonetheless…

 

Activity title: “Producing: Practice”

Cohort & rationale: The activity is designed for a final-year university Bachelor’s degree unit in which approximately 10 teams of up to 5 students undertake industry-based projects for a full year. Each team is attached to an individual industry partner with whom it works for the year. The projects are different from internships in that students are working in teams, and they are working semi-autonomously on projects given to them by the industry partner. In this sense, the student project teams work more like consultants or subcontractors than interns. Students are from the Entertainment Industries course, which educates students for intellectual careers in the entertainment sector, working as entertainment Producers. This activity is specifically designed to address a problem in the unit. At present, each team goes off and has its own learning experience, but there is very little peer-to-peer learning and support between groups. There is no peer learning from cohort to cohort, particularly because this is a final-year unit and students graduate immediately upon completing it. Although many of the teams are dealing with similar issues, each team’s learning remains fairly discrete.

The activity: Build/update the “Producing: Practice” wiki

Following the “contributing student approach” (Collis & Moonen 2001), and grounded in Kolb’s experiential learning model, this activity involves students sharing and researching their industry project experiences, and then creating (in subsequent years, updating) a wiki of support resources addressing key issues which arise in those experiences.

Weeks 1-2 Equipping: in these weeks, students watch a video of the unit coordinator explaining the assignment. Students are then given links to the various technologies they may use in their assignment work over the year, as well as links about how best to use those technologies. Students will complete one of the “are you ready for online learning” questionnaires (see ocTEL activities week1) to identify areas in which they require development. Students will be required to pass a short online quiz about the technologies before they can continue in the unit.

Weeks 3-10 each student keeps a blog about her/his project experiences. The blog will be visible to other students, but not to the wider public: it will be hosted on the university’s VLE, Blackboard. For reasons of commercial confidence (most student project teams are asked to sign non-disclosure agreements by their industry partners), the blogs will focus more on generic issues such as time management, communication, research, and management styles. To ensure inclusion, blogs can be in written or video format.

Weeks 11-13 As a cohort, the students will be required to define up to six key issues identified by the blogs. How the students do this will be up to them, although there will be a video from the unit coordinator suggesting ways in which group discussion and decisions can be handled. This will cause students some discomfort; as Panos Vlachopoulos observed this week, shifting learning responsibility over to students can cause discomfort for both students and the instructor. Ideally, this task will empower students to take ownership of the project and to begin to self-direct their learning.

Weeks 14-18 each student will contribute to building a wiki: the wiki will become a resource for Project students. The wiki will provide support and information about dealing with issues students face while doing industry Projects. Each student will select one area of the wiki on which to focus. Each student will be required to make “substantial contributions” to the wiki in the form of links to research papers, links to how-to videos and guides, recordings of interviews with students in the unit, and any other relevant online material.

Weeks 19-20. Students edit others’ wiki entries where useful.

Assessment:

  • 30% blog
  • 20% productive professional behaviour
  • 50% contributions to wiki, including editing

Intended learning outcomes:

Students who complete this unit successfully should have developed:

  • advanced research- and peer-informed professional practice
  • an understanding and identification of many of the key professional issues faced by Project students, and strategies for addressing these issues
  • a network of peers for learning and professional development
  • the ability to communicate in a variety of modes

In terms of the “development of skills and literacies needed in professional practice, and which reflect{ } the nature of the discipline” aspect of this ocTEL assignment, the project is reasonably generic in itself: this type of project could be used in any industry placement/project context. However, the resource produced will be highly profession-specific, giving students a sense of the industry-specific aspects of their future profession. How does communication work in the industry? How does time management work? This learning will complement more generic “prepare for your professional future” resources available. The activity builds students’ capacity for professional reflection, and research-informed practice. It scaffolds their becoming networked peer learners, an attribute which will be highly valuable post-graduation. The wiki becomes a reusable learning object that can be updated and drawn upon by subsequent cohorts.

 

Technical issues: Some resourcing will be required to set up the initial template for the wiki: the first cohort of students to undertake the project should not have to design the front end of the wiki. Some moderation may be necessary to ensure client confidentiality is not breeched, and to facilitate discussion. Weekly ‘prompts’ may be emailed to students to suggest questions they might address in their blog posts. Pre-unit preparation will involve filming and editing the online video lectures, building the page of possible technologies and technology support links, and including links to the online learning self-assessment material.

 

Drawback: Ideally, the peer-to-peer learning would take place in closer to real time. With this activity, the wiki is not complete until the end of the semester, when students are completing their projects. I need to consider how I could build a reflecting-in-action (Shon 1983) element into the unit. Ideas, ocTEL?

 

#ocTEL activity week 2 : approaches to learning

Deep, Surface, and Strategic approaches to learning

  • Have you seen any evidence of these different approaches in online contexts, e.g. in technology-enhanced courses you teach? How did these differences manifest themselves in terms of online learning behaviour?
  • Are you leaning towards one approach in particular on ocTEL, and if so why might that be? Perhaps you are employing strategies from more than one approach?
  • Are learners who tend to take a ‘surface’ approach likely to learn more or less effectively online versus face-to-face?
  • How might we encourage ‘deep learning’ in online contexts

While it is “generally accepted” that “deep learning is the ideal,” I think that this ideal needs to be considered before we all tailor all of our teaching efforts towards achieving this type of learning and learner.

The differentiation between deep and surface learning highlights a subjective perspective on the purpose of academic education itself. I’ll look at Biggs and Tang’s (2007) influential discussion of the concepts here as an example. For Biggs and Tang, tertiary education should be about the acquisition of knowledges, skills, and competencies in themselves: in learning for learning’s sake. This is a subjective value proposition. Biggs and Tang are scornful of the idea that tertiary education might focus on equipping students to work as industry professionals. They dismiss “extrinsic motivation,” in which students are motivated to learn based on strategic outcomes such as employability, arguing that this encourages “surface learning” (2007, 35).

Biggs & Tang’s perspective aligns with a deeper ontological divide over the purpose of tertiary education. On one side of this divide are those who see tertiary education as focussed on graduate employability and the application of learning to problems and situations beyond the university; on the other side are those who see tertiary education as focussed on learning for learning’s sake. Leinhardt et al (1995) refer to this distinction as the difference between “university” and “professional” approaches. The former often characterise their approach as “applied,” while the latter often use the value-laden term “pure” to describe their approach (for example see McCalman 2000; Klein 2000). The same can be seen in the semantics of “deep” versus “strategic” as definitional terms. Writing assessment criteria and then condemning students who work hard to address them as somehow less than “ideal” is a contradictory approach, and one that is frustratingly opaque to students.

In my experience, “strategic” learners don’t just want to please the lecturer and earn high marks just for the sake of high marks: they want to earn high marks in order to boost their employability; they want to follow assessment requirements because they have often been told that these requirements and criteria align closely to key professional attributes. “Deep” learners are great in their in-class enthusiasm, but they can miss the broader point of their learning beyond the classroom: its applications.

In one of this week’s readings, Siemens observes that in an information-saturated environment, knowing exactly what one wants and where to find it are key learning skills: these skills align more with a “strategic” learning style.

In asking students to give online comments on each others’ YouTube oral presentations, all three of these learning styles are evident. Thre are students who do the bare minimum, adding nothing of real value to other students’ videos; students who do just enough to ensure they meet the highest assessment criteria, ofering comments that demonstrate their understanding of the unit material; and students who post lengthy responses full of references to things they’ve learned or read in other units, their own thoughts, and their questioning of the terms of the assignment itself (which is precisely what I’m doing now!). In my experience, the students on the receiving end of this feedback find the strategic material most valuable. In peer learning online, strategic learners can be of high value. I know that in group work contexts, students would much rather work with a strategic learner than with a deep one.

In my own ocTEL approach, I’m taking a combination of deep and strategic approaches, which I think is the ideal combination. If I just took a deep approach, I could bog down, trying to read everything and somehow assimilate it all, make connections, follow up references. This is definitely a temptation, and one against which we were rightly warned in week 0. Without a key outcome or strategic application in mind, I could happily drift around in ocTEL for hours, learning all sorts of things. But I’m also taking a strategic approach: I see the value of the assigned activities, and how they structure my learning twoards useful intended outcomes. I have professional purposes for doing ocTEL–improving my department’s teaching performance, and my own–I use this strategic professional application as a way to manage and structure my ocTEL engagement. I find the badges a very productive way of guiding and rewarding focussed, strategic learning.

In terms of surface learners, in my experience they’re surface learners regardless of teaching medium. There are strategies to assist them with their learning, but these don’t usually boil down to “come to class” or “go online.”

Thus, I don’t think “we” should focus on “encouraging deep learning in online contexts.” I’d prefer to encourage a combination of strategic and deep learning because both are equally valid types and outcomes of education.

Biggs, J., and C. Tang. (2007) Teaching for Quality Learning at University. Berkshire, UK: Open University Press.

 

Klein, T. (2000) The value of fundamental inquiry: The view from physics. In Coady, T. (Ed.). Why Universities Matter: A Conversation about Values, Means, and Directions, pp. 99-109. Sydney: Allen & Unwin.

 

Leinhardt, G., et al. (1995) Integrating professional knowledge: The theory of practice and the practice of theory. Learning and Instruction, 5: 401-8.

 

McCalman, J. (2000) Blurred visions. . In Coady, T. (Ed.). Why Universities Matter: A Conversation about Values, Means, and Directions, pp. 132-43. Sydney: Allen & Unwin.

 

#ocTEL week one webinar notes, and badges

Badges!

week-1-check-inweek-1-webinarweek-1-tel-oneweek-1-topic week-1-tel-explorer

Here are some notes from the week one ocTEL webinar:

Kyriaki Anagnostopoulou, University of Bath

-it’s difficult to evaluate learning technologies that are embedded in contexts: how to evaluate them separately from the context?

-learning design as a choreography to the ‘dance’ of learning. At first I thought this seemed prescriptive–“left foot here, right foot here”–until I thought, it depends what kind of dance you’re talking about. First-year choreography is necessarily prescriptive–a waltz–while final-year choreography may be more interpretive dance.

-Ron Oliver’s “tasks, resources, support” model (1999) (Hi Ron!)

-focus on authentic learning: public outputs result in higher quality of student work

-Verpoorten’s 8 learning events model

-Bath Digital Literacies online student self-assessment: student completes self assessment, and is then recommended student and faculty-specific resources to address any challenge areas identified

-technology needs to enhance and add to the learning experience, not to be a clumsy bolt-on

James Little, University of Leeds

-academics feel: “not enought time to catch up!” No real solution offered other than that they should be involved in multiple networks of communicaiton about T&L practice. This doesn’t seem to address the issue, however.

-don’t’ assume the same level of technological ability across the student cohort

-students just want technology to work, not to be an “additional barrier”

-digital resident/visitor rather than native/immigrant concept

-SAMR model: modification and redefinition using technology are transformative; substitution and augmentation using technology are enhancement

 

#ocTEL activity 1.3: Champions and critics of teaching machines

Pick one or two of the following thinkers or approaches and read a bit about them, starting with the resources linked. What would they like about the Teaching Machines approach? What would they oppose, and what alternatives would they propose? Explore the notes made by two or three of your fellow participants. What patterns do you detect?

Socrates.
First, I like that doing a TEL MOOC has me reading up on Socrates. A good sign that pedagogy is at the forefront here.

Before I read the article I had the typical view of Socratic method: the scholar sitting at the feet of the master, with both asking questions of each other–the scholar asking the master for guidance, and the master asking leading questions in order to make the scholar think. I now know that Socratic practice–as opposed to method–also involved browbeating by the master. I’ll segregate method from practice here and focus on the method.

The idea that “Education is not a cramming in, but a drawing out” sits well with this cMOOC, and aligns with learner-centred teaching. The types of activities we’re doing in this MOOC–critical reflection on practices, dialogic practice, comparing one’s responses to others’–exemplify the “drawing out” approach to pedagogy. I’ve wondered before how this type of learning can be promoted/scaffolded by technology; ocTEL is providing an excellent example.

This interpretation of Socratic pedagogy conflicts with the pedagogy of Skinner’s teaching machines. In the Skinner video, there is no dialogue, no chance for students to ask questions, and no possibility of avoidance of the instructor’s “pre-determined views.” ocTEL participant Intelligent Ideation comes to the same conclusions.

Dialogue in the forums on the Skinner-Socrates comparison centres on the useful point that there is a place for both types of learning, and both types of technology. The trick is deploying the right types at the right points in the learning process.