ONLINE TUTORING e-BOOK - Chapter 4 -New Assessment Strategies

em Ensino a Distância

Mhairi McAlpine and Carol Higgison 

1 Introduction

"Good assessment of students' knowledge, skills and abilities is absolutely crucial to the process of learning." (Brown, 1999)
Assessment and feedback are integral to the learning process and, as Erwin and Knight (1995) note, they have a significant impact on what students learn.
"If all other elements of the course point in one direction and the assessment arrangements in another, then the assessment arrangements are likely to have the greatest influence on the understood curriculum." (Erwin and Knight 1995)
Therefore if, as Gibb (1999) asserts, "assessment is the most powerful lever teachers have to influence the way students respond to courses and behave as learners" then it is crucial that we use appropriate and effective methods of assessment to ensure effective learning.
"assessment methods and requirements probably have a greater influence on how and what students learn than any other single factor." (Boud, 1988)
Online environments offer new ways of learning (Chapter 1), make new demands on the tutors' knowledge and skills (Chapter 2) and they offer us the opportunity to develop new methods of assessment that can promote learning that is more effective:
"The possibility that innovative assessment encourages students to take a deep approach to their learning and foster intrinsic interest in their studies is widely welcomed." (McDowell, 1996)
The OTiS e-Workshop brought together experienced online tutors to share their experiences and reflect on their practice. It has produced a rich and detailed picture of how tutors have adapted and changed their strategies and methods of assessing students' learning online, and providing appropriate, effective and timely feedback.

Although we draw primarily on contributions to the OTiS e-Workshop we have also drawn on materials from UK projects on computer-assisted assessment and some published literature to set the context. In particular we refer to materials published by the Scottish Computer-Assisted Assessment Network (SCAAN

http://www.scaan.ac.uk) and by the UK Computer Assisted Assessment Centre (http://www.caacentre.ac.uk).

The first part of this chapter, Assessment Issues (sections 2 and 3), sets the background for online assessment and feedback, and provides pointers to resources on computer-assisted assessment.

The second part of this chapter, Assessment in Practice (sections 4-7), describes current innovative practice in assessment which exploits the potential for collaborative learning offered by the online environment. It is divided into four sections: Relationships, Quality Evaluation, Aspects of Development and Format and Structure of Assessment, based on the content of the case study reports and the discussion.



 

 2 Assessment and Feedback

This part of 'New Assessment Strategies' examines some of the issues that must be considered when exploiting the potential for new methods of assessment offered by the online environment.
 

2.1 Is what we are doing really different?

"I think what we need to do is to decide first to what extent these new online courses are revolutionary in pedagogy. How different are these courses from f to f [face-to-face] pbl [problem based learning] or project courses? Does the technology really make a difference, and where, exactly? Maybe the lessons for assessment have already been learnt?" (Macdonald-D 2000b)

2.2 How does technology allow us to rethink what we are doing in?

"The big question though is: How can we rethink what we already know about assessment to see how new technologies allow us new ways of doing things?" (Spratt-D 2000a)
As Finkelstein suggests, technology can also influence the way students learn and we need to take account of this in our assessment strategy:
"The course I taught was in the area of cultural studies, cultural and personal identity. I had noticed a tendency of students in my courses to start using the internet as a major research and information source (almost to the exclusion of printed texts at times!)…
"…So in response to that, it seemed to me that an assessment strategy was needed to play on this, and turn such activity (which at times was done unreflectively and without consideration of the accuracy, weight or reliability of net resources) into a reflective exercise." (Finkelstein-D 2000c)
Eger and Vacek consider some of these issues in planning their assessment strategy:
"Web based courses offer the possibility of implementation of online tests with online assessment. This feature can be with advantage used not only for the supervision of students progress, but also for guiding students how to proceed (if the test shows that the level of knowledge is not sufficient, student can be directed back to the respective part of the course or the system can offer some additional study materials). To use this feature rationally, we have to develop methodology of design and development of various types of tests. In 'soft' science[s], [such] as economics and management, it is not always easy to design tests based only on yes-no or multiple choice questionnaires, so the involvement of the tutor in the assessment process may be necessary, what implies possible delay of the answer. Later on, it may become possible to use expert systems for assessment of solutions of more complicated problems, but at this stage we do not plan their use." (Eger and Vacek)
Online learning environments can also provide transparency in the assessment process:
"…if we're considering new assessment strategies, then using the transparency offered by online learning environments … is also beneficial if we utilise it in a way that encourages student motivation, learning, best practice and ownership." (Finkelstein-D 2000b)
In designing our assessment strategies to take advantage of new ways of learning we still need to be clear about the basic issues, including:
  • who our audience is,
  • why we are assessing,
  • what we are assessing,
  • how we are assessing,
  • who is best placed to do the assessing.

3 Planning for Assessment

This section considers the main issues that we need to address when planning new methods of assessment and feedback:
 

3.1 Assessment as communication

It is important to understand that assessment is a form of communication. This communication can be to a variety of audiences, including:
  • the students (feedback on their learning),
  • the lecturer (feedback on their teaching),
  • the curriculum designer (feedback on the curriculum),
  • administrators (feedback on the use of resources),
  • employers (quality of job applicants, results of work-based training).
When designing and selecting assessment methods we need to ensure that the communication is as meaningful, useful and honest as possible.

3.2 The purpose of assessment

We may wish to assess students for a number of reasons that usually fall into one of two main categories (Goodall and Elvidge, 1999):
Both types of assessment can be used to motivate students and lecturers.  

3.2.1 Assessment for learning (formative)

Assessment for learning, or formative assessment, is designed to assist the learning process by providing feedback to the learner which can be used to highlight areas for further study and improve performance.
"Online quizzes are provided for the student to check his/her progress in understanding the subject. Sample documents can be uploaded for practice." (McFarlane)
As Glasson notes the focus in formative assessment is on feedback rather than assessment:
"The focus was on feedback rather than assessment. Feedback on the student's understanding of the technique. ... the (technique expert) tutor commented on and coached each participant in turn based on their original (faxed) solution and their contribution to the teleconference discussion and took questions. Formal assessment came in later modules where students were expected to apply the technique." (Glasson)
New learning environments offer new opportunities and ways for providing feedback:
"... the greatest contribution of IT to all this lies in the increased interactivity offered by asynchronous networks, which for distance courses was previously unattainable Apart from the obvious collaborative work, and possibilities for enhancing feedback, you can submit iterative drafts of assignments, or maybe negotiate assessment criteria." (Macdonald-D 2000c)
Formative assessment is intended to:
  • tell students how to improve their performance,
  • diagnose a student's strengths and weaknesses,
  • predict success in employment,
  • predict success in future courses,
  • provide feedback to lecturers,
  • provide feedback to students,
  • provide a profile of what a student has learned.
Formative assessment is often used to prepare for students for summative assessments:
"Apart from the obvious collaborative work, and possibilities for enhancing feedback, you can submit iterative drafts of assignments, or maybe negotiate assessment criteria." (Macdonald-D 2000c)
"The quiz area was used to facilitate topic study and revision for the final exam." (McFarlane)
"Formative assignments leading to final assessment by portfolio." (Sharpe and Baume)
 

3.2.1 Assessment for grading (summative)

Assessment for grading, or summative assessment, is for external purposes and does not normally provide extensive feedback to the student. It is intended to:
  • pass or fail a student,
  • grade or rank a student,
  • select for future employment,
  • select for future courses,
  • give credence to a course,
  • tell students what they have achieved,
  • help the students develop their skills of self-assessment.
 

3.3 What we assess

What we assess must be valid in that we assess what we teach and what the students learn. As Macdonald (D 2000a) notes:
"How you measure the validity and reliability of your assessment makes vast differences. .." (Macdonald-D 2000b).
".. if you are intending to train students to learn online, then you have to decide what skills they will need to acquire, and make sure that their development is supported in the assessment. It's no good concentrating on the subject of the course, and forgetting about the process of online learning." (Macdonald-D 2000a)
The assessment should be linked to, and measure, the students' learning measured against the stated aims, objectives and learning outcomes of the 'learning module'. As Gilbert-Hunt and McLaine suggest the methods we choose must be appropriate to the skills, knowledge and abilities the students are learning:
"Assessment methods are closely linked to the subject objectives and incorporate validation of the individuals active participation in their own learning and that of their peers. Thus there is a range of assessable components including: a personal learning profile; evidence of online communication with peers; bibliographic file for ten items; Instruction Package for a targeted client readership - overall plan and details of one section; written essay; self-reflective assessment." (Gilbert-Hunt and McLaine)
The types of learning we might wish to assess are summarised in Bloom's Taxonomy of educational objectives in Figure 3.1 below.

In developing these types of learning in an online environment perhaps we need to take a more integrated and supportive approach in designing assessment strategies:

"... activity based assessment, where students have to undertake various online tasks, and reflect on their experiences in written assessments. Or the use of incremental skills development in assessment, where they learn a relatively simple task, reflect on their experiences, and then build on it in a subsequent assessment." (Macdonald-D 2000b)
As Spratt (D 2000a) suggests we need to employ a diverse range of assessments to suit the needs of learners and the discipline, and the assessments should be supportive and promote effective learning:
"It seems to me that the issues you have been talking around clearly demonstrate a recognition of the need to diversify assessment to suit the discipline area, the course outcomes and the learners needs. There are other issues related to the broader curriculum too. Too often assessments can become punitive and meaningless..." (Spratt-D 2000a)
 

Figure 3.1 Bloom's Taxonomy of Educational Objectives

The table is based on the work by Bloom and Krathwohl (1956).
 
Competence
Skills demonstrated
Knowledge
  • Recall of information,
  • Knowledge of facts, dates, events, places,
  • Question words: list, define, label, describe, and name.
Comprehension
  • Interpretation of information in one's own words,
  • Grasping meaning,
  • Question words: interpret, discuss, predict, summarise, and classify.
Application
  • Application of methods, theories, concepts to new situations,
  • Question words: apply, demonstrate, show, and relate.
Analysis
  • Identification of patterns,
  • Recognition of components and their relationships,
  • Question words: analyse, arrange, order, explain, connect, infer, compare, and categorise.
Synthesis
  • Gerenalise from given knowledge,
  • Use old ideas to create new ones,
  • Organise and relate knowledge from several areas,
  • Draw conclusions, predict,
  • Question words: integrate, modify, invent, design, compose, plan, formulate, and arrange.
Evaluation
  • Make judgments,
  • Assess value of ideas, theories,
  • Compare and discriminate between ideas,
  • Evaluate data,
  • Question words: appraise, judge, evaluate, defend, rank, conclude, discriminate, and recommend.
 

3.4 How we assess and who assesses

In designing assessments three main issues we need to consider:
  • how we are going to approach the assessment,
  • our choice of assessment methods,
  • who should carry out the assessment.
Approaches to assessment include self-assessment, peer-assessment, group-based or collaborative assessment, negotiated assessment such as learning contracts, computer-assisted assessment and workplace-based assessment. The majority of OTiS examples and discussions refer to online learning mediated by asynchronous, text based methods with some use of synchronous, text based techniques. They focus on online assessment methods mediated by tutors and peers rather than computer-assisted assessment.

There exists a wide range of assessment methods for each of the approaches outline above. The majority of the OTiS exemplars focus on assessment methods that facilitate assessing the skills developed by active, reflective, constructivist and collaborative learning.

Traditionally the teacher or tutor has undertaken assessment. Other options include external agencies, the students themselves, employers, managers or customers/clients. The OTiS case studies document many examples of assessment by self, peers and tutor, as for example in Juwah's Developing Effective Online Tutoring:



"Assessment is on a continuous basis and undertaken by a combination of self, peer and tutor assessment." (Juwah)
Most of the case studies describe multifaceted approaches to assessment and use "a rich mix of group and individual assessment methods" (Phillips).
"Students were assessed in three ways, all of which took place within FirstClass: conferences, journals and a group project." (Gwynne and Chester)
"Online discussion was then utilised as the basis for group and individual assignments that were sent online to communal 'assignment spaces'. Feedback and marks were relayed back in similar transparent manner. A final exam taken in traditional university spaces and forms completed the assessment pattern." (Finkelstein)
"Each course has its own assessment process which can include short individual article analysis, online group collaboration and online presentation of the group efforts, online brainstorming, group collaborative full papers, and individual full papers. All assessment is tutor-marked." (Janes)

3.4.1 Self-assessment

Self-assessment involves the student in the process of assessing their own learning and performance. It can help move them towards becoming more reflective, autonomous and effective learners.
"Face-to-face feedback sessions were held after the videoconferences which were videotaped. The students were asked to write a short report in which they would analyze the conference and their own performances. They were also asked to suggest what they would do differently in the following conference." (Tammelin)
"At the end of the course, students were asked to assess and rate their success in the course, and they were instructed to give a grade for themselves for the course. They were coerced to think hard on their learning process in the course, and they became very critical about their own work. Giving ownership of their own learning experience (using the aid of web-based technologies) made them more aware of pedagogic impact of their course to their teaching profession in general." (Mohamad)
The Self-assessment in Professional Higher Education (SAPHE) project has produced guidelines on self-assessment for both students and staff.
 

3.4.2 Peer assessment

Peer assessment involves students assessing each other and providing feedback and opportunities.
"I have increased the expectations for both students and myself by adding assignments, which require students to upload lesson plan files for peer review on the Web site." (Hird)
Peer assessment can be combined with other approaches such as group/collaborative assessment, but like self-assessment, this approach is most often employed in formative approaches to assessment:
"The self and peer assessment elements were formative, with the tutor's assessment being summative." (Juwah)
See also resources under self-assessment.
 

3.4.3 Collaborative or group-based assessment

In collaborative or group-based assessment small, interdependent groups of students work together as a team to help each other learn - the group members are dependent on each other.
"Co-operative assignments were structured to meet the following prerequisites for effective co-operative learning (Johnson and Johnson, 1991):
. a mutual goal,
. positive interdependence, and
. individual accountability." (Clarke)


The online environment is particularly appropriate for collaborative learning approaches that emphasise group interaction. However it can be difficult for students to adapt to this way of working and the assessments need to be structured to allow them to develop the necessary skills:
"...students are introduced to online collaboration in a two step process. Reflection encouraged in the first assignment is used as preparation for the next collaborative assignment." (Macdonald)
"One assignment involved the tutorial group splitting into two subgroups and creating two web sites. This allowed the students to work in a small group on a defined project and to experience the 'forming, storming..' etc. processes, discussed in a previous tutorial activity, for themselves." (Morrison)
The case studies raise some of the problems that can be encountered in implementing collaborative assessment including resistance from students and high levels of anxiety:
"There was major resistance to assessing online team working capability, but in fact, the concept provided a much-needed incentive to participation." (McKenzie)
"Genuine collaboration is in very sharp contrast to the competitive ranking that students know well by the time they reach college. I find that I need to provide a tremendous amount of assurance to students that they do not have to get it 'right' the first time and that they will not slip off the honours list as the result of taking an online course." (Hird)
Only when students are confident in this new learning environment and with this new way of working should more demanding assessments be introduced:
"Once the groups had a reasonable expectation of performing as virtual teams, later modules assessed the quality of collaborative online working through a subject tutor allocation of marks for quality, quantity and timeliness of the contribution." (McKenzie)
Group-based and collaborative approach can be combined with self and peer assessment but marking and feedback issues are often seen as barriers:
"...there is the issue of how to issue assessment and feedback that takes into account whether you're working with large or small groups. For example, do we issue group assignments, or insist on individual assessment. Group work versus individual assignments can be tricky, particularly if you have groups with poor interpersonal dynamics (or non-contributors). At same time, group work to me is important if we're arguing for courses and modules that address, reflect on and prepare students for how 'the real world' can operate -which can very often be team driven and dependent on group contribution and effort." (Finkelstein-D 2000a)
Finkelstein suggests one approach to group feedback and assessment:
"... assessing it involved an explicit promise to return feedback in two forms: 1) a printed version of their assignment with comments on the side and a grade, for viewing privately by the group; and 2) an electronic statement of a paragraph of detailed commentary and a grade, which was posted online in a public space, and was meant to allow transparency and student awareness. Once again, it was important here because it allowed students to compare and contrast work and see for themselves what did well and what did not, offering students instant best practice. Result: a competitive edge among students to do better next time, particularly in light of the public forum their work was entered into. And interestingly enough, students did comment on other work, both positive and negative, in their online tutorial spaces, showing that they were actively visiting, reflecting, gauging, comparing and learning. Very gratifying for both tutors and students when groups did get it right and we were able to offer high marks and strong praise." (Finkelstein-D 2000b)
The online environment can enable collaborative approaches to assessment and provide new opportunities for feedback. However, these types of assessment need to be introduced with care and consideration, ensuring that the students can develop the necessary pre-requisite skills and confidence before being summatively assessed.
 

3.4.4 Computer-assisted assessment (CAA)

Computer-assisted assessment involves the use of computers in the assessment of student learning. Currently the most common application of CAA is for Multiple Choice Questions (MCQs) that can be automatically marked. However, CAA is becoming more sophisticated. It can:
  • provide the ability to include a wide range of media, catering for different learning styles (Chapter 1).
"In addition an interactive tutorial was written for each topic within the course and short formative tests, (comprising multiple choice, multiple response, fill in the blanks and diagrammatic hotspot questions) for each topic were set." (Saunders)
"A small number of current tools attempt to measure higher order skills. For example, some packages assign marks to essay-style questions 'based on key words, phrases and statements recognised as forming part of an acceptable answer' (Bocij & Greasley, 1999). Some tools do not attempt to mark the students work, but provide tools to help the student develop assessment items such as portfolios (Kjollerstrom & Martensson, 1999). In the future, multimedia assessment promises to be more practical, authentic and challenging than traditional 'pen and paper' tests (Herrington & Herrington, 1998). Other innovative forms of assessment such as a summary of statistics of learners' paths through multimedia programs may also be developed (Herrington & Herrington, 1998)." (Salter-D 2000)
  • provide easier management, administration and reporting of the assessment feedback and results.
"Randomisation of question presentation can reduce cheating. A host of statistics, such as date, time, number of attempts made, time taken, score etc can be recorded for each item. Some systems can automatically perform useful analyses such as an item analysis of the test." (Salter-D 2000)
  • provide immediate feedback to students that can be tailored to meet their individual needs, eg incorporating hints and tips, and suggesting appropriate follow-on learning activities.
"...the feedback for objective, formative assessment consists of immediate feedback on the score for individual questions plus other static, textual or graphical advice, eg the worked example." (Nightingale-D 2000)
"Students appreciate the immediate feedback provided. They may be allowed multiple attempts at a test, possible until a mastery level has been reached. After completion students may be able to review the test in their own time to see where they made errors." (Salter-D 2000)

Student Perceptions

Salter (D 2000) reported that students can perceive CAA as more accurate and objective than traditional forms of assessment although Nightingale (D 2000) suggests that this trust needs to be nurtured:
"A study by Bocij & Greasley (1999) found that students generally perceived computer-based testing to be more accurate and objective. A majority felt that their performance was equal or superior to that in a traditional exam. An interesting finding was that 'students felt more comfortable and relaxed in the laboratory - as opposed to an examination hall'". (Salter-D 2000)
"It goes without saying that the assessment engine needs to be absolutely reliable but Students need to trust and understand the basis of the judgement [and hence feedback], e.g. how the assessment engine deals with rounding errors, spelling mistakes, etc. Many of my responses to students are concerned with these issues." (Nightingale-D 2000)
As Nightingale (D 2000) points out, CAA does not provide all the answers and the human touch is still needed:
"A student needs to know if their answer is wrong and why it's wrong. The latter is much more difficult to personalise. There is a trade-off between appropriate, automated feedback versus the time constraints on Tutors." (Nightingale-D 2000)

Tutor Perceptions

CAA can be perceived to be all advantages, particularly for the tutor as Salter (D 2000) comments:
"One of the least favourite tasks of teachers is marking. Online tests promise to reduce this burden." (Salter-D 2000)
We must ensure that we do not use CAA excessively or in inappropriate ways:
"The ease with which online tests can be delivered and marked may result in inappropriate and excessive use. Most of the current online assessment tools have facilities for delivering multiple-choice, true-false and short answer style questions. These generally deal with lower order cognitive skills, such as recall of facts, rather than higher order skills, such as analysis, synthesis and evaluation. Many educators take it as a matter of faith that standardised testing procedures are true indicators of learning (Herrington & Herrington, 1998). A hasty implementation of online assessment may not only promote surface learning (Littlejohn & Sclater, 1998), but give a distorted and inaccurate picture of student performance (Bocij & Greasley, 1999). (Salter-D 2000)"
A summary of the requirements for CAA systems is outlined in Appendix 4-B.
For further information on CAA visit the Centre for Computer Assisted Assessment web-site at http://www.caacentre.ac.uk/. Their publication 'Blueprint for Computer Assisted Assessment' (written by Bull and McKenna) provides a comprehensive guide to computer-assisted assessment.
The Scottish Computer Assisted Assessment Network reports in detail on the requirements of CAA systems in three UK Higher Education Institutions, available at http://www.scaan.ac.uk/.

3.5 Taking the next step

Online learning environments offer new methods and approaches to assessment which have the potential to motivate students and promote more effective learning. However we need we need to apply these New Assessment Methods with caution.

Currently the assessment of, and feedback on, higher order skills requires human mediations (self, peer or tutor) and as Salter (D 2000) notes, we need to ensure that in our rush to adopt new assessment methods we do not adopt inappropriate forms of assessment:

"It is pleasing to see how many are advocating assessment methods that (we hope) measure higher-irder [sic] thinking skills. A number of people have also mentioned automatic marking of assessment. This promises to ease our workload, but may also result in staff using inappropriate forms of assessment for administrative expediency." (Salter-D 2000)
 
This part of 'New Assessment Methods' describes current innovative practice in assessment that exploits the potential for collaborative learning offered by the online environment. It is divided into four sections: Relationships, Quality Evaluation, Aspects of Development and the Format and Structure of Assessment, based on the content of the case study reports and the discussion.

4 Relationships

The vast majority of the material that was written, both in the case studies and in the discussion group on assessment, focused on the relationships involved in assessment. These relationships take three forms:
This in itself is interesting as assessment is traditionally seen as an interaction between the student and the institution rather than between individuals.

The emphasis given to self-reflection in assessment would appear far more extensive than is usual. This could be for two reasons. Firstly it may be that the nature of online learning lends itself to self-reflective practise, or secondly, it could also reflect the fact that tutors involved in online learning tend to be more exposed to good practice in teaching which puts a great deal of onus on students' own awareness of their learning.

It is notable that peer co-operation and interaction appears to be high, with one case study commenting on the group ethic that had developed over the course. A number of courses actively encouraged this interaction, using the assessments as a way of getting students online. It would appear that although this enforced collaboration and interaction, both through the online discussions, the group work and the collaborative learning processes were not universally popular, students did appreciate the benefits of enforced engagement.

The students' relationships with the tutors seem to be characterised by a more equal status than might be expected in traditional teaching. This might be related to the lack of indicative status symbols, found in traditional spaces, or by the lecturers' active encouragement of a more equal relationship. Feedback was an area that seemed to be particularly highlighted in the case study reports, suggesting that tutors were actively considering what kinds of responses they were giving to their students. Aspects of the feedback that were frequently mentioned were the speed of response, the consistency of the assessment and the supportive nature of the communication.





4.1 Reflection - relationships with self

4.1.1 Reflective Journals

There were a number of references to reflective journals in evidence, more than might perhaps be found in a survey of traditional courses. In Nurmela's Online training for online tutors a reflective journal is kept throughout the course:
"During the first week participants presented themselves and described their expectations with a given peer participant. This was a start for serious reflection that continued during the whole course. Participants were also asked to write their thoughts and reflections that arose during the course (a personal reflective log)." (Nurmela)
In Cowan's Personal development planning, a major part of the assessment is conducted through a reflective journal, the format of this is described below:
"The student writes a reflective journal in which s/he addresses a question for which they don't currently have an answer, which they judge to be relevant to the learning in the course and for which an answer or part answer would be useful. They try to obtain an answer. "The journal is sent as an attachment to the tutor with a covering note if the student so wishes; most do
"The tutor emphasises with the entry, asking questions where the student's thinking is not clear or when information is missing; suggesting questions (but not answers) which the student might usefully pose to herself or himself; identifying strong feelings within the text; pointing out if the journal is not reflective, and suggesting how it might be made more so. The commenting tutor strenuously avoids the use of the first person singular lest the student is tempted to make the journal a piece of correspondence with the tutor. We feel that if there is dialogue, then it should be of writer with self. Comments are all added as footnotes. "The journal with comments is returned to the student. The tutor usually writes a cover note which is personal." (Cowan)
The use of journal based assessment is also in evidence in Gwynne and Chester's Personal identity and community in cyberspace: an evaluation of teaching and learning online, where students are first asked to comment on the name that they have chosen to represent their online persona:
"The students' first task in the subject was to write a journal entry about their choice of alias. … Although some students reported that they didn't give much consideration to the alias they chose, as the semester progressed, they were challenged to see that names they adopted nonetheless provided powerful cues for the impressions that other students formed." (Gwynne and Chester)
As well as emphasising the need for self-reflection, this extract also gives some insight into the ways in which relationships are formed in online learning communities. There is restricted and mainly intentional interaction, as opposed to the complex interplay of exchange that occurs in an offline context (such as body language, dress, touch or annoying habits) which forms the basis for future interaction. This may hamper integration and relationship forming, as participants are basing their interactions with others on such a limited (and often misleading) set of cues, however in other contexts it might appear that this restricted interaction is beneficial. There is some evidence that the difficulties associated with journal writing, are not so much in evidence in an online context: it may well be that the impersonal nature of the medium makes the students more relaxed and less self-conscious than might be the case where more immediate contact was expected. Where face-to-face contact is the norm, it may well be that the students are writing with a view to how the reader will deconstruct the text, however where online contact is more prevalent, it would seem that the tutor is seen in a more objective role, perhaps because a more strictly defined relationship has been formed.
"The normal barriers to commenting on journals, of which Moon makes a fair amount of mention seemed to be much less concern to these students, who saw the commentator as even more detached and impersonal, in the sense of being intimate with highly personal thoughts recorded by the journal writer. So it was the opposite of a barrier, rather an advantage, upon which we built and continue to do so." (Cowan)
The use of reflective journals is in evidence in other case studies (White and Moussou, Mohamad, Nurmela, and Daele). In these studies the journal is not part of the formal assessment process but rather is used informally to encourage students to develop their skills in self-reflection, and self and peer assessment.

4.1.2 Self-assessment

Self-assessment emerged as a popular strategy in online learning, particularly in postgraduate and professional development courses:
"Participants …were encouraged to reflect on their existing practice and seek guidance when they perceived a weakness. Staff …were taught the specific skills they identified they needed." (Newby-Fraser and Clayton)
Self-assessment is frequently used for formative assessment of students, as for example in Gilbert-Hunt and McLaine's Critical thinking and learning in Health Science:
"By providing enough instruction to walk students through everything they needed to do, an even playing field was provided for the whole group. Those who were already competent could skip these aspects of instruction." (Gilbert-Hunt and McLaine)
Self-assessment mechanisms are frequently used in distance education to enable the students to monitor and gain feedback on their own progress, as for example in Higgison, Haragus, Scheuerman et al and Kennedy and Duffy. Online learning offers the opportunity to make these self-assessment mechanisms more interactive and responsive to individual students' needs:
"In addition an interactive tutorial was written for each topic within the course and short formative tests, (comprising multiple choice, multiple response, fill in the blanks and diagrammatic hotspot questions) for each topic were set." (Saunders)
Students are also asked to revise their material in the light of tutor feedback or discussion. The self-assessment involves the student deciding which feedback is valid and should be followed and which is not:
"The transition from discussion to test completion was deliberate because each test increasingly asked students to reflect upon their contributions to discussions and to return to the discussion area to develop and refine points they had made." (Gwynne and Chester)
This continuous, formative self-asssessment process might be encouraged partly because the technological context of the learning means that it is possible for students to revise their assignment easily. Changes and revision are more difficult to request when assignments are hand-written. However, this process might also reflect the increased responsibility that students have for their learning in an online context. The lack of formal structures, such as lectures and tutorial spaces may increase the focus on the learning activity itself rather than the formal structures surrounding it. It may also be the case that tutors who have become involved in online learning tend to be innovators within education, and as such are more aware of modern assessment theory than the majority of lecturers.

4.1.3 Self-reflection

Self-reflection features in a number of the case studies, with different projects requiring reflective practice at different points in the learning process, and it is often a pre-requisite of self-assessment.

In Rosie and Thompson's Using TOPCLASS to promote student learning, the reflective process is incorporated from the start, with students required to make explicit their demands for the course, and staff producing the expected aims, so that each could be continually compared as the teaching commenced.

"Students were explicitly asked to consider their personal learning outcomes and to see the course as a vehicle for meeting such outcomes. As part of this development the course tutor made the learning and teaching aims explicit for students and tutors." (Rosie and Thompson)
In Gwynne and Chester's Personal identity and community in cyberspace: an evaluation of teaching and learning online, reflection is introduced through the creation of an online persona, and students are requested to consider their 'online student identity' and become active participants in the creation of that identity.

In Bailey's Experiences of running online learning sets the postgraduate course consists of three modules, two of which are based on reflective practice.

Sometimes the reflective practise is built into the assessment itself as in Gilbert-Hunt and McLaine's Critical thinking and learning in Health Science:



"Throughout the subject it was important to make the learning strategies explicit for the student, this was achieved by providing reflective activities, which contributed to set assignments." (Gilbert-Hunt and McLaine)
This approach requires students to use their own experiences to comment on material. This type of reflection makes explicit a constructivist view of learning, where learning is seen as being the product of repeated accommodation and integration of new knowledge and experience to construct a coherent body of learning through the refinement and reflection on the learner's current awareness. This type of assessment integrates this model of learning into the assessment process:
"In week 6 of this 12 week course the tutor introduced the first in series of 'tests'… It gave students an opportunity to comment on the extracts and to relate them to their own experience." (Rosie and Thompson)
Group-work was a strong feature of these case studies, as a following section goes on to demonstrate. Reflective practice was also encouraged in group-work activities in a number of the projects. From the case studies, it would appear that this was a helpful practice in encouraging students to understand the difficulties that they were facing - some of which would be general group-work difficulties, while others would be more specific to online learning.
"Teams of students were expected to submit weekly progress reports about their project, highlighting achievements and barriers to progress. This served the purpose of keeping students on track, but also let students understand that others were facing similar difficulties, and led to sometimes deep discussion about project management issues." (Phillips)
"For THD 204, an early conferencing assignment was modified to encourage reflection on group management, …. A similar opportunity is offered in T171, where students are introduced to online collaboration in a two step process. Reflection encouraged in the first assignment is used as preparation for the next collaborative assignment. "Integration of the practise of online group participation with reflection is also a feature in the course B823, …. Part of the assessment requires them to critically reflect on the experience of working in a newly formed internet team, including both social and technological aspects." (Macdonald)
And self-reflection can be incorporated at the end of a course:
"An integral component of the subject was the inclusion of a student self-reflective assignment, which consisted of a questionnaire covering each of the four modules." (Gilbert-Hunt and McLaine)
It may well be that this emphasis on reflective practise is a reaction to the difficulties that the tutors are observing in online learners. For most of the students this may well be their first experience of online communication, thus the lecturers have the dual role of trying to teach the subject of study, but also to teach the practice of online learning. One possibility, which would require further consideration, may be that the tutors, through reflective practice are trying to assess how successful they have been in communicating the practice of online learning. This would suggest that reflective practice might be a lesser feature of online courses as student experience of online communication becomes more commonplace.

4.2 Collaboration - relationships with peers

Online learning supports collaborative approaches to learning and assessment that involve students relating to each other as well as to the course content and the tutors.

4.2.1 Nature of peer co-operation

It is notable that peer co-operation appears to be quite high, despite the restricted contact that individuals have with each other. In Anderson and Simpson's Programme-wide online group interaction: developing a social infrastructure, the strong group ethic is commented on:
"A strong ethic of group responsibility was developed - most online tasks were group tasks that required each person to undertake some part of a task that groups had to report on. A variety of tasks were developed." (Anderson and Simpson)
While in Finkelstein's Utilising online learning in a humanities context it is noticeable that despite no formal guidelines, group members sometimes voluntarily shared out individual parts of an assignment.
"Of interest here was the ways different groups co-operated in writing their assignments. For example, some broke the assignment down into individual tasks to be collated by the group leader, while the others left the entire task to the individual responsible for that particular task." (Finkelstein)
 

4.2.2 Student/student interaction

A number of the case studies discussed the need for student interaction to be built into the course and assessment design, in order for student to fully participate in the course.
"Interaction must be required - build activities into the course material in such a way that the students cannot avoid interacting." (Anderson and Simpson)
"Participation in certain discussions is required to complete many activities. Some discussions emerge from the students themselves, other discussions are initiated by the tutor, while some are initiated by students at the tutor's request." (Pickering and Duggleby)
An evaluation undertaken by Macdonald in Integrating online tuition with assessment at the UK Open University reveals that students appreciated this design, even where there is initial resistance as documented in McKenzie's Enriching content teaching.
"The activity-based structure requires students to learn by undertaking practical activities linked to the assessment. The evaluation revealed …, they appreciated the integration of activities with assessment, because it guaranteed the involvement of all students." (Macdonald)
"There was major resistance to assessing online team working capability, but in fact, the concept provided a much-needed incentive to participation." (McKenzie)
Where in a traditional course, students may become integrated into a learning group merely by their physical presence and non-study related interaction, in an online course this is more difficult. Shy students may find themselves marginalised as the group gains cohesion, with little opportunity to interact. Linking interaction with assessment encourages students to overcome those barriers at the start.

The manner that Janes, in Teaching online in a post-graduate certificate in technology based distributed education, has started student interaction through the posting of biographies is an interesting technique, as much initial interaction in an offline course consists of essentially verbal biography and non-course related discussion. This is accompanied by a short informal assessment, where students are invited to discuss each other's postings.

"Over the thirteen weeks of the course, the students engaged in a number of activities designed to increase their interaction with each other. The initial interaction was comprised of introductions and the voluntary posting of both formal and informal biographies. In addition, students were asked begin the course by submitting three short individual analyses of written work…papers, which discussed social issues and technology based learning and were looked at in light of frameworks provided in the readings." (Janes)
This approach is used in many of the other studies as for example Morrison's T171: The pilot year experience:
"The discussion of the tutor group activities also formed part of the first two assignments, which encouraged participation. The aim of the first tutor group activity was to provide an opportunity:
· for the group to get to know each other (by sharing a little bit about themselves),
· for individual students to make the first step towards group collaboration…"
(Morrison)


Macdonald makes the point that the students' interaction itself can be assessed with students being rewarded for interaction, regardless of content. This use of assessment appears unusual, but is in fact is quite similar to giving marks for good essay grammar and punctuation, including in public examinations such as GCSE, or deducting marks on a sliding scale for late submission. All of these examples are designed to produce 'good habits' on the part of the student, although they can be perceived as 'unfair', or 'easy' marks, particularly in high stakes examinations, or with less mature audiences. It may be desirable to explain carefully why these marks are being awarded and have only a small number allocated for this purpose. Other courses have focused on online activities at certain points dictated by the assessment and the following illustrates the variety of tasks which students may be required to undertake. At the most basic level, students may be encouraged to participate in online conferencing simply by awarding marks for a message input into the system, and this is employed for the first assignment in THD204, in order to ensure that all students get