This guide (1996) aims to draw attention to the likely benefits of adopting more innovative forms of student assessment. In doing so, it highlights some of the intrinsic qualities of so called innovative assessment practices contrasting these with the qualities of more traditional forms of student assessment. By drawing the reader's attention to published research findings on student assessment, the guide also sets out to allay the fears of some academic staff by answering the more commonly made criticisms of innovative assessment practices.

This guide has been written by Graham Mowl, Senior Research Assistant in the Educational Development Service, University of Northumbria with support from Liz McDowell and Sally Brown of the same institution.

It is always somewhat easier to say what innovative assessment is not, than to attempt to say what it actually is; it's not the three hour, unseen, anxiety-provoking exam, marked by a lecturer whose comments the student never sees. What, then, is it?

In this guide, we prefer to think of innovative assessment in terms of its characteristics and its mission. We argue that innovative assessment is not just a case of adopting one of the many recognised innovator's 'tools of the trade' although in many institutions this alone is still considered pretty radical), it involves something snore fundamental a commitment to the goals or philosophy of innovative assessment. As Harris and Bell (1990:97] suggest:

Literally defined, innovative assessment could be any form of assessment which involves the application of a new technique or method. For us, however innovative assessment has come to mean more than that. It is a term we use which encompasses a whole range of different techniques and methods, not all of which are new inventions. What unites them is a common goal: to improve the quality of student learning. Innovative assessment is also about what Heron (1981) called 'the redistribution of educational power' when assessment becomes not just something which is 'done to' learners but also 'done with' and 'done by' learners [Harris and Bell, 1990]. Assessment should be primarily for the learners and for learning. As defined by Rowntree (1977), it is about getting to know students and the quality of their learning.

All too often, assessment is associated with the end-product of the learning process oath emphasis given to measurement, evaluation, grading and social acceptance. But there is more to the process of assessment than establishing whether or not you can attend the 'passing-out parade' or whether your name will be read out first or last at the graduation ceremony. As the table below demonstrates assessment can be used to perform a number of different functions: it can be used as both carrot and stick. According to Harris and Bell (1990) assessment is also carried out for three different groups of people: the learners, the teachers, and the 'outsiders'. It could be argued that traditional methods of assessment, although perhaps not intentionally, have sometimes tended to overlook the needs of the learner in an attempt to provide readily accessible and comparable results for others e.g. teachers, employers, parents, and government statisticians. The output from many forms of innovative assessment (e.g. profiles, portfolios, pro-formas) however, lend themselves much less readily to quantification and the production of league tables of 'performance'.

Assessment:

What for? to reward to motivate to know what they know to know what they don't know to punish to certificate to classify to compare to evaluate to diagnose to appraise to empower to improve the quality of learning

Who for? learners/teachers/outsiders learners learners/teachers/outsiders learners/teachers/outsiders learners learners/outsiders learners/outsiders learners/outsiders learners/outsiders learners/teachers learners/teachers/outsiders learners learners/teachers/outsiders

Although, this table suggests that the learner is the only group for whom all the different functions of assessment are performed, in reality this isn't always the case. In some cases assessment by-passes the needs of the learner.

For example, in any form of assessment it is usually beneficial for the learner to receive detailed feedback indicating their strengths and weaknesses in the area of assessed work and suggesting areas where improvements or changes might be made in the future. If however. learners only receive a single, all encompassing mark or grade for a piece of assessed work, however good or bad that mark may be, the learner is unaware of precisely what aspects of the work were strong or weak. Consequently, the assessment fails to perform any formative (feedback giving) function in the learning process. One can see how ridiculous this situation would be in a vocational subject, such as medicine: imagine if during your training at university you got six out of ten for performing a heart triple by-pass operation on a dummy but were never told what you lost the four marks for!

Fortunately, this situation is unlikely to occur with a vocational course, such as medicine, as assessment in such subjects is predominantly 'criterion referenced': to pass the student has to demonstrate that they are 'competent' in certain skills.

Although it is obviously problematic, primarily because it is a danger to the public, to let a medical student graduate without telling them what they are doing right or wrong, it is quite common for a student to graduate in less vocational subjects, such its sociology and geography, knowing only that they performed better than X and worse than Y but not really knowing where, if anywhere, they went wrong and how they could improve on their performance in the future. The learner is left fumbling in the dark without a guide. Often it is simply a lack of communication that prevents assessment from serving the needs of the learner, the teacher is aware of their strengths and weaknesses but these are not communicated effectively to the learner, without this knowledge the learner is powerless. Innovative assessment, in contrast, is about finding ways to empower the learner.

So what are the aims of innovative assessment? Some might even say: "what's the point in changing an assessment system that has served us so well for so many years?"

But has the system really 'served us well'? Much of the impetus for innovation and change in the assessment process stems from a general dissatisfaction with traditional methods of assessment and a shared belief that these methods rather than enhancing and facilitating the learning process actually stifle deep learning and creative thinking. Many innovators would therefore challenge the belief that the existing system served us well. Innovative assessment is not just some trendy, new technique dreamt-up purely to save on the amount of time teachers spend on marking, it is a genuine attempt to improve the quality of learning in higher education. If we do save time in the process then all the better for our own learning.

In the context of student assessment therefore, innovation aims to achieve several specific outcomes.

Firstly, it aims to produce students who are:

Secondly, it aims to produce a more fertile learning environment and a more rewarding learning experience for all teachers and students.

Another guide to be published later in this series, Assessment: an innovators guide, provides an introduction to some of the techniques and methods more commonly employed by innovators' in an attempt to achieve some of these aims and offers some practical tips on how to reduce the risks of failure.

Before one can start to implement changes in the way students are assessed one has to deal with the skeptics, those who will resist change of any kind, and those who tried it once, found it didn't work and so dropped it altogether. Some teaching staff delight in telling others horror stories about so and so's 'famous failure' with innovative assessment, usually concluding the story with those well worn words of warning: 'Is that what you want? Cause that's what'll happen.' While we would be the first to concede that innovative assessment practices are not without their problems and do require careful and considered application if they are to be successful, there is much misinformation bandied about to prevent one from even considering the possible benefits which might be attained from changing the present assessment system. Many people still regard innovative assessment practices with a great deal of suspicion and have several misgivings about their supposed benefits. The lists presented on the following pages, therefore, are an attempt to allay some of the fears and to answer some of the more common criticisms of innovative assessment practices. They are a compilation of our own remaining nagging doubts about innovative assessment together with the concerns more commonly expressed to us by other members of staff.

Nevertheless, we still do not claim to have produced a list which is exhaustive!

1. Innovative assessment is just a 'soft option', simply less 'rigorous' than traditional forms of assessment. e.g. Those people who can't put together a decent essay pick up marks just by giving a ten-minute talk or worse still, marking their own work.

Innovative assessment is about valuing a range of different skills and, as such, it could be argued that it is more rigorous than traditional forms of assessment, which often only test essay writing skills and relatively short-term memory recall. To be successful under a system of innovative assessment, students have to perform consistently in a number of different skill areas and over a longer period of time; it doesn't just assess the student's performance when under considerable pressure on one hot afternoon in the middle of summer. Furthermore, the association of 'hard' with 'good' and 'soft ' with 'inferior' is somewhat questionable. Innovative assessment is softer, more flexible, and more adaptable but that is considered by its practitioners to be a virtue not a weakness. Why should learning be 'hard'? We want more people to learn and to learn more and achieve a deeper level of understanding. Putting up barriers will not enhance or enable this process in any way.

2. Innovative assessment is too demanding, putting unreasonable pressure on some students. Some people would argue that whether you're good at working in a group or not depends on 'personality' and that those who are 'loners' or 'shy' won't do well on group tasks - and it could be very stressful for them.

The same arguments can be levelled against three-hour exams and essays: they favour some students and disadvantage others. It is much fairer to assess students across a range of skills. In some ways innovative assessment is more demanding:it may require that students perform consistently well over the duration of the course rather than just on one day. Furthermore some forms of innovative assessment such as self-assessment, enable students to set their own learning goals and make their own decisions about whether these goals have been achieved. In this way they create their own level of pressure; it is not imposed on them by external bodies.

3. How can assessment be reliable if people other than the lecturers are involved in it? (e.g. employers, self/peer assessment)? Academics are, after all, the experts aren't they? Surely no student in their right mind is going to fail themselves?

Evidence (Heywood, l989) suggests that some traditional methods of assessment (e.g. tutor marked examinations) are unreliable: that results are not consistent with repeated applications. Several studies have shown that not only can the same candidate be given different marks by two different tutors but also the same tutor may give different marks to the same candidate when the same paper is remarked (Heywood, op cit). In fact, one of the major arguments for reforming assessment in higher education is the unreliability of some of the more traditional assessment practices. It is hardly fair for tutors to question whether students can reliably assess themselves when it has been demonstrated that tutor-marked assessments are themselves prone to a lack of consistency. For this reason, multi-marker continuous assessments, in which a series of staff-student corroborated marks is collected over a long period of time, are more likely, when averaged, to be an accurate assessment of an individual student than a single examination mark arrived at by one tutor.

Research conducted by Wondrak (1993) involving health care students and by Orpen (1982) involving political science and psychology students has demonstrated that students and their peers are capable of being very reliable assessors of their own work, particularly when they have a meaningful input into the development of assessment criteria. Furthermore, Wondrak (1993) found that in many cases students, peers and tutors found themselves in agreement about the merits of their written work and in some cases the positive correlations he found between student, peer and tutor grades were statistically significant. Wondrak (1993) found that students, if anything, generally under-mark themselves. Although his results are only based an a relatively small pilot study Wondrak's findings are generally supported by other research in the field of self and peer assessment (Orpen, 1982).

There is some evidence that poorer students do tend to mark themselves up under systems of self-assessment. However, in most self-assessment practices ultimate responsibility still rests with the tutor who takes on the role of moderator with the power of veto if students fail to provide sufficient evidence to support the grade they have awarded themselves. Also in practice self-assessment may be accompanied by a form of peer-assessment which is also likely to have a moderating influence on the self assessed marks (see Boyd and Cowen 1985).

One could also question the belief that academics are the experts on assessment, after all, it is the individual students who really know what they have and haven't learnt, traditional assessments are at best only a random, time-specific sample of this knowledge. Academics should become experts in facilitating a process which enables the students to realise their own strengths and weaknesses and to develop collaboratively individually tailored learning strategies which allow each student to achieve his/her learning potential.

4. Will innovative assessment work with increased students numbers? Is it more time-consuming for staff?

Research by Brown and Dove (1993) suggests that certain forms of innovative assessment such as self and peer assessment can involve considerable extra demands on staff time, especially during the initial stages of implementation. This issue is particularly pertinent at present, when many HE institutions are undergoing massive cultural changes brought about by unitisation/modularisation and the gradual erosion of the binary line. The result of these changes is that many teachers in the HE sector are effectively being asked to both teach and assess more students, while at the same time they are under pressure to produce and, more importantly, to publish more research in an attempt to attract more funds in an increasingly competitive market. Any other potential changes which may involve an even greater investment of staff time are, understandably perhaps, treated with at best suspicion. Most forms of innovative assessment are potentially labour saving for staff once designed and put in place.

5. Surely innovative assessment which is varied and requires lots of different things from students is too much of a burden for them - wouldn't they prefer just to do an exam and get it over with?

For some students, the exam is preferable to more varied and continuous forms of assessment; those students who are good at exams are quite happy to get a way with doing as little as possible for most of the year and then just cramming for a few weeks before the exams. But is this really what we want from higher education; tutors effectively reduced to merely exam setters and markers? And what about those students who are not so good at exams, who show up to every lecture and seminar, retain more from the course than the crammers but simply can't reproduce their ideas under exam conditions? Should we have a system which discriminates against these students or one which rewards effort and understanding?

Booth's (1993) research findings based on questionnaires involving history students indicates many students are very keen to be more involved in the assessment process. As he says 'No involvement leads to lack of interest' (Booth 1993:234).

6. If you give students too much help (formative feedback) - won't you end up doing the work for them? Is this a fair test of their abilities?

Is higher education about learning or certification and accreditation? If it is only about the latter and we are simply in the business of rubber stamping achievement rather than improving the quality of learning through encouragement, response and feedback (Boud, 1990) then, yes, the only form of assessment we require is 'summative'. Surely no educationalist would seriously suggest that our only means of ensuring student motivation is maintaining their fear of failure in summative assessments.

Formative assessment is not just about helping students, it is about facilitating student learning, the two are fundamentally different. When academics submit articles for consideration by refereed journals they do not expect them to be rejected out of hand without any comments or feedback given as to why they are not in their present state suitable for publication, neither do they expect them to be returned re-written by the referees and ready for publication! What the academic expects is constructive feedback about how the paper might be improved, details of what is lacking and possibly some encouragement to re-submit a revised version of the paper. As Boud (1990:102] points out, lecturers are guilty of telling students 'don't do what I do, do what I say'. The question we must ask ourselves is, what are we preparing students for? Is there any sense in having one rule for them and another for everyone else?

7. The role of HE is to develop students' academic abilities and critical thinking. Surely assessments which are based on their performance in a workplace or skills they will need at work after graduating are just a distraction?

Innovative assessment is designed to develop critical thinking including the ability to be self-critical. Research suggests that traditional assessment methods tend to push students away from critical thinking and towards a more superficial approach to learning (Boyd and Cowan 1985). With an increasingly competitive market for graduate employment, most students are very keen to get some valuable work experience or at least develop some vocational skills during their degree courses which they can offer to a prospective employer. Furthermore, whether we like it or not, the government now puts considerable pressure on institutions to produce graduates who are not just educated but are also 'employable'.

8. If there is too much assessment during a course, even if it is innovative, won't students be overburdened with work and tend to take a surface approach to their studies?

Innovative assessment doesn't necessarily mean more assessment. Unlike traditional forms of assessment, innovative assessment does not cram all the assessment in at the end of the course. Innovative assessments are usually staged more regularly over the academic year, meaning the pressure on the student is much less intense. This does not mean that the students are not under pressure, instead the pressure is more consistent over the duration of the course demanding a more disciplined approach on the part of the student. However, there is none of the last minute cramming, stress and panic caused by a few days of summative exams, which can decide one's future!

The quality of the assessment is more important than the quantity. If learners are overburdened by seemingly meaningless assessments, over which they have very little control, then it is likely that they will tend towards a surface approach to their learning. If, on the other hand, the learner feels a sense of ownership and control over what they need to learn, and an appreciation of what they actually have learnt together with an understanding of why they have learned, then they are less likely to resort to a surface approach (Harris and Bell, 1990; Ramsden, 1992; Marton and Saljo, 1990).

9. Why should students have so much choice in relation to assessment? Surely the lecturers know best what it is students need to learn in the subject and they should have to be tested on that.

Although lecturers should arguably know best what it is that students need to learn, they do not always communicate this effectively to the student. When lecturers do communicate effectively what needs to be learnt, this is not always what they actually assess (Race, 1994). Finally if they communicate what needs to be learnt and assess it, they don't always tell the student what it is they have successfully learned and what it is they haven't. Arguably, the only person who really knows what has been 'learnt' is the learner. The question is: how do you reliably and fairly assess what they have learnt?

10. Is innovative assessment fair? Often it can't be anonymous and marking is highly subjective: take the case of the black student barristers failing on video assessments although they passed the written tests.

The forms of discrimination one might find in higher education institutions will not be eradicated by using one assessment practice rather than another. Although in some cases these problems may have become institutionalised, ultimately the responsibility rests with individuals: it is not the practice of assessment that is racist or sexist it is the assessor(s). While non-anonymous marking does provide the potential opportunity for abuse by certain individuals, anonymous marking will not solve the problem of discrimination in higher education. It is by no means suggested that innovative assessment practices are a panacea for all forms of discrimination but it is hoped that by intentionally fostering a climate of openness, trust, democracy and empowerment, and by making assessment criteria more explicit and open to negotiation, innovative assessment should, at least in theory, confront discriminatory practices where they still exist.

Although many educationalists believe innovative assessment to be a good thing we have to concede that there is, as yet, little research evidence that indicates these benefits are realised in practice. What follows is our collective thoughts on why we feel innovative assessment methods are a good thing for higher education. Some of these thoughts are at present grounded more upon theory than empirical evidence, and whereas they may well be the intended outcomes of innovative assessment, they do not necessarily always represent the actual outcomes.

1.    By incorporating a range of different methods innovative assessment assesses a broader range of skills and as such it is considered to be fairer and less discriminatory. Consequently, innovative assessment should have the effect of widening access to Higher Education and perhaps widening success.

2.     Innovative assessment is more reliable assessment of student learning because it is not dependent on any one method of assessment. Innovative assessments allow for the fact that all individuals have strengths and weaknesses, by assessing an individual's performance across a range of skills a more balanced and reliable assessment can be obtained. Don't put all your eggs in one basket!

3.     Innovative assessment on the whole adopts a more positive approach to education; by spreading the assessment net more widely, it provides students with a range of opportunities to demonstrate how much they understand (Ramsden, 1992), rather than the somewhat negative approach of how little.

4.     Innovative assessment is usually formative and as such is more likely to facilitate effective, well motivated student learning. Providing timely and constructive feedback allows misunderstandings to be detected and cleared up, and students are able to make improvements where necessary. This process helps maintain student motivation, enabling them to learn more steadily and fluently. If students genuinely don't know what they are doing wrong, as they are never informed, then this can lead to frustration and a loss of interest in the subject.

5.     A range of different techniques and methods should stimulate both staff and student interest. Variety is the spice of life!

Innovative assessment usually means a number of different assessment techniques as opposed to the traditional tutor assessed exams and course work. Although innovative assessment is generally seen as a movement away from examinations, it is not necessarily an end to exams per se, but it is an end to exams in their traditional form. In an innovative form, exam questions may, for example, be revealed beforehand giving students time to prepare a well researched answer, or may emphasise the application of knowledge to a problem rather than simple memory recall. Unlike traditional assessment, innovative assessment does not rely on exams as a measure of the students ability, it involves a range of methods and utilises many different media, including: essays, seminars, projects, role plays, simulations, group work, problem solving, presentations, work placements, portfolios, reflective diaries etc.

The assessment itself may also be done by the tutor, by the students, or even by an outsider, such as an employer. Innovative assessment also encourages the assessment of learners not just as individuals but also as individuals working in groups, and as groups of individuals.

6.     Students learn and are assessed upon a much greater variety of skills and in a number of different situations. This should produce more rounded and more employable graduates. Assessments may include students demonstrating that they 'know how to' rather than just 'know about' (Race, 1994).

7.     Innovative assessment methods are usually more realistic and relevant, involving role plays, simulations and work placements; students develop a better understanding of how their specific skills and knowledge can he applied both inside and outside the academic environment.

8.     Innovative assessment is generally regarded as a possible strategy for facilitating a 'deep' rather than a 'surface' approach to learning (Marton & Saljo, l990; Boyd & Cowan, 1985; Ramsden, 1992). Brown & Dove (1993) consulted staff from four different universities who were already using self and peer assessments, the staff reported: 'students using higher levels of reflection, developing a questioning and self analytic approach to their professional practice and engaging in deep rather than surface learning.' (Brown & Dove 1993:3)

In this guide we have tried to provide an introduction to the concept of innovative assessment and the ideas which it inculcates. While we would be among the first to concede that innovative assessment techniques are not without their problems and certainly require careful implementation if they are to avoid the 'famous failure' label, we do feel that the arguments in favour of changing the way we assess more than outweigh the potential problems. The case we have made for a more innovative approach to assessment has been primarily on the grounds of improving the quality of student learning. Although, assessment is by no means the only method by which student learning can he enhanced, unlike teaching it is often viewed as something quite separate and distinct from the learning process; something that is done to students at the end of their courses, a way of testing what they know or don't know before rubber stamping their degree certificates. We believe assessment should be viewed as an integral part of the teaching and learning process. Learning rather than assessment should be the end product of higher education. Students should learn through assessment not learn to be assessed.

A second guide in this series Assessment: an innovators guide will provide a more detailed description and critique of the various techniques and methods commonly referred to as innovative assessment practices, providing references to more detailed background literature where possible. By drawing on the experiences of some of our more innovative colleagues and the news of students at the sharp end, we have also tried to make the guide of practical use to those about to embark on the road to innovation.

BOOTH, A. Learning history in university: student views on teaching and assessment, Studies in Higher Education, 18(2), l993, p227-235.

BOUD, D. J. The role of self-assessment in student grading, Assessment and Evaluation in Higher Education, 14(1), 1989, p20-30.

BOUD, D. J. Assessment and the promotion of academic values, Studies in Higher Education, 15(1), l990, pl01-111

BOYD, H. & COWAN, J. A case for self-assessment based on recent studies of student learning, Assessment and Evaluation in Higher Education, 10(3), 1985, p225-235

BROWN, S. & DOVE, P. Self and peer assessment: learning from experience. Natfhe Journal, l993.

BROWN, S. & KNIGHT, P. Assessing Learners in Higher Education, Kogan Page, London, 1994.

HARRIS, D. & BELL C. Evaluating and Assessing for Learning, Kogan Page, London, 1990.

HERON, J. Assessment revisited, IN BOUD, D. (Ed) Developing Student Autonomy in Learning, Kogan Page, London, 1981.

HEYWOOD, J. Assessment in Higher Education, John Wiley & Sons, New York, l989.

MARTON, F. & SALJO, R. Approaches to learning, IN MARTON, F., HOUNSELL, D. & ENTWISTLE, N. The Experience of Learning, Scottish Academic Press, Edinburgh, 1990.

ORPEN, C. Student versus lecturer assessment of learning: A Research Note, Higher Education, 11, 1982, p.567-572.

RACE, P. Never Mind the Teaching Feel the Learning, SEDA paper 80, Birmingham, 1994.

RAMSDEN, P. Learning to Teach in Higher Education, Routledge, London, 1992.

ROWNTREE, D. Assessing Students: How shall we know them? Harper and Row, London, 1977.

WONDRAK, R. Using self and peer assessment in advanced modules, Teaching News, 1993, p22-23.