National Seminar in Edinburgh, 27 MARCH 2007

Benchmarking and Peer Reviewing e-Learning in Higher Education: Institutional Opportunities and Challenges

A joint seminar hosted by the University of Edinburgh, the Higher Education Academy & the EC MASSIVE (Peer-review in e-learning) Project.

E-learning in higher education is now a well-embedded area of activity for most HEIs in Europe, with significant money and resources devoted to its provision and expansion. The maturity of the field has led recently to questions at the level of senior staff, practitioners and policy makers as to how to construct measures that can be used to define levels of maturity or development, and which can be used to analyse 'progress' towards desired goals. In the area of learning and teaching in UK higher education, peer reviewing has been used to assess quality of provision (eg quality assessment, accreditation), and formal and informal benchmarking methods have been developed to assist with the task of comparison between universities in service provision and research. No single method offers the optimum approach to all.

The aim of the seminar was to offer an opportunity to share perspectives, experiences, affordances, and constraints associated with the varying approaches to benchmarking/baselining of e-learning that are now being employed in the European HE sector. The event provided an opportunity to interact with people with practical experience of implementing large scale benchmarking and peer review of e-learning initiatives and with those actually implementing a variety of methodologies in HE institutions.

AGENDA

start	end	topic	speaker
09.15	09.45	Registration & coffee
09.45	10.00	Opening & Scene Setting	Jeff Haywood, University of Edinburgh
10.00	10.30	The UK HE Benchmarking Exercise	Derek Morrison, Higher Education Academy & Terry Mayes, University of Glasgow Caledonian
10.30	11.00	The Leicester Experience: the Adelie Project	Jaideep Mukherjee & Alejandro Armellini, Leicester
11.00	11.30	Experience of the OBHE Methodology at Coventry University.	David Morris, University of Coventry
11.30	11.45	Break
11.45	12.15	Experience of using the MIT90s Framework at the University of Strathclyde	Michael Coen, University of Strathclyde
12.15	12.45	The Manchester Experience: EMM	Jim Petch, University of Manchester
12.45	13.00	Discussion
13.00	13.30	Lunch Wolfson Room, Main Library
13.30	14.15	The EU 'MASSIVE' Project - Peer Review in e-Learning	Joe Cullen, Tavistock Institute & Jeff Haywood, University of Edinburgh
14.15	15.30	Round tables Groups
15.30	15.45	Break
15.45	16.30	Plenary - drawing together the themes of the day	Jeff Haywood, University of Edinburgh
16.30		Close

Roundtable topics

- Benchmarking or reviewing complex institutions - what gets missed, what gets over-attention, how can a rounded picture be achieved?
- How often do we need to benchmark or review? Is it a one off or a rolling process? Which gives an institution the best value?
- Can we convert what we 'feel instinctively' ie tacit knowledge about the state of e-learning in an institution into objective criteria?
- Can we agree on what we are we really benchmarking when we benchmark e-learning? (eg can e-learning be benchmarked separately from pedagogy?)
- How do we measure the real quality of the learner experience? (ie can we benchmark outcomes rather than inputs?)
- Does benchmarking lead to uniformity? Does peer review work against innovation? ('There is a remarkable sameness about e-learning development across the HE sector').
- Is it the process of asking probing questions that is important, rather than the detailed methodology? ('The importance of benchmarking methodologies is overemphasised').