Mixed reaction to RAE proposals

Next week sees the opening of a 4-month consultation period during which higher education institutions in the United Kingdom will have chance to respond to Gareth Roberts' report, published last week, on the future of research assessment and funding allocation in the United Kingdom.The opening event is a conference organized by Universities UK, the Higher Education Policy Institute (HEPI), and the Standing Conference of Principals. Margaret Hodge, minister of state for lifelong learning and higher education, will be the keynote speaker and is widely expected to reaffirm the government's intent to concentrate research in fewer institutions and to grant university status to teaching-only institutes.This tendency toward concentrating research in centers of excellence is one of the main themes to be reflected implicitly in the recommendations of Roberts' report.The report was commissioned by the four UK funding councils with the aim of appraising the Research Assessment Exercise (RAE), by means of which funding is currently allocated. Its main recommendations are to retain peer review of research institutions; to have a 6-year review, with the next review in 2007–2008; to have an institution-wide review of research strategy 2 years before the main review; to vary the review process according to how much public money the institution is seeking; to have fewer but more broadly based subject assessment panels; and to assess individual researchers. "Retention of peer review is welcome," said Michael Whitaker from the School of Molecular and Cellular Biosciences at Newcastle University. "Many of us were concerned that to reduce the administrative burden of the RAE, the report might recommend assessment on the basis of, say, grant income or citation indices only." Though attractive at first sight, reliance on such performance indicators, concludes the Roberts report, would not attract widespread support from academia, even if it did reduce the administrative burden.Though there seems to be consensus on the issue of peer review, it is not clear how the report's other recommendations will be received. Mired as they are in the exam season, few academics seem to have considered the report in detail. Nevertheless, the main recommendations are already prompting questions about what they will mean in practice.Take the idea of different assessment processes. This means that those with international reputations seeking the most money will undergo the most rigorous review, but will stand to lose the most. Those undertaking research of national significance will get less money, but will undergo a simpler review. Some may choose to opt out of the system entirely. It's not clear whether the effort of getting this idea accepted will be worth the difference it will make," said Bahram Bekhradnia, director of HEPI.The report's most radical ideas, said Bekhradnia, are the assessment of individual researchers and the introduction of fewer, more broadly based subject panels. These ideas are meant to address some of the main concerns that prompted the Roberts review in the first place—namely, that the current process can lead to massive drops in funding from one RAE to another and that it is difficult to know whether the same rigor is being applied in physics as, say, in history, philosophy, or biology.The sudden drops in funding can affect a university's cash flow to the tune of millions of pounds. How the system worked is Kafkaesque and byzantine, said Whitaker. Briefly, departments have under the old system balanced how many people they submitted for assessment against their international standing as researchers. To get the maximum income, each "unit of assessment" (which can include academics from different departments) needs to ensure that it has 50% (not a figure spelled out explicitly) or more people doing internationally competitive work. If this percentage can be achieved for a unit of 100 people, it gets more money than a unit with the same percentage that contains only 80 people. If the balance is incorrectly gauged, and the percentage drops below 50%, the unit loses a grade and the income to the university can drop substantially.The new system is intended to smooth the funding curve so that institutions will not have to manage such abrupt disruptions. It will rate each individual's work (though keep the assessment confidential) and, in effect, produce an aggregate score for each unit, which will allow a more accurate comparison among units from different institutions. As a result, the funding councils will be able to fine-tune their block grants.Mike Shipston, a physiologist at the University of Edinburgh, sees the value of the approach but questions whether confidentiality is such a good idea. He wonders whether that might not obscure hot spots of research. Under the current system is it possible to flag sub-groups that are doing internationally rated research, even if they are not in a unit that overall does not score so highly.Addressing the question of whether subject panels compare like with like seems set to be an equally vexed question. The solution proposed in the report is to have fewer, more broadly based subject panels, each supported by a greater number of expert subpanels than at present. Whitaker remains unconvinced that this will overcome the charge of comparing apples with pears, but he points out that it allows for shifting disciplinary boundaries in dynamic academic areas like biology.Given the questions already arising, how the Roberts report will look after the academic community has read more than the executive summary is anyone's guess. There are 4 months to dissect its elements, but what seems sure is that some form of research assessment is here to stay.

Mixed reaction to RAE proposals

Consultation begins on proposals to shake up UK's approach to research assessment

Interested in reading more?

Become a Member of