Measuring research quality in a more inclusive way: Evidence from the UK Research Excellence Framework

Abstract

Evidence suggests that common metrics of research quality—e.g. journal publications and citations—are systematically biased against certain groups. But does relying solely on them to evaluate quality lead to lower diversity in academia? In this paper, we start to answer this question by analysing data from the UK’s nationwide research assessment exercise, the Research Excellence Framework. We find that narrowly focussed output-based measures of departmental research quality do indeed negatively correlate with the diversity of departmental staff, while measures of research impact and of the quality of the research environment correlate positively. An aggregate measure that incorporates all three components is therefore likely to better promote staff diversity compared to more narrowly defined output-focused measures. More generally, our results suggest that comprehensive definitions of research quality may be more effective at promoting diversity in academia compared to narrower measures. We further argue that funding decisions informed by broader measures result in more efficient resource allocations across the higher education sector.