11 Sep Tempest in the rankings teapot – An African perspective
It is that season when ranking entities announce their ‘findings’ on the comparative stature of the world’s universities. As usual, the ‘premier’ universities remain at the top and the rest are relegated to the bottom – African universities in particular. The ‘rankers’ go about their business, some with audacity, but too often without sufficient concern for veracity, authenticity or integrity in their methodologies and, especially in the case of Africa, without sufficient data.
Facts vs perceptions
For the last three years, the University of KwaZulu-Natal in South Africa has been the first in the country in academic productivity, as measured by the Department of Higher Education and Training. The department undertakes the task of ranking using parameters that meticulously measure research and academic outputs.
Yet, according to the QS World University Rankings released in June – which allocates 60% of the criteria to academic reputation – the University of KwaZulu-Natal now stands below six other South African universities. This points to a glaring tension between data and dubious assessment based on reputation.
Building reputation – Unpacking the numbers
The QS ranking is a mix of survey responses and data across six indicators, compiled and weighted to formulate a final score. It claims that over 70,000 academics and 30,000 employers contribute to the rankings through the QS global surveys. QS states that it analyses 99 million citations from 10.3 million papers before 950 institutions are ranked.
The Times Higher Education, or THE, states that their methodology is a unique piece of research that involves “questionnaires [that] ask over 10,500 scholars from 137 countries about the universities they perceive to be best for teaching and research”. It claims that the Academic Reputation Survey “uses United Nations data as a guide to ensure that the response coverage is as representative of world scholarship as possible”.
THE goes on to state that where countries were over- or underrepresented, the responses were weighted to “more closely reflect the actual geographical distribution of scholars”, throwing more uncertainty on the changing parameters of the rankings.
There appears to be a conflation between ‘world of scholarship’ and ‘geographical distribution of scholars’, without any clear definition of ‘scholar’ or ‘scholarship’. China, India and Brazil may have the largest number of ‘scholars’ and by that account more scholarship, yet they barely make it to the top in the rankings.
According to THE, only 2% of the survey participants are Africans, presumably located on the continent. As about 50% of research in Africa is undertaken in South Africa, one may presume that the number of survey participants in the rest of Africa tapers off to 1%.
Around 100 academics in Africa, then, outside of South Africa, participated in the reputation index “evenly spread across academic disciplines”. Thus, for the 11 disciplines considered in the THE rankings, that would mean about 10 responses per discipline from Africa. A similar problem is presented in the Latin American and Middle Eastern contexts, which see survey representation of 5% and 3%, respectively. Continue Reading ….http://www.universityworldnews.com/article.php?story=20170905085216591