The mathematician Etienne Ghys comes back on the strange formula which presides over the establishment of the Shanghai ranking.
Carte blanche. The famous Shanghai ranking list of universities was published as every year in August. We learn that the top trio is made up, as always, of Harvard, Stanford and Cambridge, and that the universities of Paris-Sud and the Sorbonne occupy the 37th and 44th positions. This ranking is criticized from all sides, except of course by the universities that are well placed. It is perhaps useful to explain how it is constructed, to show how little sense it makes.
First, the ARWU (Academic Ranking of World Universities) assesses five “indicators” for each university. These are the number of Nobel Prize or Fields Medal winners who work there, the number of alumni who have received these same honors, the total number of published articles, those published in the two journals Nature and Science, and finally the number of “highly cited” researchers.
Each of these indicators is problematic. For example, the list of most cited researchers includes 90 mathematicians, 16 of whom sign their papers … in Saudi Arabia. On the other hand, there are no French mathematicians in this list. Without being chauvinistic, this makes no sense.
Of course, these five indicators favor the big institutions and leave little chance to the small ones, even if they are excellent. To try to remedy this, a sixth indicator is used, which is a sort of average of the previous ones, divided by the total number of researchers in the university.
As in the decathlon
The icing on the cake is the formula used to aggregate all this and make an overall ranking. The “score” assigned to a university is an average of the square roots of the six indicators, assigned certain coefficients. You read that right: it is an average of the square roots. To understand the idea, we can refer to the decathlon. How do you aggregate a sportsman’s results in ten disciplines as different as high jump and shot put? The solution is to first transform each of the ten performances in a certain way, specific to each discipline, before calculating averages. An improvement of 1 cm in the high jump will earn you many more points if you jump 2.45 m (world record) than if you jump “only” 1.50 m. For a university that already has a lot of Nobel Prize winners, however, it is easier to recruit one more than it is for a university that has none. In order to take this into account, the ARWU did not look very far and decided to transform all indicators in the same way and to use the square root.
There are at least two differences between academics and decathletes. Firstly, there has been much debate among athletes in the past about what a good formula should be. Nothing of the sort has occurred among academics, and the arbitrary choice of the square root is puzzling. On the other hand, a decathlete participates in a competition that he has freely chosen and for which he knows the rules. This is not the case of universities, which do not have the mission of following rules imposed unilaterally by a Chinese institute that promotes square roots.
The ARWU also establishes world rankings by discipline. I have of course consulted the one concerning mathematics. There you learn that Princeton is first, the Sorbonne is second, Paris-Sud is in fifth place, and the French department of mathematics that follows, in a very honorable 27th place in the world, is my laboratory at the Ecole normale supérieure de Lyon. In the end, these rankings are not so bad…