En la presentación disponible de Nian Liu, miembro del equipo de la universidad Jiao Tong que elabora uno de los rankings, se explican los criterios usados en su caso (se pueden consultar también en el mismo sitio de la Universidad), sino que se valoran los problemas de la clasificación:
Education is the basic function of any university, however, it would be impossible to rank the quality of education due to the huge differences among the national systems.Liu menciona algunos problemas propios de encuadrar un material tan variado como lo son las instituciones universitarias:
Contribution to the national economic developmentis becoming increasingly important for universities, however, it is impossible to obtain internationally comparable indicators and data.
(sin embargo...) The academic or research performance of universities,a good indication of their reputation, can be ranked internationally.
(...) Many well-known institutions specialized in humanities and social sciences are ranked relatively low.
(...) English is the language of international academic community.
(...) Any ranking based on academic performance will be biased towards institutions in English-speaking countries.
Possible solution: papers published in non-native languages are offered a special weight.
(...) Universities which started after 1911 do not have a fair chance.
(...) Disciplines not related to the awarding fields do not have a fair chance. Other important awards include Abel, Pulitzer, Turing, Tyler, etc.
(...) Institutions for winning awards and those for doing the researches may not be the same.
(...) Institutions for obtaining degrees and those for pursuing the studies may not be the same.
Postdoctoral training is not considered.
(...) The weight of the Size indicator for per capita performance is rather low. Large institutions have relatively high positions in the ranking.
However, it’s very difficult to obtain internationally comparable data on the number of academic staff.
The types of academic staff: such as purely teaching staff, teaching and research staff, purely research staff.
The ranks of academic staff: such as professor, associate professor, reader, lecturer, research scientist etc.
Many universities have more than one commonly used names: such as Virginia Tech and Virginia Polytechnic and State University.La respuesta de Liu a estos problemas está en su propia identificación: el localizar los problemas permite una mejor aproximación a la calificación.
Variations due to translation: such as Univ Kolnand Univ Cologne, Univ Vienna and Univ Wien.
Abbreviated names: such as ETH Zurich for Swiss Federal Institute of Technology Zurich.
Some authors only write their departmental or institute names without mentioning their university names.
University systems: such as Univ California system, Univ London system.
Affiliated institutions and research organizations: such as Ecole Polytechnique Montreal(affiliated to University of Montreal), CNRS Labs (affiliated to French universities).
Teaching and affiliated Hospitals: complex!
Merging, splitting, inheriting, discontinuing, name-changing of institutions such as:
Univ Kwazulu-Natal in South Africa merged from Univ Natal and Univ Durban-Westville.
University of Innsbruck in Austria splitted into Univ Innsbruck and Innsbruck Medical Univ.
Humboldt Univ Berlin and Free Univ Berlin inheriting the Nobel Prizes of the Berlin University before world war II.
Vrjie Universiteit Brusseland Universite Libre Bruxellesshare the same English name of Free University of Brussels.
Los materiales de la discusión se localizan en el sitio de la conferencia.