Call for 'time out'

The Star - Sunday May 9, 2010
Call for 'time out'

by Prof James Campbell

UNIVERSITY rankings seem notoriously fickle and wide open to dispute. Modern universities are diverse and complex places where a multitude of tasks and activities take place.

Some are easily measurable while others are more difficult to capture.

For critics of currently constituted university rankings, simply reducing this complexity to a number - university X is better than university Y - can be damaging.

The complexity of what different higher educational institutions offer to students, and the possibility that these institutions cater to students in different ways, is lost when we reduce this to a simple number in a rankings scale.

The notion that one can have such a precise and accurate rendition of a higher educational institution's value with reference to a ranking would be amusing, if not for the fact that the consequence of believing in such a measure is not so serious.

Dubious data

Finally, dubious data and even more dubious methods for collecting and assessing such data suggest that university rankings as currently constituted are open to serious doubt.

No better illustration of this issue can be found than in the current mea culpa put forward by Phil Baty, editor of the Times Higher Education (THE) World University Rankings, and deputy editor of Times Higher Education (THE) Magazine.

Baty's article which appeared in StarEducation last week under the heading “Flawed Rankings”, was an apology.

It may have come as a shock to some, but for those of us who have studied and debated the issue of university rankings over the years, the sentiments and information put in by Baty's article comes as no surprise.

He went on to point out that, “Those who have used our rankings to cast judgment on the state of Malaysian higher education (and many, in very senior positions have done so), must be told that the annual tables had some serious flaws - flaws which I have a responsibility to put right.”

Such honesty and forthright criticism is welcome and his openness has provided critics of the current rankings system with a real chance to make their case.

For example with respect to peer reviews as used by the Quacquarelli Symonds (QS) survey he argues: “QS achieved only a tiny number of respondents to this survey.

“In 2009, around 3,500 people provided their responses – a fraction of the many millions of scholars throughout the world.”

Given the critique presented by Baty, there is a temptation for us to accept on face value the claim that the new Thomson Reuters survey instrument which THE plans to use will be “a massive improvement” over the QS survey instrument.

However, we should consider the response by Nunzio Quacquarelli of Quacquarelli Symonds (QS) to the criticisms offered of his methodology.

Quacquarelli writing in an article in the QS Top Universities website claimed that: “THE consistently praised the QS methodology throughout the six-year publishing collaboration.

Quacquarelli also interestingly points out the institutional shift that THE has made in changing from the QS system, which draws its data from with the Scopus database of Elsevier, to its new partner, the Thomson Reuters' academic citation database.

New ammunition

It seems that critics of THE are gaining new ammunition in their understanding of the limitations of rankings by the public dispute that has now erupted between THE and QS.

Quacquarelli in another article in the QS Top Universities website argues that “it seems that THE believes the only way to legitimise producing its own new rankings is to pretend dissatisfaction with QS”.

Are we seeing a business dispute and fight over market position, or a true argument over legitimate methodology? What are we to make of it all?

Perhaps rather than blindly accepting a new rankings system which may or may not “be a massive improvement”, we ought to take time to ponder why we follow rankings in the first place, and of what use they are to us.

In the Malaysian context, we need to ask ourselves: what is it that we want Malaysian higher educational institutions to do?

The extent to which rankings help Malaysian higher educational institutions reach their goals may be of some use.

However, we ought to pause and reflect the extent to which rankings produce behaviours that are counter-productive to institutional goals and interests.

Indeed, the confusion and uncertainty that surrounds current rankings, and the strong reservations that accompany our understanding of rankings, raises a few questions.

Is it not time for institutions of higher education to take charge of this debate?

Do we need “time out” from the current rankings discourse?

Should we not consider some breathing space to let rankings agencies get it right, rather than continuously participate in a system that from its most articulate advocates is deeply flawed?

Should we consider approaches more in keeping with the values, needs and interests of Asian universities such as the Alternative University Appraisal System, which is being developed and implemented by among others the Asian Institute of Technology in Thailand, TERI University in India, United Nations University-Institute of Advanced Studies, Universiti Sains Malaysia (USM) and Yonsei University (Korea)?

It is clear that the critics of rankings, and in particular the critics of the THE rankings have been vindicated.

The question before the higher educational institutions in Malaysia is whether they rush headlong into accepting the new THE ranking game, or call for time out and insist that if rankings agencies want Malaysian participation, they should prove if their methods are valid and if they fit the Malaysian context.

The opportunity for Malaysian institutions to engage and lead this debate has been provided by the revelations of the THE and QS. It is an opportunity that may not present itself again.

Prof James Campbell who is from Deakin University in Australia, is a visiting researcher at Universiti Sains Malaysia's Centre for Policy Research and International Studies.

Source: Call for 'time out'