Categories
Contents Highlights Technical Reviews

The Crises of 2020 and international comparison indicators

As an epoch changing rupture event in human society, we can expect that almost all areas of the science system will be in some way affected by covid-19. These disruptions will lead to new challenges, new solutions to some challenges and newly impoverished actors who suddenly find themselves severely tested by the new environment.

As an epoch changing rupture event in human society, we can expect that almost all areas of the science system will be in some way affected by covid-19. These disruptions will lead to new challenges, new solutions to some challenges and newly impoverished actors who suddenly find themselves severely tested by the new environment.

We should expect some major shifts in the norms of international comparison in the next year. Some institutions that are responsive and active will increase their profile very rapidly, others will find themselves in serious trouble. We expect a range of indicators that will very possibly change – some will no longer make sense, while others will need to be introduced to make sense of the new landscape. This short report will suggest where Brazilian universities should look to position themselves in the best way possible.

Research

The crisis has seen an extraordinary explosion in publications – the scientific community is engaged in the largest and most intense collective challenge of the modern era. A recent search of the Dimensions database – chosen for its inclusion of pre-prints and other repositories that publish faster than peer reviewed journals – returned around 102,000 results, this almost a third of all scholarship ever published on pandemics. Among these, there are 532 datasets, 955 grants, 15,000 patents, 1,600 clinical trials in operation, and citations in over 2,000 policy documents. This explosion of productivity will reshape the research landscape in coming years, with universities organising quickly into producing new knowledge through new means. In that respect, initiatives such as Unicamp’s open innovation platform should situate it well in the new landscape.

Interest in social and economic impact of research has finally begun to cross from scientometric study into some national evaluations, and the interests of ranking agencies in recent years. The Times Higher Education Impact ranking is a clear indicator of this direction of travel. Society’s need for science has perhaps never been clearer in public understanding than at the present moment – almost every citizen minimally aware of the size of the challenges ahead are also aware of the need for scientific progress and research. 

Citation analyses, usually used to judge the scientific output of universities, are too abstracted from this real impact to be used alone to judge the scientific performance of universities. Therefore, we would expect that other indicators to become much more prominent in future rankings, as well as more attention to the priniciples of open science. Consideration of open datasets, clinical trials, policy papers and other productions will become important indicators of a university’s performance in rankings. Universities should be taking steps at least to map and measure these other types of production, and other types of impact. The crisis will greatly hasten this direction of travel.

Reputation 

For the rankings based on reputation scores (specifically, the Times Higher Education and the QS), it would be reasonable to expect much more volatility than usual. Some universities will come out as huge winners; Johns Hopkins University for their coronavirus dashboard, and Imperial College London for their curve calculations, for instance, while others who have gone “missing in action” will likely suffer dramatic falls. We would expect that those who take a strong institutional communications line on coronavirus to fare better than those who do not, so the communication of research and outreach activities will become much more important, as will appearances in international media. The Altmetric impact of research must be monitored closely, as should appearances by university staff in international sources as well as local.

Because most scientific conferences will be cancelled this year – or held online to much lower attendances – the chance to interact with science  outside of a researcher’s immediate network will be diminished. We should therefore assume that the strength of existing international links will be more important in reputation than ever before – these interactions should be prioritised.

The communication that universities make of their research results is of paramount importance, and a parallel press office that communicates these stories in English should be considered as an important means of maintaining the university’s profile when physical interaction is cut off. There are a variety of best practices we have identified in university communications strategies, and of the universities we think are managing this best.

Financing

Many of the world’s elite universities, especially those dependent on student fee income – predominantly those in the United States and the United Kingdom will find a hole in their finances running to hundreds of millions of dollars. For Ivy League and Oxbridge institutions, this will be covered in some way by endowment funding and donations – national flagships are never allowed to sink. Many middle ranking universities with whom the São Paulo state universities share ranking positions are facing serious and urgent questions of survival if the school year is not able to begin as normal in September.

While the state universities should rightly be concerned about the incoming large contraction in the ICMS, which will place the universities in even more financial misery, their medium term futures will be better assured that mid-sized Anglophone universities. Their financial models have become very dependent on the turnover of students, and particularly on the recruitment of international students. While careful navigation will be required to steer the state universities through this, they will not be looking at the need to restructure the entire financing of higher education. This is predominantly thanks to the free at point of access model over the student fee model, which suddenly appears extremely vulnerable.

Brazilian universities must still seek to diversify the sources of their funding, opening donations to their foundations and expanding endowment and alumni drives. Things will not be easier than before, but there are other models that will suffer more than Brazil. This gives them a relative advantage of more breathing room to focus on other priorities.

Internationalisation

One area in which the universities struggle in rankings is the proportion of full degree students from abroad. This indicator, purports to measure academic mobility and international prestige. In reality, it is a measure of universities’ ability to attract undergraduates and taught postgraduates from abroad in order to pay higher fees for the global ‘brand’ attached to a prestigious university. 

The numbers for this in 2020 will be close to zero – it will not be possible to accurately measure this in this year’s set of rankings, we would not expect it to be included at all.

As for future years, it is unclear whether this sort of student recruitment that depends on very large numbers from China, India and the Middle East will return to previous patterns. Provision in these countries has grown enormously in both extension and quality over the past decade, this may merely accelerate a process already in progress. History suggests that sudden changes in social behaviour in response to shocks does not automatically return with the resumption of normal – we should not assume that the Chinese, Indian and Middle Eastern middle classes will still all choose to send their children to foreign universities, outside of those with the kudos of the Ivy League or Russell Group.

Science will remain as internationalised as ever, possibly more so in confronting unprecedented global health, economic and environmental challenges. Return to normal will probably see a rapid return to academic mobility – visiting PhDs, post-docs and professors will continue to operate within research networks. The future of mass recruitment of Chinese MBA students looks far more uncertain.

For the future we would expect indicators of international coauthorship to take a more prominent role in university rankings, rather than recruitment. Therefore the universities should ensure that these networks are maintained and expanded.

Online presence

Currently, the only ranking to measure the online presence of universities properly is the Webometrics ranking, which appeals mostly to a specialist audience. Given that universities currently exist almost entirely online, monitoring of the indicators involved will be of crucial importance in measuring the university’s performance amid the crisis. Specifically, we suggest that monitoring the indicators “visibility” and “transparency” would be advisable. Visibility is a measure of the number of pages registered to the university’s domain. Following this would suggest how much the university expanded digitally during the crisis, even if not for specifically crisis-related reasons. This figure is given by Google. 

The second will likely have a large impact on international reputation and visibility. It is the number of backlinks that connect a different domain to the university’s domain. For example, when the New York Times uses Johns Hopkins’ coronavirus dashboard, and includes a link, this link is counted towards the total. These backlinks form the basis of Google’s PageRank algorithm, and broadly measure how far the university has permeated the internet outside of its own domains. The sources used by Webometrics for this are Ahrefs and Majestic. It should be noted, however, that there is no measure of the number of times these links are used.

Furthermore, universities should be attentive not to their totals, but the move from pre-crisis to post-crisis.

Conclusion

There are some aspects of this crisis that will put São Paulo state universities in positions of relative (if not absolute) advantage – the fall in internationalisation and sudden deep financial insecurity provoked by running commercial tuition fee models, for example. There are others that put them at a serious disadvantage unless steps are taken to anticipate probable changes and demands placed upon them. The following four issues should be the focus for universities:

  • The importance of an effective communication strategy of contributions, results and impacts, in both Portuguese and English.
  • A systematic understanding of how the university’s researchers are interacting in global research networks, and an ability to encourage them.
  • Monitor the evolution of both the size of the university’s online presence, and how far it permeates other sources.
  • The need to measure and record a wider variety of research outcomes, both in terms of the type of result, and the type of impact.
  • Ensure correct institutional attribution of all research publications with ORCID, including in pre-prints and proceedings.