CHEPS/European Commission
U-Multirank is a comprehensive benchmarking tool for universities produced by CWTS. Instead of aiming to rank and find the ‘best’ university from a global list, it instead aims to offer comparative benchmarking of performance from user defined criteria based on location, size and institutional focus. This means that it preserves most of the largest benefits to be derived from rankings; increased comparability of institutional performance, greater transparency in governance and data reporting, without creating the unhealthy competitive incentives for universities.
U-Multirank depends heavily upon two other separate but related initiatives; U-Map which utilises a much more restricted set of indicators, which USP and UNICAMP still use, while UNESP has chosen to adopt the full range of indicators available to U- Multirank, and the CWTS Leiden Ranking, from which it gathers its external bibliometric information.
Like other benchmarking tools, U-Multimap is committed to the idea of overcoming the idea of a singular model of excellence, recognising that a strong higher education system requires many different forms of excellence. This approach allows U- Multirank to take a much wider variety of metrics into account than a composite ranking, without creating heavily aggregated scores of dubious descriptive power, compared to the Times Higher Teaching score, for example, which combines five different factors to produce something wherein universities with very different profiles
Compiling Team and Financing
U-Multirank is developed and implemented on the initiative of the European Commission by an independent consortium led by the Centre for Higher Education (CHE) in Germany, the Center for Higher Education Policy Studies (CHEPS) at the University of Twente and the Centre for Science and Technology Studies (CWTS) from Leiden University, both in the Netherlands. The consortium is headed by Professor Dr. Frans van Vught of CHEPS and Professor Dr. Frank Ziegele of the CHE. Associate partners include specialist organisations (e.g. on data collection, web design), national ranking organisations and stakeholder organisations.
Lead Partners
- Centre for Higher Education (CHE)
- Center for Higher Education Policy Studies (CHEPS)
- Centre for Science and Technology Studies (CWTS)
Partners
Bertelsmann Stiftung Folge 3 Push
Associate Partners
- Fundación CYD
- Perspektywy Education Foundation
- Conference of European Schools for Advanced Engineering Education and Research (CESAER)
- European Student Union (ESU)
- Universities of Applied Sciences Network
- StudyPortals
- Erasmus Student Network (ESN)
The project is financed by seed funding from the European Commission’s Erasmus+ programme.
Specificity
Given the vast range of indicators available, and the ability for universities to report as much, or as little, as they choose and the ease of comparing and benchmarking this against nearly 1000 universities worldwide, U-Multirank can provide vital insight into a university’s performance in a clear way. While it does not exert the kind of influence that the major rankings do, in part because of its failure to generate large headlines and in part because of its comparatively low level of recognition, universities would be well advised to participate in the initiative for the ease with which comparisons can be made with international partners. U-Multirank can be a useful tool for understanding other rankings, as well as giving a clearer picture of the university’s performance than the rankings themselves.
Full U-Multirank Methodology and Metrics
| Area | Metric | Description |
|---|---|---|
| General | Total number of students | Total number of students in the degree programme. |
| General | Students in their 1st year | Number of first-year students in the degree programme. |
| General | International students | Number of international students in the degree programme. |
| General | Total number of students in field (major) | Total number of students taking the subject in the department, not including minor subject students. |
| General | Percentage of female students | Percentage of female students enrolled at the department. |
| General | Academic staff (fte) | Number of full-time equivalent academic staff at the institution. |
| General | Period of study | The normative period of study for the degree programme (years). |
| General | Female students bachelor | The number of female students enrolled in bachelor programmes as a percentage of total enrolment in bachelor programmes |
| General | Female students master | The number of female students enrolled in master programmes as a percentage of total. |
| General | Female academic staff | The number of female academic staff as a percentage of total number of academic staff. |
| General | Female professors | |
| Teaching and Learning | Student - staff ratio | The number of students (headcount) per member of the academic staff (fte). Staff solely involved in research is excluded. |
| Teaching and Learning | Graduating on time (bachelors) | The percentage of graduates that graduated within the time expected (normative time) for their bachelor programme. |
| Teaching and Learning | Graduating on time (masters) | The percentage of graduates that graduated within the time expected (normative time) for their masters programme. |
| Teaching and Learning | Academic staff with doctorates | The percentage of academic staff holding a doctorate (PhD or equivalent). |
| Teaching and Learning | Contact with work environment (bachelors) | A composite measure representing at bachelor level: (1) the inclusion of internships / phases of practical experience or external projects in the curriculum; (2) the percentage of students doing an internship; (3) teaching by practitioners from outside the university departments; and, (4) the percentage of degree theses made in cooperation with industry/external organisations. |
| Teaching and Learning | BA graduation rate | The percentage of new entrants that successfully completed their bachelor programme. |
| Teaching and Learning | MA graduation rate | The percentage of new entrants that successfully completed their master programme. |
| Teaching and Learning | BA graduates in normative time | The percentage of graduates that graduated within the time expected (normative time) for their bachelor programme. |
| Teaching and Learning | MA graduates in normative time | The percentage of graduates that graduated within the time expected (normative time) for their masters programme. |
| Teaching and Learning | Relative BA graduate unemployment | The percentage of bachelor graduates unemployment 18 months after graduation. |
| Teaching and Learning | Relative MA graduate unemployment | The percentage of master graduates unemployment 18 months after graduation. |
| Teaching and Learning | Contacts with work environment | A composite measure representing at bachelor level: (1) the inclusion of internships/phases in work; and (2) the percentage of students doing an internship; and (3) teaching by practitioners from outside university departments. |
| Teaching and Learning | Contact with work environment (masters) | A composite measure representing at bachelor level: (1) the inclusion of internships / phases of practical experience or external projects in the curriculum; (2) the percentage of students doing an internship; (3) teaching by practitioners from outside the university departments; and, (4) the percentage of degree theses made in cooperation with industry/external organisations. |
| Teaching and Learning | Graduates in normative time | Percentage of graduates that graduated within the normative time to degree for their programme. |
| Teaching and Learning | Relative rate of graduate unemployment | Rate of unemployment of graduates 18 months after graduation, relative to the national graduate unemployment rate. |
| Teaching and Learning | Hospital beds available for teaching | The number of beds available for teaching in university hospital and affiliated hospitals per 100 students. |
| Teaching and Learning | Innovative forms of teaching and assessment | The percentage of examinations (in medical doctor training programmes) which use innovative forms of assessment (assessment of practical work by faculty and strctured clinical cases). |
| Teaching and Learning | Graduation rate long first degree | The percentage of new entrants that successfully completed their long first degree programme. |
| Teaching and Learning | Graduating on time (long first degree) | The percentage of graduates that graduated within the time expected (normative time) for their long first degree programme. |
| Teaching and Learning | Relative graduate unemployment long first degree | The percentage of long first degree programme graduates unemployment 18 months after graduation. |
| Teaching and Learning | Innovative forms of teaching and assessment | The percentage of examinations (in medical doctor training programmes) which use innovative forms of assessment (assessment of practical work by faculty members and strctured clinical cases). |
| Teaching and Learning | Community service learning | The percentage of credits given in service-learning activities, in relation to total number of credits. Service-learning involves students in community service activities and applies the experience to personal and academic development. |
| Teaching and Learning | Community service learning | The percentage of credits given in service-learning activities, in relation to total number of credits. |
| Teaching and Learning | Overall learning experience | An assessment of the quality of the overall learning experience, based on a survey of the students. |
| Teaching and Learning | Quality of courses & teaching | An assessment of the quality of teaching provision, based on a student satisfaction survey. |
| Teaching and Learning | Organisation of program | An assessment of the organisation of the programme, based on a student satisfaction survey. |
| Teaching and Learning | Contact with teachers | An assessment of the feedback given by teachers, based on a student satisfaction survey. Inclusion of work/practical experience An assessment of the inclusion of work experience and of elements related to work practice, based on a student satisfaction survey. |
| Teaching and Learning | Library facilities | An assessment of the quality of library services for students, based on a student satisfaction survey. |
| Teaching and Learning | Laboratory facilities | An assessment of the quality of laboratories available to students, based on a student satisfaction survey. |
| Teaching and Learning | IT provision | Student assessment of the quality of IT services for students, based on a student satisfaction survey. |
| Teaching and Learning | Room facilities | An assessment of lecture halls and seminar rooms, based on a student satisfaction survey. |
| Teaching and Learning | Linking clinical/preclinical teaching | The integration of pre-clinical/theoretical and clinical courses, based on a student satisfaction survey |
| Teaching and Learning | Skills Labs | An assessment of the skills labs and training centers concerning maintenance, accessibilty, technical facilities and mentoring, based on a student satisfaction |
| Teaching and Learning | Bedside teaching | An assessment of bedside teaching concerning mentoring, suitability of rooms and variety of diagnostic techniques applied, based on a student satisfaction survey. |
| Teaching and Learning | Inclusion of practical experience/clerkships | The integration of practical experience with patient contact into the study programme, based on a student satisfaction survey. |
| Research | External research income | Research revenue that is not part of a core (or base) grant received from the government. Includes research grants from national and international funding agencies, research councils, research foundations, charities and other non-profit organisations. Measured in €1,000s using Purchasing Power Parities (PPP). Expressed per fte academic staff. |
| Research | Doctorate productivity | The number of doctorate degrees, relative to the number of academic staff (fte). |
| Research | Research publications (absolute numbers) | The number of department’s research publications indexed in the Web of Science Core Collection database, where at least one author is affiliated to the source university or higher education institution. |
| Research | Citation rate | The average number of times the department’s research publications from the period 2010-2013 are cited in other research (published in 2010-2015), adjusted (normalized) at the global level for the field of science and the year in which a publication appeared. |
| Research | Top cited papers | The proportion of the department’s research publications that, compared to other publications in the same field and in the same year, belong to the top 10% most frequently cited. |
| Research | Interdisciplinary publications | Percentage of department’s research publications within the field’s top 10% publications with the highest interdisciplinarity scores. |
| Research | Research orientation of teaching | The degree to which the education is informed by research in the field (based on a survey of students in the programme). |
| Research | Post-doc positions | The number of post-doc positions relative to the full-time equivalent number of academic staff. |
| Research | External research income | Revenue for research that is not part of a core (or base) grant received from the government. Includes research grants from national and international funding agencies, research councils, research foundations, charities and other non-profit organizations. Measured in € 1,000s, using Purchasing Power Parities (PPP). Expressed per fte academic staff. |
| Research | Research publications (size-normalised) | The number of research publications (indexed in the Web of Science database), where at least one author is affiliated to the university (relative to the number of students). |
| Research | Publication output | Number of all research publications included in the institution’s publications databases, where at least one author is affiliated to the institution (per fte academic staff) |
| Research | Art related output | The number of scholarly outputs in the creative and performing arts, relative to the full-time equivalent (fte) number of academic staff. |
| Research | Citation rate | The average number of times the university’s research publications (over the period 2011-2014) are cited in other research; adjusted (normalized) at the global level to take into account differences in publication years and to allow for differences in citation customs across academic fields. |
| Research | Top cited papers | The proportion of the university’s research publications that, compared to other publications in the same field and in the same year, belong to the top 10% most frequently cited worldwide. |
| Research | Interdisciplinary publications | Extent to which reference lists of university’s publications reflect cited publications in journals from different scientific disciplines. |
| Research | Post-doc positions | The number of post-doc positions relative to the number of academic staff (headcount). |
| Research | Research publications (absolute numbers) | The number of university’s research publications (indexed in the Web of Science Core Collections database), where at least one author is affiliated to the source university or higher education institution. |
| Research | Professional publications | The number of professional publications per fte academic staff. Professional publications are all publications published in journals, books, and other media that are addressed to a professional audience and that can be traced bibliographically. These publications are not peer reviewed as in the category “academic publications”. |
| Research | Strategic research partnerships | The number of strategic partnerships per fte academic staff |
| International orientation | International orientation of bachelor programmes | A composite measure taking into account (1) the existence of joint/dual degree programmes; (2) the inclusion of study periods abroad; (3) the percentage of international (degree and exchange) students; and (4) the percentage of international academic staff. |
| International orientation | International orientation of master programmes | A composite measure taking into account (1) the existence of joint/dual degree programmes; (2) the inclusion of study periods abroad; (3) the percentage of international (degree and exchange) students; and (4) the percentage of international academic staff. |
| International orientation | Opportunities to study abroad | An assessment of the opportunities for studying abroad, based on a survey of the students. |
| International orientation | International doctorate degrees | The percentage of doctorate degrees that are awarded to international doctorate candidates. |
| International orientation | International joint publications | The percentage of the department’s research publications that list at least one affiliate author’s address in another country. |
| International orientation | International research grants | The proportion of external research revenue – including public and private funding organisations and businesses – that comes from other countries. |
| International orientation | Foreign language BA programs | The percentage of bachelor programmes that are offered in a foreign language. |
| International orientation | Student mobility | A composite of international incoming exchange students, outgoing exchange students and students in international joint degree programmes. |
| International orientation | International academic staff | The percentage of academic staff (on a headcount basis) with foreign citizenship. |
| International orientation | International doctorate degrees | The percentage of doctorate degrees that are awarded to international doctorate candidates. |
| International orientation | International joint publications | The percentage of the university’s research publications that list at least one affiliate author’s address located in another country. |
| International orientation | Foreign language MA programs | The percentage of masters programmes that are offered in a foreign language. |
| International orientation | Program international orientation | International orientation of degree programmes: composite of joint/dual degree programmes, inclusion of study periods abroad, international students, international staff, teaching in foreign language. |
| International orientation | Foreign language long first degree programmes | The percentage of long first degree programmes that are offered in a foreign language. |
| Regional Engagement | Student internships in the region | Out of the students who did an internship, the percentage where the internship was with a company or organisation located in the region. |
| Regional Engagement | BA theses with regional organisations | Degree theses of bachelor graduates done in cooperation with organisations (industry, public, non-profit organisations) in the region. |
| Regional Engagement | MA theses with regional organisations | Degree theses of master graduates done in cooperation with organisations (industry, public, non-profit organisations) in the region. |
| Regional Engagement | Regional joint publications | The percentage of department’s research publications that list at least one co-author with an affiliate address in the same spatial region (within a distance of 50 km from the university). |
| Regional Engagement | Income from regional sources | The proportion of external research revenues – apart from government or local authority core/recurrent grants – that comes from regional sources (i.e. industry, private organisations, charities). |
| Regional Engagement | BA graduates working in region | The percentage of bachelor graduates who found their first job (after graduation) in the region where the university is located. |
| Regional Engagement | Student internships in region | Out of all the university’s students who did an internship, the percentage where the internship was with a company or organisation located in the region. |
| Regional Engagement | Regional joint publications | The percentage of the university’s research publications that list at least one co-author with an affiliate address located in the same spatial region (within a distance of 50 km). |
| Regional Engagement | Income from regional sources | The proportion of external research revenues - apart from government or local authority core/recurrent grants – that comes from regional sources (i.e. industry, private organisations, charities). |
| Regional Engagement | MA graduates working in region | The percentage of masters graduates who found their first job (after graduation) in the region where the university is located. |
| Regional Engagement | Graduates employment in the region | Percentage of graduates working in the region 18 month after graduation. |
| Regional Engagement | Strategic research partnerships in the region | The number of strategic research partnerships with partners in the region as a percentage of the total number of strategic research partnerships |
| Regional Engagement | Student internships in the region | Out of the students who did an internship, the percentage where the internship was with a company or organisation located in the region. |
| Regional Engagement | BA theses with regional organisations | Degree theses of bachelor graduates done in cooperation with organisations (industry, public, non-profit organisations) in the region. |
| Knowledge Transfer | Income from private sources | Research revenues from private sources as a share of total external research income. |
| Knowledge Transfer | Co-publications with industrial partners | The percentage of a department’s research publications that list an author affiliate with an address that refers to a for-profit business enterprise or private sector R&D unit (excludes for-profit hospitals and education organisations). |
| Knowledge Transfer | Patents awarded | The number of patents assigned to (inventors working in) the university over the period 2001-2010. |
| Knowledge Transfer | Co-patents with industry | The percentage of the number of patents assigned to (inventors working in ) the university over the period 2001-2010, which were co-applied with at least 1 applicant from the industry. |
| Knowledge Transfer | Publications cited in patents | The percentage of the department’s research publications that were cited in the reference list of at least one international patent (as included in the PATSTAT database). |
| Knowledge Transfer | Income from private sources | Research revenues and knowledge transfer revenues from private sources (incl. not-for profit organisations), excluding tuition fees. Measured in €1,000s using Purchasing Power Parities. Expressed per fte academic staff. |
| Knowledge Transfer | Co-publications with industrial partners | The percentage of the university’s research publications that list an author affiliate with an address referring to a for-profit business enterprises or private sector R&D unit (excludes for-profit hospitals and education organisations). |
| Knowledge Transfer | Patents awarded (size-normalised) | The number of patents assigned to (inventors working in) the university over the period 2002-2011 (per 1,000 students). |
| Knowledge Transfer | Industry co-patents | The percentage of the number of patents assigned to (inventors working in ) the university over the period 2002-2011, which were co-applied with at least 1 applicant from the industry. |
| Knowledge Transfer | Spin-offs | The number of spin-offs (i.e. firms established on the basis of a formal knowledge transfer arrangement between the institution and the firm) recently created by the institution (per 1000 fte academic staff) |
| Knowledge Transfer | Publications cited in patents | The percentage of the university’s research publications that were mentioned in the reference list of at least one international patent (as included in the PATSTAT database). |
| Knowledge Transfer | Income from continuous professional development (CPD) | The percentage of the university’s total revenues that is generated from activities delivering Continuous Professional Development courses and training. |
| Knowledge Transfer | Patents awarded (absolute numbers) | The number of patents assigned to (inventors working in) the university (over the period 2002-2011). Graduate companies The number of companies newly founded by graduates per 1000 graduates |
Data Collection and Publication Dates
U-Multirank derives its data from a mixture of publicly available citation information, patent registries, a student survey, self-reported data and prefilling.
Bibliometric
Bibliometric information gathered from the CWTS licensed edition of Clarivate Web of Science, including the SCI, SSCI and Arts and Humanities Index. Arts and humanities are excluded from the citation counts, because it is near impossible to find a consistent and reliable citation metric for these areas, and they very often depend on other forms of publication, such as books, artwork and conference proceedings. Therefore, just citations in peer-reviewed journals is not a reliable indicator of quality.
There are two groups of institutions covered by U-Multirank, the top 850 universities in the Leiden ranking, of which all three of the state universities are, and full participants in U-Multirank, which UNESP is.
CWTS processes and cleans all data ‘top down’, but accepts voluntary submission of data on commonly confused author and institutional attributions. This is especially important where attributions may be in multiple languages; variations on University of Sao Paulo, Universidade de São Paulo and University of São Paulo can appear as three separate institutions, while attribution for medical research split between hospitals and medical schools can also prove problematic.
The impact and citation measures are largely taken from the CWTS Leiden ranking, as they are produced by the same institute.
Patent Information
Information on patents is taken from the global patent index Patstat, while non-patent references (NPRs) within filed patents are cross referenced with Web of Science.
Self Reported Data
Data are reported via a questionnaire to be filled out, available here, which is for all data unrelated to citations and patents. It is then independently assessed and validated by the U-Multirank team.