Is there a problem in the National Institutional Ranking Framework (NIRF)?

Ranking universities and institutions in a country as diverse as India is a difficult undertaking.
In 2016, the Ministry of Education launched the National Institutional Ranking Framework (NIRF) to evaluate the performance of institutions using key indicators.
Every year, institutions eagerly await their rankings in this nationally recognised system.

The NIRF publishes rankings in a variety of categories, such as ‘Overall,’ ‘Research Institutions,’ ‘Universities,’ ‘Colleges,’ and specific fields.
The rankings are a valuable resource for prospective students navigating India’s higher education scene.
The NIRF ranks institutes based on their overall score, which is calculated using five indicators:
Teaching, Learning & Resources (30% weightage)
Research and Professional Practice (30%)
Graduation Outcomes (20%)
Outreach and Inclusivity (10%)
Perception (10%)

Bibliometrics’ Role: Bibliometrics is the quantitative examination of scholarly publications, which includes metrics such as the number of publications, citations received, and journal impact factors.
Limitations: Bibliometrics may fail to effectively account for elements such as research quality and relevance, innovation, social impact, and contributions other than traditional publishing.
Be Wary of Over-Reliance: To provide a more holistic assessment of institutional performance, a comprehensive evaluation technique should evaluate a broader range of characteristics.

Reliance on Commercial Databases: The NIRF collects bibliometric data for measuring research output and impact using commercial databases such as Scopus and Web of Science. However, there may be limits in terms of coverage, accuracy, and the inclusion of non-traditional research results in these databases.
Questions about bibliometric data accuracy and usage: There are questions about the accuracy of bibliometric data, the potential manipulation of citation counts, and the misuse of metrics for promotional purposes. The integrity and authenticity of the data used in ranking assessments must be ensured.
Leaving Out Non-Traditional Contributions: The emphasis on research papers in bibliometric indicators may lead to the omission of other valuable intellectual contributions, such as books, book chapters, patents, policy reports, and other non-traditional scholarly outputs.
Disincentive for addressing local issues: The concentration on internationally recognised publications and worldwide research trends may prevent academics from addressing local concerns and doing research relevant to national or regional goals.

Inadequate Transparency: Institutions and stakeholders should have access to complete information regarding the methodology, data sources, weightage allocated to different indicators, and data collection and analysis processes.
While the NIRF openly discusses its rating methodology, there is a need for more complete and accessible documentation that provides a detailed picture of the evaluation process. This would improve stakeholders’ knowledge and allow for a more informed evaluation of the rankings.
Addressing the Disparity: To avoid ambiguity and misinterpretation, clear and explicit definitions for metrics such as research quantity and quality are essential. To provide a consistent and fair review, clear rules and criteria should be created.

Promoting complete Evaluation: To give a more complete assessment of institutional performance, evaluation approaches that go beyond bibliometrics and encompass a larger range of qualitative and quantitative aspects are needed.
Transparency, Diverse Factors, and Balance: Ensuring transparency in ranking systems, taking into account diverse factors, and striking a balance between quantitative measurements and qualitative assessments would all help to a more accurate and meaningful evaluation of Indian colleges.

Source: https://www.thehindu.com/sci-tech/science/nirf-overreliance-bibliometric-data-flaws/article66965043.ece