MIT Libraries receives grant from National Science Foundation to explore open science evaluations with ICOR

Project will investigate options for researchers to embed assessments into existing initiatives to measure impact

MIT Libraries has received an Early-concept Grants for Exploratory Research (EAGER) grant of $300,000 from the National Science Foundation to explore a transformative model for evaluating open science policies, practices, and interventions. The project is a collaboration between the Libraries’ Center for Research on Equitable and Open Scholarship (CREOS), a hub for rigorous research to inform the development of a more equitable and open system of scholarly communications, and Incentivizing Collaborative and Open Research (ICOR).

“Open science is widely seen as promising transformative results – accelerating discovery, fostering collaboration, increasing the reliability of research, and promoting public trust in science,” says Chris Bourg, founding director of CREOS and director of the MIT Libraries. “But far too often, the impact of open science policies and practices is based on scant evidence or assumptions. Developing a new evaluation model will let us know if we are really making progress toward these outcomes.”

The EAGER project aims to advance a research model that would enable more systematic understanding of open science by embedding experiments and measurements into existing open science systems and activities. CREOS and ICOR will inventory opportunities for embedded interventions, conduct a listening tour with diverse stakeholders, develop a strategy for a proposed fellowship program, and create guidance for standardized reporting on studies of equitable open scholarship.

“With the increase in open scholarship, it’s important to study the consequences of new policies and emerging best practices. This NSF EAGER grant will allow us to study points of entry into the process of developing a future fellowship program that can scale and adapt to learn what works and what doesn’t,” said Kristen Ratan, ICOR CoFounder. “The one year grant provides an opportunity to engage with and learn from the community of researchers studying issues of equity and the intersection with open scholarship.”

This is the first partnership between CREOS and ICOR, combining their strengths in research in equitable and open scholarship and real-world implementations of open research practices. Developing a reproducible model for evaluating programs and practices aligns with the missions of both organizations and fills a need given the evolving open scholarship policy landscape.

The MIT Libraries launched CREOS in late 2018 in response to a need for credible research to inform efforts for a more equitable and open scholarly ecosystem — a complex set of institutions, actors, and infrastructures that contribute to the creation, distribution, and use of scholarship, including universities, publishers, libraries, scholarly societies, funders, and technologies.

ICOR was started in 2020 by Kristen Ratan (Strategies for Open Science) and Sarah Greene (Rapid Science) to identify policies, tools and practices that, when implemented in concert, will challenge the status quo of closed research by (1) producing evidence of the benefits of open and collaborative research with real-world implementations, (2) addressing issues of non-reproducibility and paucity of innovation, and (3) de-emphasizing high-impact publication as the primary means of career advancement.