Hannah Hok Kim

Kim received her PhD in developmental psychology from the University of Chicago, where she studied social cognitive development and how children think about rules. While at the Center for Research on Equitable and Open Scholarship (CREOS) she is working with faculty mentor Rebecca Saxe, Associate Dean and John W. Jarve (1978) Professor in Brain and Cognitive Sciences.

“Rhetorical justifications about open scholarship and open science — such as the way it contributes to the academic ecosystem and to the workflows of individual researchers — have played an important role in shaping perceptions, driving adoptions, and influencing policy decisions surrounding practice. Despite growing support behind open scholarship and open science as a movement, the adoption of open science practice can vary across disciplines. This disparity may stem from methodological divides such as between qualitative or quantitative methods. Further, disciplines are rich with differences in internal norms and epistemological styles.

In this project, we examine how different epistemological styles of scholars, as well as how rhetorical justifications underpinning open science statements, affect researcher attitudes and beliefs. We plan to 1) survey current scholars across varied disciplines on their epistemological beliefs and open science attitudes; and 2) experimentally manipulate different types of rhetorical justifications to examine rates of open science endorsement across scholars. These findings have the potential to inform tailored strategies for fostering equitable and effective open scholarship practices across disciplines.”

James Michaelov

Michaelov is a scholar of cognitive science whose research focuses on language models and psycholinguistics. He earned his PhD in Cognitive Science at the University of California, San Diego in 2024, and previously studied philosophy and linguistics at the University of Edinburgh, where he earned his master’s degree in Cognitive Science. While at CREOS he is working with faculty mentor Roger Levy, Professor in Brain and Cognitive Sciences.

“The field of artificial intelligence, particularly in the area of large language models, has seen rapid advancements in the past few years. The increase in the apparent capabilities of such systems has led to their increased adoption by both individuals and companies. Concurrently, there has also been a rise in the amount of scientific research involving large language models, both in terms of fundamental machine learning and applications. From a scientific perspective, one of the more concerning trends is that the majority of the most powerful models today are proprietary, and thus there is a lack of transparency about how these models are trained and on what data. In addition to possible ethical and legal ramifications, this situation can also make it difficult to successfully evaluate scientific claims surrounding them.

In this project, we plan to develop a framework for understanding the scientific inferences that can be drawn from research on large language models, with a particular focus on how open science practices can help to strengthen the validity of the scientific claims made. Based on this, we plan to develop a set of recommendations and best practices for open science in research involving large language models, with the goal of improving the ability of scientists to understand their capabilities and risks, and thereby separate the reality from the hype.”

To learn more about CREOS research, visit libraries.mit.edu/creos.