MIT Releases New Research Associating Generative AI With “Cognitive Debt”
Earlier this year, researchers at the Massachusetts Institute of Technology, MIT, released a 206-page article on the neurological side-effects of using Large Language Models (LLMs) like ChatGPT to perform tasks related to writing and critical thinking. Schools across the country, including our own Centenary College, are still navigating policy and outlook regarding these LLMs. While some have chosen a more accepting stance towards generative AI, the results of the study performed at MIT could sway those on the fence.
The study was particularly interested in the cognitive effects when partaking in essay writing. The researchers used three groups: LLM writers, Search Engine writers, and Brain-Only writers in four writing sessions, the fourth requiring the LLM writers to switch to Brain-Only, and the Brain-Only writers to switch to using LLMs. The participants were considered advanced writers and were recruited from the following Boston-area universities: MIT, Wellesley, Harvard, Tufts, and Northeastern. For each session, three essay prompts taken from the SAT writing exam were provided, often centering on philosophical topics like happiness, art, or enthusiasm.
After all four sessions were performed and the EEGs and interviews analyzed, the researchers’ conclusions showed drastic differences between all three groups. Unsurprisingly, the Brain-Only groups' essays showed the most individuality, the Search Engine group showed influence from the sites visited, and the LLM group’s essays were statistically homogenous. LLM-Only writers showed little ownership of their essays and could not recall quotes even minutes after writing, compared to the Brain-Only writers who demonstrated high ownership. All of this could be guessed prior to the research, but this study showed a particularly novel way of understanding LLM use through their concept of “Cognitive Debt.”
During the study, LLM users demonstrated a diminished ability to critically evaluate the LLM’s output. Additionally, as the sessions progressed, Brain-Only users showed peak levels of neural pathway connection, while LLM users showed very little activity. In the fourth session, when Brain-Only writers switched to LLMs, they continued to demonstrate high neural pathway connection and critical thinking skills, while LLM users continued to have little activity when using the Brain-Only writing method. This link between writing styles indicates that even after students have ceased using Large Language Models like Chat-GPT, they are not automatically catapulted to the same brain activity and critical thinking skills as their non-LLM-using peers. Instead, they must catch up on their “Cognitive Debt,” effectively starting from square one.
As schools continue to form policy around LLM use, this study provides relevant topics for consideration. The article suggests that use of LLMs in essay writing in any stage reduces the student’s ability not only to critically think about the essay at hand, but also greatly reduces their metacognition ability for future assignments. Use of LLMs, then, can not only set our students back in their writing, but make them less competent academics and educated individuals.