Why the Growing Popularity of AI LLMs for Admissions Essay Writing is Bad News for Aspiring Ivy Leaguers
College application season is upon us, and this year may herald a new admissions era—for the first time in history, all applicants have access to generative artificial intelligence (AI). Already, surveys suggest roughly 20% of students are using chatbots to cheat. And a slew of recent articles evaluating bot-generated writing has intensified the buzz surrounding AI’s impact on education. But Dimension Admissions is here to warn you away from temptation. Read on for our reasoning.
Origin: Why Test the Competency of AI LLMs in Admissions Essay Writing?
Existing coverage of bot-generated admissions essays in particular has been limited to work with a single AI LLM on only one or a handful of essays—a method of inquiry we suspected might undersell the bots. In order to give generative AI a fair shake, our essay experts ran four mainstream bots (ChatGPT, GPT-4, Bing, and Bard) through a rigorous, extended, multi-phase trial. We also conducted an abbreviated evaluation of a lesser-known but lauded bot: Claude 2, that arrived on the scene after our trial concluded. Using our expertise as narrative specialists and application strategists, we evaluated the caliber of each chatbot essay, and tried systematically to elevate the quality of the work on subsequent trials.
Ultimately, after 60+ hours of testing, reams of chat-logs, dozens of essays, and hundreds of iterations, we reached the performance threshold no bot could surpass. There and along the way, the dismal downsides to AI-sourced admissions essays revealed themselves. In short, our essay experts reached a unanimous verdict: even the most sophisticated AI chatbots are incapable of generating Ivy-caliber essays, no matter how targeted and tenacious the coaching.
So, may this post serve as a Public Service Announcement for all college hopefuls: Beware the bots—enlisting AI large language models to ghostwrite your admissions essay is risky, short-sighted, self-defeating, and altogether unwise. In short, no bot can unbolt the Ivy gates.
Methodology: How Did We Conduct Our Trial of AI Essay Writing?
Over the course of two weeks, our core advisors ran all four mainstream AI LLMs through iterative trials to craft application essays using anonymized profiles drawn from past clients. In Trial Phase I, we fed all three bots the same essay prompt and evaluated their output to gauge performance across platforms.
In Trial Phase II, the longest of the three phases by some margin, we challenged the bots to respond to multiple Common App prompts and school-specific supplements. This afforded us a range of essays—short and long, abstract and deeply personal—for the sake of comparison. We fed them pre-defined coaching prompts to enhance content and delivery, including: tone, imagery, syntax, and revelatory detail, and tracked their moves, playing the bots off one another in a drafting competition.
In Trial Phase III, each of us teamed up with GPT-4—the far-and-away superior bot per Trial Phase II—and competed against one another with the goal of generating the best possible bot-born essay. There were no set coaching prompts, only the goal of taking generative AI to its application essay-writing limits. The crowned winner earned everlasting respect, high-fives, and a cupcake.
We have also incorporated an addendum addressing Claude 2, a bot that did not make it into our mainstream line-up, but which has attracted sufficient attention about its writing competency to pique our interest.
Summary Conclusion: What Did We Learn about AI LLM’s in Admissions Essay Writing?
Turns out, the bots are pretty good at the basics. Their generated essays were competitive with many of the first drafts that come across our desks. But as one Johns Hopkins lecturer noted, “It’s not the stuff of admissions-office lore.” The revision process is what separates the wheat from the chaff, and those essential next steps involving analysis, self-reflection, exposition, and vibrancy are where the bots failed and failed again. Ultimately, our results-based conclusions were startlingly consistent across all three phases: the bots are cut out for all kinds of jobs, but writing your admissions essay isn’t one of them.
The core of the issue is intrinsic to generative AI. Chatbot software uses sprawling language models to predict the next word it should churn out in a sequence—think of an auto-fill function on steroids. This means that bot-generated content is by nature derivative; its writing is often “bland and vague,” or overstated and cliched, or—somehow—all of the above.
Ultimately, your goal in writing an admissions essay is to reflect all your idiosyncratic, many-splendored glory. If you enlist AI to write your essays, they will have no nuance, no insight, no body heat. They are blunt force objects, when what an applicant needs to unlock admission at our nation’s top colleges is a precision instrument—like a heart.
To understand how we reached this conclusion, please see our next installment of our AI LLM series later this week!
Christopher holds a B.A. from Yale University, an M.F.A. in Fiction from the Program for Writers at Warren Wilson College, and an M.A.Ed. from NYU Steinhardt School of Culture, Education, and Human Development, where he was inducted into the Kappa Delta Pi International Honor Society in Education. He is a certified independent educational consultant through UC Irvine and is a professional member of both the National Association of College Admissions Counselors (NACAC) and the Independent Educational Consultants Association (IECA).
Christopher founded Dimension Admissions in the summer of 2019, following eight years as an independent school instructor, administrator, and admissions file reader. During this time, he also conducted alumni interviews for Yale University. He is an expert in educational advising, English language and literature, teaching, personal narrative writing, academic and extracurricular planning, school selection, and admissions.
His objective is to empower each client to articulate how their lived experiences have shaped their personal identity, and to determine how they will utilize this foundation to engender future growth and contribute meaningfully to their communities. While his primary goal is to send each of his clients to their dream school, his success is also contingent on whether they emerge from their work with Dimension Admissions more self-aware and confident as they embark on the next chapter of their life’s journey.
