While much of the attention around large language models has focused on their question-answering prowess, reading comprehension abilities, and coding skills, some of their most remarkable and transformative capabilities lie in the creative realm. From fiction writing and storytelling to open-ended ideation, prompt engineering and freeform language generation, LLMs are shattering preconceived notions of what AI can accomplish.
The experts at LMSYS have specifically honed in on evaluating and ranking LLM performance across a range of creative, open-ended language tasks through human evaluations and benchmarks. The results reveal certain models that exhibit almost supernatural levels of imagination, expressiveness and language creativity.
Let's take a look at which LLM providers are setting the bar for ideation and artistry:
While Claude emerged as Anthropic's flagship model emphasizing truthfulness and reasoned outputs, their counterpart ChatGPT was designed with a bit more free-spiritedness in mind for open-ended language generation.
This paradigm seems to pay major creative dividends, based on ChatGPT's exceptional LMSYS scores for imaginative language tasks:
Across these creative domains, ChatGPT consistently ranks at or near the top, showcasing fluent, naturalistic language generation abilities that almost seem to channel shades of human-like creative flow. Its responses often have a distinctive voice and personality.
This makes ChatGPT a prime candidate for writing aids, freeform storytelling, worldbuilding, prompt engineering, and many other open-ended creative language workflows.
This AI startup founded by former Google researchers has taken things a step further by doubling down on specialization for expressive, freeform language generation with their MLX-Writer model.
Billed as an "AI writing muse," MLX-Writer exhibits off-the-charts performance in LMSYS inventive language evaluations:
In sample after sample, MLX-Writer's outputs demonstrate an almost visceral imaginative flair - elaborately describing scenes, constructing intricate narrative arcs, personifying rich characters, and infusing imagery and emotions.
While it may not always maintain perfect factual accuracy or logical coherence, the model seems intentionally geared towards prioritizing unbounded creative expression over constraint. It's an AI engine for unleashing uninhibited imagination.
For fiction authors, poets, screenwriters, or any creative professionals looking for an AI muse to stimulate ideas and provide thought-provoking material to riff off of, MLX-Writer could be an intriguing (albeit potentially inconsistent) companion.
Not to be outdone, Google has also taken an interest in highlighting their PaLM model's more romantic, artistic talents as well through product integrations like the Bard creative writing mode.
Evaluating Bard, LMSYS measured very high caliber writing sample outputs putting it roughly on par with ChatGPT and GPT-4 in creative domains like:
However, where Bard appears to differentiate itself is through integrating PaLM's multimodal capabilities to enhance the creative process through various sensory data inputs.
For example, Bard can analyze an existing image and generate descriptive narratives based on the visual scenes. Or it could accept an audio clip as inspiration to compose poetic lyrics. This cross-pollination of modalities introduces a new dynamic to creative language workflows.
While still in its nascency, Bard hints at what true multimodal creative AI could look like by fusing Google's language and sensory models together. Providing diverse inputs like sights, sounds and more seems to unlock deeper imaginative facets within PaLM.
Of course, the leading LLM players are far from the only ones pushing into creative ideation tasks. More niche offerings like:
...are all also vying to carve out a space across the wide spectrum of imaginative use cases and open-ended language workflows.
And chances are, we've only scratched the surface of what specialization around unrestrained, unconstrained language generation could unlock in the creative realms.
After all, every breakthrough in art, storytelling, worldbuilding and ideation throughout human history stemmed from that spark of unshackled imagination first. With LLMs now exhibiting scalable abilities to mimic those neural creative pathways, we may be opening doors to realms of artistic expression and fictional world that once seemed unimaginable even for AI.
So while the current open-ended LLM leaders are already redefining what's possible, the drive to push further into language creativity and generation will only intensify. Buckle up writers, dreamers and imaginative sojourners of all kinds - an AI muse is about to whisper its first seeds for entire new worlds to blossom.
For a comparison of rankings and prices across different LLM APIs, you can refer to LLMCompare.