Personas - An Answer to Collapsed Data...
or late-night snack for a hungry model?
Have you heard of the Persona Hub?
According to an article in Medium in August 2024, “Persona Hub is an innovative, persona-driven data synthesis methodology that leverages a collection of 1 billion diverse personas to generate high-quality synthetic data.”
How does this relate to The Sims article I wrote back in June 2024?
Read on.
First, let’s unpack that quote above.
Persona Hub is comprised of one BILLION "personas," each one distinct: from profession and personality, to full-on, fictitious backstory. Oversimplified, at least as I understand things, the Persona dataset, which is statistically identical to “real” data, can be used to maintain entropy (degree of disorder or randomness in a system) in the creation of a synthetic data set.
The amount of actual “real human” data from which large language models (LLMs) can learn and create context is FINITE. And because these LLMs are HUNGRY, companies that train these models need to generate copious quantities of lab-grown, entropy-laden yummy-ness for the ravenous beasts.
Why Does ENTROPY Matter?
Entropy is ESSENTIAL when creating synthetic data in order to prevent something called “collapsed data.” What’s that? Well, ever have Chat GPT or one of the other LLMs get caught in a loop and keep serving up the same 2-3 examples of something? That’s likely because the model’s distribution has lost entropy and richness. Where there is collapsed data, basically, you get a tired comedian that can only tell the same 3 jokes. Nothing arbitrary, random, or variable… Just those tired old jokes, over and over.
These days, AI companies are Jonesing for rich, entropy filled, QUALITY synthetic data sets to pump up the volume! Enter Persona Hub. Their individual personas can be used to keep things fresh, interesting, and FULL of entropy.
When companies can generate synthetic data which accurately reflects those things that make our world the complex, unpredictable, subtle, and nuanced hot mess and miraculous mystery it is, much becomes possible. From sophisticated, intricate reasoning, to more human-like communication, and AI applications the likes of which we can barely imagine today - high-quality synthetic data will be at the core of it all.
Back to Persona Hub…
Okay, so How Did They Do It?
When I first heard about the “1,000,000,000 personas,” I wondered… Might this be a Sims-derived dataset compiled from individual characters (Sims) created within the game. Seemed a reasonable question. I mean, could there be one BILLION discrete Sims? Maybe!?
I mean where ELSE could one come up with fodder to create that many individual, human-like personalities with fully fleshed-out fictional lives?
In the Words of Kate Bush, “Pause for the Jets” (welcome to my inner monologue)
You likely beat me to the punch: the answer, I thought… TikTok.
This Got Me Thinking
While I’m not ready to ditch my whole hypothesis regarding the value of the data that comes from decades of Sims life history (and how that could be used to generate high-quality synthetic data), I DO wonder, might there be some tracible line from TikTok to Persona Hub?
Persona Hub comes out of Tencent’s artificial intelligence research center, Tencent AI Lab, in Seattle, WA. — ByteDance Ltd. owns TikTok Inc. — Tencent acquired ByteDance gaming studios in March 2024.
While this does NOT mean Tencent has access to TikTok data via ByteDance, there does seem to be some historical competitive and cooperative spirit between Tencent and ByteDance. Enough to access the data needed to create one billion personas?
I mean, think of all the TikTok user data, the videos created, and what could be scraped now that vision models can interpret, tag, label, geolocate, and grok everything IN frame AND the metadata behind it all. Wowzah!
Who knows. I simply find it all fascinating!
That’s it for tonight! Next up PANERA! Stay tuned.
FYI, if you’re interested, there’s a solid article on Medium (Aug 2024) all about Persona Hub and how they scale data creation and utilize their “personas” to bring life to the data-party. Check it out on Medium
Image Notes - Generated via DALL-E 3