Fresh Thoughts #140: Managing Generative AI
- Newsletter
Generative AI can feel very human.
That is in part because a simplified model of how human brains work underpins the current generation of generative AI.
Listening to the hype - you will hear generative AI reproducing many positive aspects of human achievement:
- writing stories
- generating images
- solving complex maths
- reviewing legal documents
- speaking in rare dialects
But over the past year, as I have used and built products with generative AI daily, I have noticed something else.
Along with all the positives, generative AI comes with some frailties and quirks.
I've written in the past about social engineering AI - in a similar way to scammers sending phishing emails.
But - in a way - different generative AI models appear to have "personalities".
At the risk of anthropomorphising AI - let me explain.
Think of a time when you managed or worked with a team you knew well.
Over time, you worked out each person's strengths and relative weaknesses.
What they were good at...
And where their areas of relative weakness lay.
More specifically - you knew which tasks they should work on to be effective and which you would never give them.
In a sense - you gained an understanding of their personalities.
Over the months, I have found a similar situation with the generative AI models I use. My generative AI report cards read:
- GPT4-Turbo: has an enthusiastic, if scattergun, approach to tasks. You will get a wildly different answer whenever you ask a question. Good for idea generation but challenging to focus on the task at hand.
- GPT4o: has a more mature approach than GPT4-Turbo. A solid performer who can be creative but isn't exceptional at one specific task compared to peers. But when you find the sweet spot, GPT4o can happily produce solid results all day.
- Claude Sonet 3.5: is a stable high performer with exceptional skill for writing software. As a result, tend to be assigned software development tasks because of the higher quality output than peer models.
- o1-mini: regularly produces excellent work. However, it can be overly verbose and a little controlling. For example, planning tasks and providing the minutiae of detail can cause problems.
- o1-preview: fantastic at logical reasoning and planning. The perfect candidate for solving maths, scientific, and logic problems. However, o1-preview can be far too literal and shows limited to no signs of creativity.
Do any of these personalities sound familiar?
Final Thoughts
For the next few years, it seems unlikely that there will be one model to rule them all.
Instead, like human teams, different generative AI models will excel in various areas.
Just like managing a human team, it is essential to understand each model's strengths and blend them to create a high-performing team.
The ideas and techniques we use today in team management will be around for a while - albeit with the participants being non-human.
October 15, 20242 Minutes Read
