The Gist
Husk, a viral influencer known for challenging AI models, tested OpenAI's new voice model, GPT-Live, by conducting a spelling test. Despite its claims of being more natural and human-like, GPT-Live made fundamental mistakes, suggesting that it still has room for improvement.
How It Worked
Husk set the GPT-Live model to its highest smart setting and asked how many E's are in 'seventeen.' The model incorrectly claimed there were only two E's, despite spelling the word correctly later. Husk continued the interaction by asking for human-like advice, prompting the model to suggest giving personal compliments to strangers. The awkward pauses and responses during these exchanges highlighted the model's limitations in creating a truly natural conversation.
Results
GPT-Live failed a basic spelling test, miscounting E's in 'seventeen.' Additionally, it demonstrated awkward conversational behavior, such as heavy sighs and long pauses, which detracted from the naturalness it aimed to achieve. While it showed improvements over previous models in metrics like pleasantness and conversational flow, these instances reveal that it can't yet match human interaction.
Why It Matters for You
If you're working with AI chatbots or voice assistants, consider Husk's findings as a reminder of the importance of human-like interaction. While AI is evolving, incorporating empathy and natural pauses can enhance user experiences. Always test your models against real-world scenarios to identify gaps in their capabilities. This ensures that your AI can genuinely assist rather than confuse users.
