What happened
A company launched an AI support bot connected to their help center in January, aiming to deflect customer support tickets. After training the bot on their most common ticket types, they initially saw only a 6% deflection rate by the third month, which later increased to 8% by month eight. Despite these numbers aligning with industry benchmarks for complex B2B environments, the team began to question if these figures truly represented success.
Why this matters
The realization that their bot was performing below expectations prompted a reevaluation of their technology. When comparing their AI support tool to others in the market, particularly one that reportedly achieved a 47% deflection rate, it became clear that the architecture of the AI system plays a crucial role. This led to concerns about the validity of the benchmarks and whether the existing solutions were genuinely effective or just repackaged traditional ticketing systems.
Context
Historically, AI support tools have been developed to improve customer service efficiency. However, many solutions simply overlay machine learning models on existing ticketing systems without fundamentally changing how issues are resolved. This has led to a perception that certain deflection rates are acceptable, even though they may not reflect optimal performance. As companies increasingly rely on AI for customer support, understanding the architecture and core capabilities of these tools becomes essential.
What this means
The findings indicate a significant divide in AI support effectiveness based on underlying system design. If the average deflection rate is indeed as low as 8%, organizations may need to reconsider their investments. It raises questions about whether many AI support solutions are capable of delivering real value or if they are merely dressed-up ticketing systems. This situation urges businesses to critically assess their AI tools, ensuring they are leveraging solutions that are built for effective resolution rather than just automation.
