Key Takeaways
Enabling Bella Capri with AI-driven conversation insights
Introduction
Bella Capri Pizzeria, a premium chain with an average ticket value of over R$110, had automated more than 80% of customer service via WhatsApp and Instagram — but had no visibility into whether AI was losing sales or driving customers away. Without concrete data, strategic decisions were made in the dark. This project was critical to turn conversations into actionable insights about AI accuracy and human agent performance.
"Impressive! This is an analysis we always wanted to carry out but never had the human resources to do it. Now it's time to apply the solutions to the problems we identified." — Bella Capri
The Challenge
Bella Capri faced significant challenges in assessing the effectiveness of their AI after automating over 80% of customer interactions. They lacked visibility into AI performance, accuracy, and lead qualification, which hindered strategic decisions. Previously, evaluations were manual, requiring up to 30 minutes daily for just 10 conversations, limiting real-time insights.
Before: Manual conversation analysis took up to 30 minutes daily for a limited number of evaluations, hindering comprehensive performance insights.
After: Significant reduction in evaluation time and increased AI autonomy enabled comprehensive insights into conversational dynamics and performance.
Our Approach
Our team developed a Node.js script integrating Livechat's API, GPT-4o for conversation analysis, and Database for data storage. The script parsed conversations deterministically, followed by AI-driven structured judgments, ensuring reliability and detail in evaluating over six key dimensions like bot performance and lead scoring.
Tech Stack
Fetch and Classify Messages
Retrieve messages from Livechat, classify by sender type, and prepare for structured analysis.
Parse Data
Extract objective data points such as timestamps and message counts.
AI Analysis
Utilize GPT-4o for subjective analysis, producing structured JSON outputs across six key dimensions.
Score Computation
Merge outputs to compute final scores, apply stage-based capping with no AI restrictions on math.
Results Storage
Store comprehensive analysis results in Database for dashboard visualization.
The Solution
We built a robust solution that automatically analyzes customer conversations from various platforms, assesses AI performance across six dimensions, and outputs detailed reports into a dashboard. This setup leverages GPT-4o for nuanced analysis and Database for efficient data management, drastically reducing manual evaluation time.
Automated Lead Qualification
Utilizes AI to analyze conversational quality and potential conversion.
Abandonment Tracking
Identify at which stage in the funnel conversations drop off to improve retention strategies.
Comprehensive Bot Performance Evaluation
Details AI's handling capabilities and areas for improvement with precise metrics.
Results & Impact
With the human response rate jumping from 40% to 95%, Bella Capri recovered leads that were previously lost due to lack of response. Manual evaluation — which consumed 30 minutes daily — was completely eliminated, freeing the team to focus on what matters: selling. The analysis also revealed that peak sales hours had the worst human response rate, an insight no manual evaluation would have captured.
"We discovered that our peak hours had the worst human coverage — without this analysis, we would never have identified that." — Bella Capri
AI Autonomy Before and After Implementation
Human Response Rates Before and After
Conclusion
The Bella Capri project set a new standard in AI analysis for customer service, revealing critical operational insights that no manual process could capture. Bella Capri is already expanding the analysis to new channels, with real-time integration for automatic detection and correction of service gaps.
Next Steps
- Integrate real-time analysis for automatic detection and correction of service gaps.
- Expand analysis coverage to additional communication channels.
Embed Station delivers innovative digital solutions, combining cutting-edge technology with strategic thinking to transform businesses.
Contact: hello@embedstation.com | embedstation.com