Nov 2025 – Apr 2026
AI call quality scoring
Recorded calls transcribed and scored by Gemini on an 8-criterion rubric, with a score card for each operator
- AI
- AI quality control
- Duration
- 5 months
- Company
- Boxette Group
- Role
- Designed the rubric; built the workflow and visual
Overview
An n8n pipeline that took each recorded call-center call, had Gemini transcribe it and score it against an 8-criterion rubric, stored the result in Postgres, sent the operator a personal score card and fed a Power BI dashboard. It was built in November 2025 and developed through April 2026.
Tools
- Gemini
- PostgreSQL
- Power BI
- Mattermost
Too few calls heard
Supervisors could listen to only a small share of calls, so operators got little consistent feedback, and the old 4-criterion scoring was too coarse to coach from. The aim was to rate every call the same way and give each operator the reasons behind the score.
A rubric and a model test
I designed an 8-criterion, 100-point rubric (service, clarity, empathy, active listening, accuracy, problem solving, efficiency, handling stress) with deduction rules and a Georgian QA-auditor prompt. Gemini 2.5 Pro and Flash were then compared on the same calls and priced. Flash scored more realistically (81 against 93 on the same call) at about one sixth of the cost.
How the pipeline worked
The n8n workflow took one unrated call at a time from the telephony call table and downloaded the recording; Gemini returned a transcript, a call category and the scores, which the workflow parsed, priced and saved. Unreadable responses were logged with their cost and the call went back into the queue, and when the LangChain Gemini node began failing on audio, that step was rebuilt as a direct HTTP request with inline audio. Each operator got a Mattermost message with their score and the reasons, and a Power BI custom visual in TypeScript added per-cell comment tooltips to the score table, which the standard table cannot do.
Result
By 20 April 2026 the database held 32,013 calls, 2,199 of them rated by AI with an average score of 78.4 out of 100, and 3,507 calls were rated by 28 April. The projected cost for 300 calls a day was about 150 GEL a month on Flash against about 820 GEL on Pro. New error handling recovered 124 stuck calls.