Skip to content

Quality flags

Each end-of-call record is auto-evaluated by bridge/src/calls.js::computeFlags(). Flags surface in the per-pilot dashboard as red pill labels and in the daily digest count.

The flags

Flag Trigger What to do
missing_callback callback_number is empty, non-numeric, or fewer than 10 digits after stripping non-digits Listen to the recording. If Polly didn't ask, the prompt regressed. If she asked but caller didn't give a clean number, that's a real-world miss — tweak validation copy.
no_caller_name caller_name is empty or — Usually means Polly didn't ask up-front. Check the prompt's order — name must be the first ask after the greeting.
no_job job is empty or — Catastrophic for the trade. Listen — Polly may have ended the call too early.
short_call duration < 30s Often hang-ups before Polly finished. Could be voicemail-style caller behaviour, real wrong-number, or Polly being too slow off the mark. Look at the transcript.
not_clean_end endedReason is not assistant-ended-call or customer-ended-call Network / silence-detection / Vapi internal. Rarely a prompt issue, more often a config one.
extraction_empty structured-data object empty Worst case — Polly ran the call but didn't produce structured output. Almost always Vapi/model error mid-call. Check Vapi's logs.
sms_failed The job-handoff SMS to the trade errored at Twilio Check Twilio balance + auth. Surface in the dashboard with the error message.

How they're used

  • Per-call: displayed as red pills next to that call in the dashboard's per-pilot view.
  • Aggregated: the daily digest reports total flagged across all pilots; ⚠️ banner appears if >0.
  • Improvement loop: the improvement loop describes how flagged calls get turned into prompt tweaks.

What flag-free looks like

A "good" Polly call has all six structured fields filled, duration 60-180s, clean assistant end, SMS-to-trade landed. We expect ~80% of real customer calls to come out flag-free once the prompt is well-tuned.

Early in a pilot's life, expect more flags — that's the signal to iterate on the base prompt.