Speech-to-Text in Products: How Modern ASR Works and How to Use It Well

Speech-to-text, or automatic speech recognition (ASR), turns spoken language into text. It powers meeting transcription, call centre analytics, voice assistants, medical documentation, captions and accessibility features. Accuracy has improved markedly with large neural models, making voice practical in many more products.

How modern ASR works

Older systems combined separate acoustic, pronunciation and language models. Modern systems are typically end-to-end neural networks that map audio directly to text. A notable example is OpenAI’s Whisper, described in a 2022 paper, which was trained on 680,000 hours of multilingual audio and showed that large-scale weak supervision produces robust recognition across accents and conditions.

A typical pipeline includes:

1. Audio capture and pre-processing: sample rate conversion, noise reduction, voice activity detection.
2. Recognition: the model converts speech segments to text.
3. Speaker diarisation: identifying who spoke when, if needed.
4. Post-processing: punctuation, capitalisation, number formatting and custom vocabulary.
5. Downstream use: search, summarisation, analytics or commands.

Measuring accuracy

Word error rate (WER) is the standard measure: the number of substitutions, deletions and insertions divided by the number of words in a reference transcript. Lower is better. Evaluate on a sample of your own recordings with human reference transcripts, broken down by speaker group, environment and topic, and pay particular attention to critical terms such as names, numbers and medication names.

What causes errors

APIs or self-hosted models?

Product use cases

• Meeting and interview transcription with summaries and action items.
• Contact centre analytics: transcripts for quality review, compliance checks and topic trends.
• Clinical documentation: ambient notes reviewed by clinicians — see our guide to AI in EHR systems.
• Captions and accessibility for video and live events.
• Voice commands and assistants in apps, vehicles and field work.

Designing voice features

• Show live transcription where users speak to the product, so they can correct errors immediately.
• Confirm critical values such as amounts, dates and names before acting on them.
• Offer text alternatives for users who cannot or prefer not to speak.
• Handle silence and interruptions gracefully, with clear prompts.
• Combine with language models carefully: summaries and extracted actions should link back to the transcript so users can verify them.

Cost considerations

Cloud ASR is usually priced per minute of audio, with different rates for streaming, batch and enhanced models. At low volumes APIs are cheaper and simpler; at sustained high volumes, self-hosting on GPUs can cost less but adds operations work. Model your expected minutes per month, required latency and languages before deciding, and remember that diarisation, translation and summarisation may be priced separately.

Privacy and compliance

• Inform people that they are being recorded and transcribed, and obtain consent where required.
• Minimise retention of audio; keep transcripts only as long as needed.
• Redact sensitive information such as card numbers or health details.
• Check where providers process audio and whether it is used for training.

Frequently asked questions

How accurate is speech-to-text today?

For clear audio in widely spoken languages, very accurate; accuracy drops with noise, accents, jargon and overlapping speech. Measure on your own recordings.

Can ASR handle multiple languages in one conversation?

Many modern models handle language switching, but accuracy varies. Test with representative audio.

Is real-time transcription possible?

Yes, with streaming APIs or optimised models, typically with a short delay and lower accuracy than offline processing.

How do we improve accuracy for our terminology?

Use custom vocabulary features, provide context or fine-tune on your own labelled audio.