What does a 1.62% character error rate mean in practice?

In practical terms, 1.62% CER corresponds to roughly 16 character edits per 1,000 reference characters after the stated normalisation rules (dialect and standard spellings, Latin and phonetic English, and numeric forms count the same). It was measured on real multi-dialect Arabic business meetings with human-verified references, on the same material as the six external ASR model configurations. It is a result on one evaluation set, not a guarantee for every room.

Is this the word error rate figure that used to be on this site?

No. That figure is superseded. This evaluation reports the character error rate of the M3 system against six named external ASR model configurations with published scoring rules, and it replaces every earlier MeetriX accuracy figure on this site.

Was the evaluation independently verified?

No. Lisan Research ran it. The protocol, scoring rules, examples and evidence map are published so the scoring can be reproduced, and a pilot on your own meetings is the way to check it on your material.

Does MeetriX send our audio to a cloud speech API?

In the evaluated deployment the transcription path called no external transcription service (verified in the evaluated deployment). As a product capability, M3 can run within customer-controlled infrastructure so audio, transcripts and summaries stay inside your data-governance boundary; deployment options are on the security and on-prem page.

What does the 1.83% English result measure?

It is the character error rate of M3 on the English evaluation set, the same metric as the Arabic result. In practical terms, 1.83% CER corresponds to roughly 18 character edits per 1,000 reference characters after the stated normalisation rules. Every evaluated system was run on exactly the same audio clips with an explicit English language setting, against reference transcripts verified by a human listening to the audio. Punctuation, letter case and whitespace are ignored, except that a missing space that fuses two words counts as one error.

Can the Arabic and English CER figures be compared directly?

No. Both are CER, but they come from separate language-specific evaluation sets. Compare systems within each language table; do not read 1.62% and 1.83% as a ranking of the two languages.

How do we run a pilot?

Book a pilot or write to support@lisan.com. We agree a sample of your meetings, prepare a human-verified reference, share the scoring rules before the run, and report the result the same way this page does.

M3 Results: Arabic and English Transcription Benchmarks

Research

Published5 October 2026
VersionArabic and English, September 2026
AuthorshipLisan Research

Full report on lisan.com Announcement Cite

Executive summary

M3 recorded the lowest character error rate in both language-specific evaluations: 1.62% on the Arabic set and 1.83% on the English set. Each evaluation compares M3 with six external ASR model configurations on real business meetings: the same six models, each set to the language under test.

Accuracy. M3 recorded a lower CER than every external configuration on both evaluation sets.
Real-meeting evidence. The results are paired with human-verified references and segment-level examples covering short turns, names, numbers and business terminology.
Deployment. The evaluated workflow processed one hour of recorded audio in 45 seconds on an L4 GPU and can run inside customer-controlled infrastructure.

Arabic and English were evaluated on separate source material, so each headline figure is read within its own language table, not as a ranking of the two languages.

1.62%Arabic character error rateReal Arabic meetings, human-verified references

1.83%English character error rateReal English meetings, human-verified references

28% to 83%Lower Arabic CERM3 versus each external configuration

42% to 77%Lower English CERM3 versus each external configuration

45 sPer hour of audio80x real time on an L4 GPU

6External ASR configurations compared with M3The same six models, set to each language

Arabic benchmark

Arabic results: character error rate (CER)

What 1.62% means for a meeting record

A meeting record earns trust one detail at a time: the number in the budget line, the abbreviation in the technical update, the name of the vendor, the one-word answer that closed a decision. The evaluation behind this page scored exactly those details. It counted every character that a transcription system got wrong, dropped or invented against what a human verified was said, across real business meetings in several Arabic dialects with English terms inside the Arabic. Measured. M3 recorded 1.62%. In practical terms, 1.62% CER corresponds to roughly 16 character edits per 1,000 reference characters after the stated normalisation rules. The closest external configuration, Google Chirp 3, recorded 2.26%.

For a government committee, a board or a project team, the practical reading is in the examples further down: which systems kept 200 files and the scores 98 and 94, which kept the Sprint Planning and VAT terms in a form a search box will find later, which wrote SDK as SDK and which spelled it out in Arabic letters, and which returned nothing for a greeting shorter than a second. Those are the failures that make a rapporteur go back to the audio, and the evaluation was built to catch them.

The M3 system records the lowest character error rate against six external ASR model configurations on the same human-verified meetings

Lower is better. Character error rate (CER) after defined lexical normalisation.

Same source audio, same human-verified reference, same scoring rules for every entry. This comparison evaluates transcription only. External configurations received the same participant-separated segments, so external speaker diarization was not tested. Evaluation snapshot September 2026. See "How the Arabic benchmark was measured" for the scoring rules.

The M3 system was benchmarked against six external ASR model configurations. One evaluation set, one set of rules. M3 at 1.62% is followed by Google Chirp 3 at 2.26%, Cohere Transcribe Arabic at 4.93%, ElevenLabs Scribe v1 at 5.01%, Deepgram Nova-3 at 6.09%, Google Latest Long at 9.01% and Sonix at 9.8%. The table adds character accuracy (100 minus CER) and the relative reduction of M3 versus each configuration, from 28% against Google Chirp 3 to 83% against Sonix. Read the reduction column as the gap on this evaluation set, not as a ranking of the external configurations by quality.

Character error rate and character accuracy on Arabic meetings for the M3 system and the six external ASR model configurations, with M3's relative reduction versus each configuration. Lower CER is better. Measured on the same human-verified meetings; scoring rules under How the Arabic benchmark was measured.
Rank	Evaluated entry	CER (%), lower is better	Character accuracy (%)	M3 vs this entry
1	M3	1.62	98.38	baseline
2	Google Chirp 3	2.26	97.74	28% lower (0.64 points)
3	Cohere Transcribe Arabic	4.93	95.07	67% lower (3.31 points)
4	ElevenLabs Scribe v1	5.01	94.99	68% lower (3.39 points)
5	Deepgram Nova-3	6.09	93.91	73% lower (4.47 points)
6	Google Latest Long	9.01	90.99	82% lower (7.39 points)
7	Sonix	9.8	90.2	83% lower (8.18 points)

The practical result. M3's advantage was not limited to a lower aggregate error rate. Across the reviewed examples, it more consistently retained numbers, short utterances, product names, and English technical terms in a form that remains readable and searchable in a business record.

Arabic examples: where a record breaks, segment by segment

The cards show the human-verified reference, the M3 system's output and every external configuration's output for individual segments, copied verbatim from the evaluation. Speaker labels are fictional placeholders. Each card carries the source clip: an excerpt of an internal Lisan meeting, the same audio every system transcribed. Each card was chosen because a reader without Arabic can still see the failure: an empty cell, a missing digit, a Latin term that survived or did not, a switch into another script. Where M3 itself makes a mistake, the caption says so. Under the scoring rules a phonetic spelling of an English term is not an error, so some of the differences affect readability and search rather than the CER.

Arabic example 1English terminology

SDK, rendering, editor, and Chrome extension

Source clip

Preserved term
Unsupported text
Filler-like insertion
No transcript

M3

صباح الخير بالنسبة لي امبارح كان معظم الشغل مع تيم العربي الجديد بخصوص الـ SDK (preserved term) اتسكر تقريبا مشكلتين و إلهن علاقة بالـ rendering (preserved term) الـ highlights (preserved term) بالإديتور اللي معمل له الـ Configuration (preserved term) من عندن بصفى مشكلة وحدة طلبنا منهم اجتماع مشان نكمل معهم اليوم ان شاء الله، كمان امبارح صرت اشتغل على موضوع الـ Chrome extension (preserved term) بين كوركتو و Lisan (preserved term) ما كثير يعني خلصت هذا الموضوع، اليوم ان شاء الله بدي كمل مثل ما قلت بخصوص العربي الجديد وكمان الـ Chrome extension مع كوركتو، يعطيكم العافية.

Reference

صباح الخير، بالنسبة لي امبارح كان معظم الشغل مع تيم العربي الجديد بخصوص الـ SDK اتسكر تقريبا مشكلتين ولهم علاقة بالـ rendering للـ highlights بالـ editor اللي معمل له configuration من عندن بصفى مشكلة وحدة طلبنا منن اجتماع مشان نكمل معن اليوم ان شاء الله كمان امبارح صرت اشتغل على موضوع الـ Chrome extension بين كوركتو و Lisan ما كتير خلصت هذا الموضوع اليوم إن شاء الله بدي كمل متل ما قلت بخصوص العربي الجديد وكمان الـ Chrome extension مع كريكتو يعطيكم العافية

M3

صباح الخير بالنسبة لي امبارح كان معظم الشغل مع تيم العربي الجديد بخصوص الـ SDK (preserved term) اتسكر تقريبا مشكلتين و إلهن علاقة بالـ rendering (preserved term) الـ highlights (preserved term) بالإديتور اللي معمل له الـ Configuration (preserved term) من عندن بصفى مشكلة وحدة طلبنا منهم اجتماع مشان نكمل معهم اليوم ان شاء الله، كمان امبارح صرت اشتغل على موضوع الـ Chrome extension (preserved term) بين كوركتو و Lisan (preserved term) ما كثير يعني خلصت هذا الموضوع، اليوم ان شاء الله بدي كمل مثل ما قلت بخصوص العربي الجديد وكمان الـ Chrome extension مع كوركتو، يعطيكم العافية.

ElevenLabs Scribe v1

آخر (not supported by the audio) بالنسبة لي، امبارح كان معظم الشغل عـ-- (not supported by the audio) مع team العربي الجديد بخصوص الـ SDK (preserved term)، آآآ (filler-like insertion) فـ-فـ- (not supported by the audio)آآآ (filler-like insertion) تسكر تقريبًا مشكلتين و آآآ (filler-like insertion) إلـ (not supported by the audio) إلهم علاقة بالـ آآآ (filler-like insertion) بالـ rendering (preserved term) للـ highlights (preserved term) بالـ editor (preserved term) اللي معمله configuration (preserved term) من عندهم. آآآ (filler-like insertion) بصفا مشكلة واحدة طلبنا منهم اجتماع مشان نكمل آآآ (filler-like insertion) معهم اليوم إن شاء الله. كمان امبارح آآآ (filler-like insertion) صرت أشتغل على موضوع الـ Chrome extension (preserved term) بين Correcto و Lisan (preserved term). آآآ (filler-like insertion) ما كتير آآآ (filler-like insertion) يعني آآآ (filler-like insertion) خلصت هذا الموضوع. اليوم إن شاء الله بدي أكمل مثل ما قلت بخصوص العربي الجديد وكمان الـ Chrome extension مع Correcto. يعطيكم العافية

Why this example matters. A daily stand-up full of developer vocabulary. M3 keeps SDK, rendering, highlights, configuration, Chrome extension and Lisan in Latin script, which is what a search across your archive will find. ElevenLabs Scribe v1 keeps them too, with heavy filler. Google Chirp 3 is complete but phonetic, which the scoring rules accept.

Arabic example 2Filler and punctuation

Sprint Planning and VAT

Source clip

Preserved term
Unsupported text
Filler-like insertion
No transcript

M3

صباح الخير يعطيكم العافية. امبارح كان في عندنا متابعة باجتماعات الـ Sprint Planning (preserved term) كان في كمان متابعة مع الشي المطلوب من هيئة الزكاة والضرائب بالسعودية موضوع الـ VAT (preserved term) ضريبة القيمة المضافة. كنت عم بتابع بهدول المواضيع يعطيكم العافية.

Reference

صباح الخير يعطيكم العافية امبارح كان في عنا متابعة باجتماعات السبرنت بلاننج كان في كمان متابعة مع الشي المطلوب من هيئة الزكاة والضرائب بالسعودية موضوع الفات ضريبة القيمة المضافة كان كنت عم بتابع بهدول المواضيع يعطيكم العافية

Why this example matters. A project update containing Sprint Planning and VAT. M3 preserves both terms in Latin script and keeps the Arabic expansion of VAT, making the record easier to search and read.

Arabic example 3Short speech

A sub-second spoken greeting

Source clip

Preserved term
Unsupported text
Filler-like insertion
No transcript

M3

كيفكم؟

Reference

كيفكم؟

Why this example matters. A greeting shorter than a second. M3, Cohere Transcribe Arabic, Sonix and Google Chirp 3 match the reference exactly. Deepgram Nova-3 and Google Latest Long returned nothing, which scores as deletions, and ElevenLabs Scribe v1 produced an English word that was not spoken.

Arabic example 4Numbers and terms

Files, quality figures, and model work

Source clip

Preserved term
Unsupported text
Filler-like insertion
No transcript

M3

صباح الخير امبارح خلصت الملفات كلهم طلعوا جوا تبع TranslateX (preserved term) حوالي 200 (preserved term) file (preserved term) الدقه طلعت يعني بالـ style (preserved term) حوالي 98 (preserved term) يعني بالـ translation (preserved term) كجوده بال 94 (preserved term) هلا هذا بعد طبعا عده تعديلات هلا اليوم بدي ارفع الـ version (preserved term) وكان في كمان شغل على MOE (preserved term) حاولنا نعمل fine tuning (preserved term) هلأ اليوم كمان حأتابع

Reference

صباح الخير امبارح خلصت الملفات كلهم تبع نجوى تبع TranslateX هلأ هنن حوالي 200 file الدقه طلعت يعني بالـ style حوالي 98 يعني بالـ translation كجوده بال 94 هلأ هذا بعد طبعا عدة تعديلات هلأ اليوم بدي ارفع الـ version وكان في كمان شغل على MOE حاولنا نعمل fine-tuning، هلأ اليوم كمان حتابع بنفس الموضوع

Why this example matters. Three spoken figures: 200 files, a style score of 98, and a translation quality score of 94. M3 preserves all three, along with TranslateX, version, MOE, and fine tuning. Cohere Transcribe Arabic and Google Chirp 3 also preserve the three figures, while several other systems alter or omit one or more values or technical terms. M3 omits the name Najwa in this segment.

Arabic example 5English terminology

SDK, dark theme, web app, and monitoring

Source clip

Preserved term
Unsupported text
Filler-like insertion
No transcript

M3

صباح الخير، بالنسبة ليوم الخميس الشغل كان مع فضاءات، كان في عندهم مشاكل العلاقة بالـ SDK (preserved term) والـ suggestions (preserved term) اللي عم تطلع، انحل اغلب الاخطاء، صفيان وحدة بدنا نعمل كول عليها اليوم إن شاء الله، كمان بدي اشتغل بالـ بنسخة الـ dark theme (preserved term) بالـ web app (preserved term)، وكمان بدي أعمل monitoring (preserved term) للنسخة الجديدة تبع الـ AI agent (preserved term)، يعطيكم العافية.

Reference

صباح الخير، بالنسبة إلي يوم الخميس الشغل كان مع فضاءات، كان في عندهم مشاكل العلاقة بالـ SDK والـ suggestions اللي عم تطلع، انحل اغلب الاخطاء، صفيان وحدة بدنا نعمل كول عليها اليوم إن شاء الله، كمان بدي اشتغل بالـ بنسخة الـ dark theme بالـ web app، وكمان بدي أعمل monitoring للنسخة الجديدة تبع الـ AI agents، يعطيكم العافية.

Why this example matters. Consecutive product terms. M3 keeps SDK, suggestions, dark theme, web app, monitoring, and AI agent in Latin script. The reference uses the plural “AI agents,” while M3 returns the singular “AI agent.”

Arabic example 6Dialects and terminology

Saudi dialect, Kubernetes, scaling, and paper

Source clip

Preserved term
Unsupported text
Filler-like insertion
No transcript

M3

صباح الخير كنت أمس أشتغل على موضوع في الـ Kubernetes (preserved term) من ناحية scaling (preserved term) كان في مشكلة بسيطة قاعد أدور لها حل وأيضا قاعد أشوف الموضوع البيبر ويعطيكم العافية

Reference

صباح الخير كنت أمس أشتغل على موضوع في الـ Kubernetes من ناحية scaling كان في مشكلة بسيطة قاعد أدور لها حل وأيضا قاعد أشوف الموضوع البيبر ويعطيكم العافية

Why this example matters. A Saudi-dialect status update containing Kubernetes, scaling, and paper. M3 preserves Kubernetes and scaling in Latin script and captures the full update. ElevenLabs Scribe v1 preserves all three English terms in Latin script, while several other systems render them phonetically or omit them.

How each system behaved on the Arabic evaluation

Qualitative. Beyond the CER, the reviewers noted how the M3 system and each external ASR model configuration handled English terms inside Arabic, content at the start of segments, fillers and event tags, punctuation and repetition. No separate aggregate rates were calculated, and the reviewers' phrase behind every cell is published under the matrix, so the strong, mixed, weak and not rated levels can be checked against the wording.

Google Chirp 3 deserves the credit the review gives it: strong handling of English technical terminology within Arabic speech, low content loss, and substantially fewer empty outputs than Google Latest Long. On several example segments it is as complete as M3, with the difference limited to phonetic rather than Latin spellings. ElevenLabs Scribe v1 preserved English terminology particularly well but produced frequent spurious filler-like vocalisations and one switch into an unexpected language or script. Cohere Transcribe Arabic was highly accurate on Classical Arabic, while the review identified content omissions, inconsistent terminology, repeated punctuation and repetition loops. Deepgram Nova-3 showed participant-name errors and segment-start omissions. Google Latest Long rejected many non-speech segments correctly but missed a small number of speech-bearing ones and frequently omitted English terms. Sonix showed errors in names and English terms, segment-start omissions and excessive sentence-final punctuation. The M3 system and all six external ASR model configurations produced Arabic transcripts across the dialects represented in this evaluation set.

M3 recorded the lowest CER and, in the reviewers' notes, preserved business-critical content, numbers and terminology more reliably and did not show the same concentration of filler, tag or repeated-punctuation artifacts. Speaker identification was not evaluated for the external configurations; each received participant-separated audio prepared by the M3 capture workflow and was scored on transcription only.

The benchmark includes seven evaluated entries: the M3 system and six external ASR model configurations. The matrix covers nine criteria, with one level per cell. The M3 column carries the brand rule only; status colours never mark emphasis. The final row repeats the measured CER.

Observed output behaviour by criterion and system, from the reviewers' notes on the Arabic evaluation set. Levels follow one rule from the report's wording: strong where the reviewers state a positive result, weak where they state a defect, mixed where the note is qualified or says not separately scored or no recurring issue was isolated, and not rated where the criterion was not evaluated. The reviewers' phrase behind each cell is in the notes below. Separate aggregate rates were not calculated.
	M3	Cohere Transcribe Arabic	ElevenLabs Scribe v1	Deepgram Nova-3	Google Latest Long	Google Chirp 3	Sonix
Classical Arabic	Strong	Strong	Strong	Strong	Weak	Strong	Strong
English inside Arabic	Strong	Weak	Strong	Mixed	Weak	Strong	Weak
Text omission	Strong	Weak	Mixed	Weak	Mixed	Strong	Weak
Punctuation	Mixed	Weak	Mixed	Mixed	Mixed	Mixed	Weak
Spelling and terminology	Strong	Weak	Mixed	Weak	Weak	Strong	Weak
Word repetition	Strong	Weak	Mixed	Mixed	Mixed	Mixed	Mixed
Dialects	Strong	Strong	Strong	Strong	Strong	Strong	Strong
Speaker identification	Strong	Not rated	Not rated	Not rated	Not rated	Not rated	Not rated
Filler words and event tags	Strong	Weak	Weak	Mixed	Mixed	Mixed	Mixed
Character error rate (CER), lower is better	1.62%	4.93%	5.01%	6.09%	9.01%	2.26%	9.8%

Reviewer notes behind each cell

Classical Arabic: M3 Highly accurate. · Cohere Transcribe Arabic Highly accurate. · ElevenLabs Scribe v1 Accurate. · Deepgram Nova-3 Accurate. · Google Latest Long Many errors. · Google Chirp 3 Accurate. · Sonix Accurate.
English inside Arabic: M3 Preserved English technical and business terminology more consistently within Arabic speech. · Cohere Transcribe Arabic English technical and business terms were sometimes omitted, mistranscribed, or rendered inconsistently within Arabic speech. · ElevenLabs Scribe v1 English technical terminology was preserved particularly well in the reviewed output. · Deepgram Nova-3 English-term performance was not scored separately; reviewed examples showed inconsistent rendering alongside segment-start omissions. · Google Latest Long English technical terms were frequently omitted. · Google Chirp 3 English technical terminology was preserved strongly within Arabic speech. · Sonix Names and English technical terminology were frequently transcribed incorrectly or omitted.
Text omission: M3 Preserves business-critical content more reliably and reduces the loss of important numbers, terminology, and organisation or product names. · Cohere Transcribe Arabic May omit or distort business-critical content, including numbers, technical terms, and organisation or product names. · ElevenLabs Scribe v1 No separate omission rate was calculated; the principal qualitative issue was inserted filler-like content rather than a recurring segment-start omission pattern. · Deepgram Nova-3 Substantial content loss was observed, particularly at the beginning of source segments. · Google Latest Long Correctly rejected many non-speech segments, but missed a small number of speech-bearing segments and sometimes omitted English terms or numbers. · Google Chirp 3 Low content loss was observed, with substantially fewer empty outputs than Google Latest Long. · Sonix Content loss was observed particularly at the beginning of source segments.
Punctuation: M3 Not separately scored; M3 produced clearer sentence boundaries, but punctuation was not evaluated as a separate aggregate measure. · Cohere Transcribe Arabic Repeated punctuation patterns that reduce readability were observed in the reviewed output. · ElevenLabs Scribe v1 Not separately scored; readability was affected more by filler-like insertions than by a recurring punctuation defect. · Deepgram Nova-3 Not separately scored; no recurring punctuation defect was isolated in the qualitative review. · Google Latest Long Not separately scored; no recurring punctuation defect was isolated in the qualitative review. · Google Chirp 3 Not separately scored; no recurring punctuation defect was highlighted in the qualitative review. · Sonix Punctuation quality was weak, with excessive sentence-final marks observed in the reviewed output.
Spelling and terminology: M3 Produces more consistent business terminology for reading and search. · Cohere Transcribe Arabic Technical and business terms may appear inconsistently or in phonetic form. · ElevenLabs Scribe v1 English technical terms were handled well, although filler-like insertions reduced overall transcript cleanliness. · Deepgram Nova-3 Participant names were frequently transcribed incorrectly. · Google Latest Long English technical terms were frequently missed, and some numbers were omitted. · Google Chirp 3 English technical terms were rendered reliably within Arabic speech. · Sonix Performance was weak on participant names and English technical terminology.
Word repetition: M3 Showed substantially fewer repetition problems in the reviewed output. · Cohere Transcribe Arabic Token and phrase repetition loops were observed in the reviewed output. · ElevenLabs Scribe v1 No separate repetition rate was calculated; frequent filler-like vocalisations were the more prominent insertion issue. · Deepgram Nova-3 No recurring repetition issue was isolated in the qualitative review. · Google Latest Long No recurring repetition issue was isolated in the qualitative review. · Google Chirp 3 No recurring repetition issue was isolated in the qualitative review. · Sonix No recurring word-repetition issue was isolated; punctuation was the more prominent readability problem.
Dialects: M3 Produced Arabic transcripts across the dialects represented in this evaluation set. · Cohere Transcribe Arabic Produced Arabic transcripts across the dialects represented in this evaluation set. · ElevenLabs Scribe v1 Produced Arabic transcripts across the dialects represented in this evaluation set. · Deepgram Nova-3 Produced Arabic transcripts across the dialects represented in this evaluation set. · Google Latest Long Produced Arabic transcripts across the dialects represented in this evaluation set. · Google Chirp 3 Produced Arabic transcripts across the dialects represented in this evaluation set. · Sonix Produced Arabic transcripts across the dialects represented in this evaluation set.
Speaker identification: M3 Uses participant-linked audio channels and platform participant information. Platform-provided speaker attribution remained linked throughout the evaluated meetings. · Cohere Transcribe Arabic Not evaluated for the external configurations. Each configuration received the same participant-separated audio segments prepared by the M3 capture workflow and was scored only on transcription. · ElevenLabs Scribe v1 Not evaluated for the external configurations. Each configuration received the same participant-separated audio segments prepared by the M3 capture workflow and was scored only on transcription. · Deepgram Nova-3 Not evaluated for the external configurations. Each configuration received the same participant-separated audio segments prepared by the M3 capture workflow and was scored only on transcription. · Google Latest Long Not evaluated for the external configurations. Each configuration received the same participant-separated audio segments prepared by the M3 capture workflow and was scored only on transcription. · Google Chirp 3 Not evaluated for the external configurations. Each configuration received the same participant-separated audio segments prepared by the M3 capture workflow and was scored only on transcription. · Sonix Not evaluated for the external configurations. Each configuration received the same participant-separated audio segments prepared by the M3 capture workflow and was scored only on transcription.
Filler words and event tags: M3 Did not show the same concentration of filler-like, event-tag, or repeated-punctuation artifacts in the review. · Cohere Transcribe Arabic Filler-like insertions, event-style tags, and repeated-punctuation artifacts were observed in the reviewed output. · ElevenLabs Scribe v1 Frequent spurious filler-like vocalisations such as “آآآ” were observed, and one segment switched to an unexpected language or script. · Deepgram Nova-3 No recurring filler-word or event-tag pattern was isolated in the qualitative review. · Google Latest Long Handled many non-speech segments correctly, although a small number of speech-bearing segments were also rejected. · Google Chirp 3 No recurring filler-word or event-tag pattern was highlighted in the qualitative review. · Sonix No recurring event-tag pattern was isolated; excessive punctuation was the more prominent artifact.

English benchmark

English results: character error rate (CER)

This section reports the English comparison. It uses the same metric as the Arabic benchmark, character error rate, on a separate English evaluation set, so its values are read within their own source material rather than as a ranking of the two languages.

M3 recorded 1.83%. In practical terms, 1.83% CER corresponds to roughly 18 character edits per 1,000 reference characters after the stated normalisation rules. The closest external configuration, Google Chirp 3, recorded 3.18%, a gap of 1.35 percentage points: M3's CER is 42% lower on this English set. Against the other five external configurations the relative reduction ranges from 48% to 77%. These are comparisons on this evaluation set, not universal vendor rankings.

1.83%M3 English CERLowest value in the English table

98.17%English character accuracy100 minus the M3 CER

42% to 77%Lower English CERM3 versus each external configuration

7Evaluated entriesM3 and six external ASR model configurations

On English meetings, the M3 system also records the lowest character error rate against the six external ASR model configurations

Lower is better. Character error rate (CER) after the stated normalisation rules.

Same source audio, same human-verified reference, same scoring rules for every entry, with every system set to English. This comparison evaluates transcription only. Character accuracy and M3's relative reduction are derived from the aggregate CER values. See "How the English benchmark was measured" for the model and language settings.

Character error rate and character accuracy on English meetings for the M3 system and the six external ASR model configurations, with M3's relative reduction versus each configuration. Lower CER is better. Accuracy and relative reduction are derived from the aggregate CER values; settings under How the English benchmark was measured.
Rank	Evaluated entry	CER (%), lower is better	Character accuracy (%)	M3 vs this entry
1	M3	1.83	98.17	baseline
2	Google Chirp 3	3.18	96.82	42% lower (1.35 points)
3	Cohere Transcribe Arabic	3.54	96.46	48% lower (1.71 points)
4	Sonix	4.21	95.79	57% lower (2.38 points)
5	Deepgram Nova-3	5.58	94.42	67% lower (3.75 points)
6	ElevenLabs Scribe v1	6.41	93.59	71% lower (4.58 points)
7	Google Latest Long	7.98	92.02	77% lower (6.15 points)

English qualitative review by criterion

This second English table applies the same qualitative structure used for Arabic. It separates short-utterance handling, omissions, punctuation, names and terminology, repetition, speaker identification and unsupported insertions, instead of turning aggregate CER into a second one-row ranking. Its source is the English reference-error review; the final row repeats the aggregate CER.

Observed English output behaviour by criterion and system. Qualitative levels summarise the English reference-error review; the final row repeats the separately reported aggregate CER values. The assignment method is in the notes below.
	M3	Cohere Transcribe Arabic	ElevenLabs Scribe v1	Deepgram Nova-3	Google Latest Long	Google Chirp 3	Sonix
Overall English transcription	Strong	Strong	Weak	Mixed	Weak	Strong	Mixed
Short utterances	Strong	Strong	Mixed	Mixed	Weak	Mixed	Mixed
Text omission	Strong	Mixed	Strong	Mixed	Mixed	Strong	Weak
Punctuation	Mixed	Mixed	Weak	Mixed	Mixed	Mixed	Mixed
Names, numbers and terminology	Strong	Strong	Mixed	Weak	Mixed	Mixed	Weak
Word repetition	Strong	Strong	Weak	Weak	Mixed	Mixed	Mixed
Speaker identification	Not rated	Not rated	Not rated	Not rated	Not rated	Not rated	Not rated
Unsupported fillers and insertions	Strong	Strong	Weak	Mixed	Mixed	Mixed	Mixed
Character error rate (CER), lower is better	1.83%	3.54%	6.41%	5.58%	7.98%	3.18%	4.21%

Reviewer notes and assignment method

Overall English transcription: The level is the displayed CER band: Strong at 4% or lower, Mixed above 4% and below 6%, and Weak at 6% or higher.
Short utterances: Two clips in the English review are shorter than five seconds. M3 and Cohere Transcribe Arabic match both references; Google Latest Long shows the clearest loss on the short clips, while the other four configurations show minor transcript differences.
Text omission: The reviewed outputs show the lowest deletion concentration for ElevenLabs Scribe v1, Google Chirp 3 and M3; Cohere Transcribe Arabic, Deepgram Nova-3 and Google Latest Long form a qualified middle group; Sonix shows the most concentrated content loss. ElevenLabs Scribe v1 is rated strong on omission only; its insertions are rated in a separate row.
Punctuation: Punctuation is excluded from CER, so every entry stays Mixed unless the review shows a recurring defect. ElevenLabs Scribe v1 is Weak here because it produced substantially more punctuation than the reference and repeated punctuation sequences.
Names, numbers and terminology: The review checked that names, acronyms, numerals and business terms such as RFP, OSS, IP, Matrix and MVP stayed visible. M3 has the most consistent result; Deepgram Nova-3 and Sonix show the most recurring name or terminology loss.
Word repetition: M3 shows no unsupported adjacent repeat and Cohere Transcribe Arabic one isolated repeat. Repetition is concentrated in ElevenLabs Scribe v1 and Deepgram Nova-3; Google Latest Long, Google Chirp 3 and Sonix show a qualified middle pattern.
Speaker identification: Speaker identification was not scored as a separate accuracy metric in the English benchmark. M3 kept the platform participant link as a product-workflow capability; the external configurations received participant-separated clips and were evaluated for transcription only. Every entry is therefore Not rated in this row.
Unsupported fillers and insertions: M3 has the cleanest reviewed output. ElevenLabs Scribe v1 shows the clearest recurring filler and insertion pattern; the other entries are rated from the relative concentration visible in the reviewed outputs.
Scope: The qualitative rows summarise the English reference-error review and follow the same assignment rule as the Arabic matrix: Strong for a clear positive result, Weak for a recurring or concentrated defect, Mixed for a qualified result, and Not rated where the criterion was not evaluated. They are not additional CER measurements or statistical-significance claims. Only the final row repeats the aggregate CER values.

English examples: three selected clips from the evaluation

For each example, every system's output is shown alongside the human-reviewed reference transcript.

The examples are qualitative illustrations and do not recalculate the aggregate English CER. Human-verified reference transcripts are available for all three clips, so each output can be compared directly with what was spoken.

English example 1Terminology and number

Core42 and Compass in a two-second turn

Source clip

Preserved term
Unsupported text
Filler-like insertion
No transcript

M3

access to core 42 (preserved term) compass models (preserved term), right?

Reference

access to core 42 compass models, right?

Why this example matters. M3 matches the reference exactly, keeping “core 42,” “compass models” and the closing “right.” Cohere Transcribe Arabic also keeps the wording, while the other five configurations alter one or more of these terms.

English example 2Single-word meaning

“Clear” versus “good” or “great” changes the meeting record

Source clip

Preserved term
Unsupported text
Filler-like insertion
No transcript

M3

No, no, everything is clear (preserved term) for now. Thanks.

Reference

No, no, everything is clear for now. Thanks.

Why this example matters. The reference says “clear,” and M3 keeps it, as do Cohere Transcribe Arabic, ElevenLabs Scribe v1 and Deepgram Nova-3. Google Chirp 3 and Sonix substitute “good” and “great,” and Google Latest Long turns the answer into “Not everything is clear,” which reverses it.

English example 3Role and participant name

Note taker and “Adam” in a short question

Source clip

Preserved term
Unsupported text
Filler-like insertion
No transcript

M3

This is not your note taker (preserved term), Adam (preserved term)?

Reference

This is not your note taker, Adam?

Why this example matters. M3 matches the reference exactly, keeping the negation, the role “note taker” and the participant name “Adam.” Cohere Transcribe Arabic also matches; the other five configurations add a filler, omit a word, or alter the role, the name, or both.

How the English benchmark was measured

The English evaluation uses real English-language business meetings. This page reports one explicitly English-configured entry for the M3 system and for each of the six external ASR model configurations.

Source material
Real English-language business meetings. Every scored segment was paired with an English reference transcript verified by a human reviewer listening to the source audio.
Model and language settings
The reported M3 entry used M3 + soxr + 1 s silence with --language en. Every external configuration was also set to English before transcription: Google Chirp 3 (Speech-to-Text V2, model chirp_3) and Google Latest Long (Speech-to-Text V1, model latest_long) with en-US; ElevenLabs Scribe v1 (scribe_v1) with eng; Deepgram Nova-3 (nova-3) with en; Cohere Transcribe Arabic (cohere-transcribe-arabic-07-2026, Cohere's latest transcription model, which covers Arabic and English) with language=en; and Sonix with en. For the separate Arabic evaluation, Cohere Transcribe Arabic used language=ar rather than automatic language detection.
Same evaluation inputs
M3 and all six external ASR model configurations were evaluated on exactly the same audio clips and segment boundaries, against the same human-verified reference transcripts.
Metric
All English figures on this page are character error rates: character substitutions, deletions and insertions divided by the number of reference characters after normalisation.
Normalisation
Punctuation, letter case and whitespace are ignored. The exception is a missing space that fuses two words together, which counts as one error.
Derived reporting
Character accuracy is 100 minus CER. M3's relative reduction versus another entry is (that entry's CER minus M3's CER) divided by that entry's CER, rounded to the nearest whole percent.

Evidence map for the English benchmark

Claim	Status
M3 1.83% CER versus 3.18% for Google Chirp 3, 3.54% for Cohere Transcribe Arabic, 4.21% for Sonix, 5.58% for Deepgram Nova-3, 6.41% for ElevenLabs Scribe v1 and 7.98% for Google Latest Long.The aggregate English results table; lower CER is better.	Reported in the results.
The English evaluation uses real English-language business meetings with human-verified reference transcripts.A human reviewer verified every scored reference while listening to the source audio.	Verified in the evaluation process.
Every evaluated system was explicitly set to English before transcription.M3 + soxr + 1 s silence with en; Google Chirp 3 and Google Latest Long with en-US; ElevenLabs Scribe v1 with eng; Deepgram Nova-3, Cohere Transcribe Arabic and Sonix with en.	Verified in the run metadata.
All seven entries were scored on exactly the same audio clips with the same normalisation rules.Punctuation, letter case and whitespace are ignored; a missing space that fuses two words counts as one error.	Confirmed for the English evaluation.
M3 character accuracy is 98.17%, and its CER is 42%, 48%, 57%, 67%, 71% and 77% lower than the six external configurations respectively.Arithmetic on the reported aggregate CER values; relative reductions rounded to whole percentages.	Derived from reported values.
The English matrix summarises short-utterance handling, text omission, punctuation, names and terminology, repetition, speaker identification and unsupported insertions for the seven entries.Criterion-level review of the reference-error analysis. The overall row uses the displayed CER bands; the other rows are qualitative summaries of the review, not additional aggregate rates.	Observed in the review; assignment notes under the matrix.
Three short English clips reproduce the source audio, the M3 output and the outputs of the six external configurations.Human-verified reference transcripts are available for all three clips.	Documented qualitatively against the human-verified references.

Speed, and what the figure covers

Validated. The M3 meeting transcription workflow processes a one-hour meeting in 45 seconds on an L4 GPU, 80x real time. The figure is batch throughput for recorded audio, measured on a meeting-duration basis; live delivery, while a meeting is in progress, is measured separately. For a minutes workflow it describes the transcription stage: how long M3 takes to process an hour of recorded audio once it is available. Speaker identity, timestamps and transcript order are preserved through processing (product capability).

One hour of meeting audio in 45 seconds

80x real time on an L4 GPU.

Batch throughput of the M3 meeting transcription workflow on an L4 GPU, for recorded audio.

Inside MeetriX

Product capability. The benchmark scores transcription only. In MeetriX, M3 sits inside a longer workflow: audio is captured with participant identity attached from the moment of capture; silence, short turns and speech boundaries are handled before transcription; multi-dialect Arabic is transcribed with English names and terms written in their recognised business form; every segment stays linked to its speaker, meeting time and source audio; and a post-meeting stage produces the summary, key points, decisions and next steps that are emailed to the participants under your access, consent and retention policies. In the evaluated deployment, the transcription path called no external transcription service, and M3 can run within customer-controlled infrastructure so audio, transcripts and summaries stay inside your data-governance boundary. Platform-provided speaker attribution remained linked throughout every evaluated meeting in the M3 participant-channel workflow; this is a coverage statement, and no speaker-attributed error rate is reported.

Arabic and English transcription: dialects, code-switching (Arabic and English inside one sentence), named speakers and custom vocabulary.
AI summaries: the summary, minutes and action items built from the transcript.
Security and on-prem: deployment options for customer-controlled infrastructure.

Note

Product capability, not benchmark. Summaries, minutes and delivery were not scored in this evaluation. The CER figures on this page cover transcription only.

How the Arabic benchmark was measured

For the Arabic benchmark, the M3 system and six external ASR model configurations were given the same material and scored by the same rules. The steps below condense the protocol and the scoring rules, and we share them before any pilot so your result is scored the same way.

Source material
Real multi-dialect Arabic business meetings, with human-verified reference transcripts prepared for every scored segment.
Evaluation-data separation
The meetings and human-verified reference transcripts used in this evaluation were not used to train M3.
Same audio for every system
The M3 system and the six external ASR model configurations received the same source audio, pre-segmented by the M3 capture workflow into participant-separated segments. External configurations were scored on transcription only; their speaker diarization, the step that splits audio by who is speaking, was not used or evaluated. Every figure for the external configurations on this page was produced by running the vendor's API on the evaluation audio; none is taken from a vendor publication.
Same evaluation inputs
The M3 system and all six external ASR model configurations were evaluated using the same audio segments and segment boundaries. All outputs were scored against the same human-verified references using the same normalisation and scoring rules. The reference transcripts were used only for scoring and were not provided to any system during transcription.
Configurations
Google Chirp 3 is Google Cloud Speech-to-Text V2 with model chirp_3 and ar-XA; Google Latest Long is Google Cloud Speech-to-Text V1 with model latest_long and ar-SA; ElevenLabs Scribe v1 is model scribe_v1 with language_code=ara; Cohere Transcribe Arabic is model cohere-transcribe-arabic-07-2026, Cohere's latest transcription model, set explicitly to language=ar for this Arabic evaluation (and to language=en for the separate English evaluation); Deepgram Nova-3 and Sonix (Sonix.ai) ran as named, with no further model string given. M3 ran as the complete meeting system with participant-linked audio channels from its capture workflow.
Why the comparison uses ElevenLabs Scribe v1
To verify that the use of Scribe v1 did not disadvantage ElevenLabs, we reran the same evaluation audio with Scribe v2 under the same input and scoring conditions. Scribe v2 did not improve transcription accuracy on this material, and Scribe v1 performed slightly better. We therefore retain the Scribe v1 result in the main comparison. This choice reflects the observed results on this evaluation set and should not be interpreted as a general ranking of ElevenLabs model versions.
Metric
CER = (substitutions + deletions + insertions) divided by the characters in the human-verified reference, after normalisation. Character accuracy is 100 minus CER.
Equivalences
A dialect spelling and its Modern Standard Arabic form count the same when the meaning is the same. An English word counts whether written in Latin letters, phonetically in Arabic script, or as an equivalent Arabic word. Numbers are compared by value (15, ١٥ and خمسة عشر are the same) and stay in the scored content, as do English business terms.
Ignored
Punctuation entirely; Latin case; Arabic letter variants such as the forms of alef are normalised; whitespace, except that a missing space that fuses two words counts as one error.
Remaining differences
Every remaining missing, added or substituted character is an error. Only segments with a verified reference enter the CER. An empty output against a speech-bearing reference counts as deletions; extra unsupported output counts as insertions.
Qualitative review
Reviewers recorded content omission, names and English terminology, filler-like insertions, unexpected language changes, repeated punctuation and repetition loops for the M3 system and each external ASR model configuration. No separate aggregate rates were calculated.

Evidence map for the Arabic benchmark

Claim	Status
M3 1.62% CER versus 2.26% for Google Chirp 3, 4.93% for Cohere Transcribe Arabic, 5.01% for ElevenLabs Scribe v1, 6.09% for Deepgram Nova-3, 9.01% for Google Latest Long and 9.8% for Sonix; relative reductions of 28%, 67%, 68%, 73%, 82% and 83%.The same human-verified evaluation set of real meetings, same reference, same scoring rules.	Measured.
The behaviour of the M3 system and each external ASR model configuration as summarised in the matrix (name errors, segment-start omissions, filler-like vocalisations, a language or script switch, non-speech rejection, strong English-term handling, excessive sentence-final punctuation).Qualitative review of the same evaluation set.	Observed qualitatively; separate aggregate rates not calculated.
Platform-provided speaker attribution remained linked throughout every evaluated meeting.M3 participant-channel workflow; external configurations evaluated for transcription only.	Measured for the M3 participant-channel workflow.
80x batch throughput; 45 seconds per hour of meeting audio.M3 meeting transcription workflow for a one-hour meeting on an L4 GPU.	Validated; batch throughput for recorded audio.
No external transcription call.Evaluated transcription path.	Verified in the evaluated deployment.

Limitations

One evaluation set per language. Every CER figure comes from a single human-verified evaluation set of real meetings in its language, one Arabic and one English. Each shows how the M3 system and the six external ASR model configurations compared on that material, not a universal rate for any of them. Your meetings, dialect mix and microphones are the benchmark that matters.
Separate language-specific evaluation sets. Both the Arabic and English figures are CER, but the two sets use different source material. Compare systems within each language table; do not read 1.62% and 1.83% as a ranking of the two languages.
A snapshot in time. The benchmark was run in September 2026 using the language-specific configurations described in the English and Arabic methodology sections. Vendors update their models, so the figures describe the versions available at the time of each run.
A small margin at the top. In the Arabic benchmark, the gap to Google Chirp 3 is 0.64 percentage points, and on several example segments Google Chirp 3 is as complete as M3.
Throughput scope. The 45 seconds per hour and 80x figures describe batch processing of recorded audio on one hardware configuration, an L4 GPU. Live latency is measured separately.
Qualitative observations carry no aggregate rates. The matrix levels follow one rule applied to the wording of the reviewer notes, shown with their phrases so they can be challenged.
The English qualitative levels are review summaries. Only the overall-transcription row uses the displayed CER thresholds. The other Strong, Mixed, Weak and Not rated cells summarise the reference-error review and the observed transcript differences; they are not additional aggregate rates or statistical-significance tests.
External speaker diarization was not evaluated. Every external configuration received participant-separated audio prepared by M3. The comparison isolates transcription and says nothing about how those configurations attribute speakers on mixed audio. The M3 speaker-attribution result is a coverage statement; no speaker-attributed error rate is reported.
M3 is not perfect. The Arabic examples show a spoken first name omitted, a singular where the reference has a plural, and words added that the reference does not contain.

The canonical report lives on lisan.com. Cite that page.

Cite this page

Lisan Research (2026). M3: multilingual meeting transcription, measured on real meetings. Arabic and English, September 2026. Lisan, published 5 October 2026. https://lisan.com/company/research/m3/

BibTeX

@techreport{lisan2026m3,
  author = {Lisan Research},
  title = {M3: multilingual meeting transcription, measured on real meetings},
  institution = {Lisan},
  type = {Evaluation report},
  year = {2026},
  month = {October},
  note = {Arabic and English, September 2026. Published 5 October 2026},
  url = {https://lisan.com/company/research/m3/},
}

Executive summary

Arabic results: character error rate (CER)

What 1.62% means for a meeting record

The M3 system records the lowest character error rate against six external ASR model configurations on the same human-verified meetings

Arabic examples: where a record breaks, segment by segment

SDK, rendering, editor, and Chrome extension

Sprint Planning and VAT

A sub-second spoken greeting

Files, quality figures, and model work

SDK, dark theme, web app, and monitoring

Saudi dialect, Kubernetes, scaling, and paper

How each system behaved on the Arabic evaluation

English results: character error rate (CER)

On English meetings, the M3 system also records the lowest character error rate against the six external ASR model configurations

English qualitative review by criterion

English examples: three selected clips from the evaluation

Core42 and Compass in a two-second turn

“Clear” versus “good” or “great” changes the meeting record

Note taker and “Adam” in a short question

How the English benchmark was measured

Source material

Model and language settings

Same evaluation inputs

Metric

Normalisation

Derived reporting

Evidence map for the English benchmark

Speed, and what the figure covers

One hour of meeting audio in 45 seconds

Inside MeetriX

How the Arabic benchmark was measured

Source material

Evaluation-data separation

Same audio for every system

Same evaluation inputs

Configurations

Why the comparison uses ElevenLabs Scribe v1

Metric

Equivalences

Ignored

Remaining differences

Qualitative review

Evidence map for the Arabic benchmark

Limitations

Cite this page

Run M3 on your own meetings

Frequently asked questions