Parakeet TDT v3 runtime benchmark · M1 Pro · 16 GB · macOS 14.7.3
The model and recordings stay fixed. The runtime implementation changes. Compare the official and community Parakeet TDT v3 runtimes on one Mac, using direct native requests instead of Wasper.app.
Short dictation
7 complete runtimes of one model on one Mac. Lower word error rate is better. Higher processing speed is better. Focus or hover a point for exact values.
Choose a workload
One model, one Mac, two recording lengths.Choose a workload to update the map and full comparison together.
Runtime results
Each row runs the same model through its own local runtime.Wasper uses its Metal server and Axiom's custom Metal kernels.
| Artifact | ||||
|---|---|---|---|---|
Wasper Metal INT8Wasper · Wasper 1.8.0 | osa911/wasper-parakeet-tdt-0.6b-v3-onnx-int8 · v6INT8 encoder | 9.33% WER | 82.8× | 115.2 ms |
MLX Community F32/BF16MLX Community · parakeet-mlx 0.5.2 / MLX 0.32.2 | mlx-community/parakeet-tdt-0.6b-v3F32 weights, BF16 runtime | 8.36% WER | 28.8× | 342.5 ms |
Local MLX INT8Wasper · parakeet-mlx 0.5.2 / MLX 0.32.2 | Local group-64 INT8 derivativeBased on MLX CommunityINT8 weights, group 64 | 8.40% WER | 47.2× | 211.4 ms |
Handy Q8Handy Computer · transcribe.cpp 0.2.3 / 63a44d9239 | handy-computer/parakeet-tdt-0.6b-v3-gguf Q8_0Q8_0 | 8.63% WER | 52.3× | 190.0 ms |
NVIDIA Q8NVIDIA · NeMo-Speech.cpp 0.1.0 / 4f96762 | nvidia/parakeet-tdt-0.6b-v3 Q8_0 GGUFQ8_0 | 8.59% WER | 47.1× | 212.0 ms |
Istupakov ONNX INT8Istupakov · onnx-asr 0.12.0 / ONNX Runtime 1.30.0 | istupakov/parakeet-tdt-0.6b-v3-onnxINT8 | 9.82% WER | 33.9× | 293.5 ms |
Fluid Core MLFluidInference · FluidAudio / 69e42dae8e | FluidInference/parakeet-tdt-0.6b-v3-coremlMixed precision | 8.59% WER | 12.8× | 775.1 ms |
Method and sources
The model, source recording, and Mac stay fixed. Only the local runtime implementation changes. This is a runtime comparison, not an app workflow comparison.
Published September 2026 baseline. Wasper displayed version 1.8.0 but was a locally rebuilt development binary, not necessarily the public 1.8.0 release. The frozen snapshot does not include its exact binary hash.
Hardware, operating system and Wasper builds differ between runs. These results do not isolate the effect of macOS, and displayed app versions do not establish identical binaries.
Google FLEURS test set supplied 243 prompted recordings: 27 each in German, Greek, English, Spanish, French, Italian, Dutch, Polish, and Portuguese.
21 complete talks, speeches, readings, and addresses. Each source stays with its original publisher.
Each runtime received the same complete source recording directly on one Mac. Every row runs Parakeet TDT v3 through its local runtime implementation. The Wasper result measures the native Parakeet server, not Wasper.app.
Response-only timing after one discarded warm-up request. The frozen public aggregate does not preserve matching short- and long-recording memory cohorts, so this comparison does not publish sortable physical-memory results.
This is a local derivative, not a Wasper download. Start from MLX Community’s FP32 model, then create and save a separate eight-bit, group-64 copy. The original model remains unchanged.
parakeet-mlx==0.5.2 and mlx==0.32.2.config.json and tokenizer files into a new model folder."quantization": {"bits": 8, "group_size": 64} to the copied configuration.model = parakeet_mlx.from_pretrained(source, dtype=mx.float32)
nn.quantize(model, bits=8, group_size=64)
mx.eval(model.parameters())
model.save_weights("local-int8/model.safetensors")To match this benchmark, load the copied configuration before the saved weights, use BF16 activations, and process long audio in 120-second windows with 15 seconds of overlap.
Every runtime received the complete source recording. MLX Community and Local MLX INT8 handle long recordings internally as 120-second windows with 15-second overlap; that is their runtime-owned behavior, not benchmark-created segmentation.
Evidence SHA-2565b229c5b24408f7d439fb92924c2bfdda5ccb7419e0d134198980ebfdf8fa819
Method and sources
The model, source recording, and Mac stay fixed. Only the local runtime implementation changes. This is a runtime comparison, not an app workflow comparison.
October 1, 2026 additional run. Runner 4000ea7 with shutdown timeout locally increased from 5 to 60 seconds. Installed Wasper displayed 1.8.0 and was built locally. The operator confirms the local change was README-only, with no runtime code changes. Native server SHA-256: bdd71e87e1bc032c0b056205a9c55c8e2ca964b76826cfa77525627f3bb9b076. Machine, RAM, macOS marketing version and AC power are reported by the operator; the structured evidence records arm64 and Darwin 27.0.0.
Hardware, operating system and Wasper builds differ between runs. These results do not isolate the effect of macOS, and displayed app versions do not establish identical binaries.
All 5,544 expected outcomes were saved: 5,430 successes and 114 long-recording errors. Every runtime completed 729/729 short requests. Missing detailed failure logs prevent attributing the long errors to a particular cause. Raw transcripts are not published.
Google FLEURS test set supplied 243 prompted recordings: 27 each in German, Greek, English, Spanish, French, Italian, Dutch, Polish, and Portuguese.
21 complete talks, speeches, readings, and addresses. Each source stays with its original publisher.
Each runtime received the same complete source recording directly on one Mac. Every row runs Parakeet TDT v3 through its local runtime implementation. The Wasper result measures the native Parakeet server, not Wasper.app.
Response-only timing after one discarded warm-up request. Physical footprint was sampled after each response. These samples are not peak memory or total system memory requirements; no sortable memory ranking is published.
This is a local derivative, not a Wasper download. Start from MLX Community’s FP32 model, then create and save a separate eight-bit, group-64 copy. The original model remains unchanged.
parakeet-mlx==0.5.2 and mlx==0.32.2.config.json and tokenizer files into a new model folder."quantization": {"bits": 8, "group_size": 64} to the copied configuration.model = parakeet_mlx.from_pretrained(source, dtype=mx.float32)
nn.quantize(model, bits=8, group_size=64)
mx.eval(model.parameters())
model.save_weights("local-int8/model.safetensors")To match this benchmark, load the copied configuration before the saved weights, use BF16 activations, and process long audio in 120-second windows with 15 seconds of overlap.
Every runtime received the complete source recording. MLX Community and Local MLX INT8 handle long recordings internally as 120-second windows with 15-second overlap; that is their runtime-owned behavior, not benchmark-created segmentation.
Evidence SHA-256b4c25996b2ad568a0d8a0c86b356cfb26ebd3b0e7b5bb8e68c9ebe1c4fa9a4bd
Method and sources
The model, source recording, and Mac stay fixed. Only the local runtime implementation changes. This is a runtime comparison, not an app workflow comparison.
October 1, 2026 additional run. Runner 4000ea7 with shutdown timeout locally increased from 5 to 60 seconds. Installed Wasper displayed 1.8.0 and was built locally. The operator confirms the local change was README-only, with no runtime code changes. Native server SHA-256: bdd71e87e1bc032c0b056205a9c55c8e2ca964b76826cfa77525627f3bb9b076. Machine, RAM, macOS marketing version and AC power are reported by the operator; the structured evidence records arm64 and Darwin 27.0.0.
Hardware, operating system and Wasper builds differ between runs. These results do not isolate the effect of macOS, and displayed app versions do not establish identical binaries.
All 5,544 expected outcomes were saved: 5,430 successes and 114 long-recording errors. Every runtime completed 729/729 short requests. Missing detailed failure logs prevent attributing the long errors to a particular cause. Raw transcripts are not published.
Google FLEURS test set supplied 243 prompted recordings: 27 each in German, Greek, English, Spanish, French, Italian, Dutch, Polish, and Portuguese.
21 complete talks, speeches, readings, and addresses. Each source stays with its original publisher.
Each runtime received the same complete source recording directly on one Mac. Every row runs Parakeet TDT v3 through its local runtime implementation. The Wasper result measures the native Parakeet server, not Wasper.app.
Response-only timing after one discarded warm-up request. Physical footprint was sampled after each response. These samples are not peak memory or total system memory requirements; no sortable memory ranking is published.
This is a local derivative, not a Wasper download. Start from MLX Community’s FP32 model, then create and save a separate eight-bit, group-64 copy. The original model remains unchanged.
parakeet-mlx==0.5.2 and mlx==0.32.2.config.json and tokenizer files into a new model folder."quantization": {"bits": 8, "group_size": 64} to the copied configuration.model = parakeet_mlx.from_pretrained(source, dtype=mx.float32)
nn.quantize(model, bits=8, group_size=64)
mx.eval(model.parameters())
model.save_weights("local-int8/model.safetensors")To match this benchmark, load the copied configuration before the saved weights, use BF16 activations, and process long audio in 120-second windows with 15 seconds of overlap.
Every runtime received the complete source recording. MLX Community and Local MLX INT8 handle long recordings internally as 120-second windows with 15-second overlap; that is their runtime-owned behavior, not benchmark-created segmentation.
Evidence SHA-256b4c25996b2ad568a0d8a0c86b356cfb26ebd3b0e7b5bb8e68c9ebe1c4fa9a4bd
Method and sources
The model, source recording, and Mac stay fixed. Only the local runtime implementation changes. This is a runtime comparison, not an app workflow comparison.
Published September 2026 baseline. Wasper displayed version 1.8.0 but was a locally rebuilt development binary, not necessarily the public 1.8.0 release. The frozen snapshot does not include its exact binary hash.
Hardware, operating system and Wasper builds differ between runs. These results do not isolate the effect of macOS, and displayed app versions do not establish identical binaries.
Google FLEURS test set supplied 243 prompted recordings: 27 each in German, Greek, English, Spanish, French, Italian, Dutch, Polish, and Portuguese.
21 complete talks, speeches, readings, and addresses. Each source stays with its original publisher.
Each runtime received the same complete source recording directly on one Mac. Every row runs Parakeet TDT v3 through its local runtime implementation. The Wasper result measures the native Parakeet server, not Wasper.app.
Response-only timing after one discarded warm-up request. The frozen public aggregate does not preserve matching short- and long-recording memory cohorts, so this comparison does not publish sortable physical-memory results.
This is a local derivative, not a Wasper download. Start from MLX Community’s FP32 model, then create and save a separate eight-bit, group-64 copy. The original model remains unchanged.
parakeet-mlx==0.5.2 and mlx==0.32.2.config.json and tokenizer files into a new model folder."quantization": {"bits": 8, "group_size": 64} to the copied configuration.model = parakeet_mlx.from_pretrained(source, dtype=mx.float32)
nn.quantize(model, bits=8, group_size=64)
mx.eval(model.parameters())
model.save_weights("local-int8/model.safetensors")To match this benchmark, load the copied configuration before the saved weights, use BF16 activations, and process long audio in 120-second windows with 15 seconds of overlap.
Every runtime received the complete source recording. MLX Community and Local MLX INT8 handle long recordings internally as 120-second windows with 15-second overlap; that is their runtime-owned behavior, not benchmark-created segmentation.
Evidence SHA-2565b229c5b24408f7d439fb92924c2bfdda5ccb7419e0d134198980ebfdf8fa819
Each row links to the model bundle and runtime version used here. The Local MLX INT8 row includes the group-64 recipe for the locally derived runtime.