After my child was born I wanted a diary of how they were growing and how our relationship was changing, so I built an iPhone app that records the day and leaves a draft in the evening. Transcription and "is this voice mine" run on the phone. Only the summaries go to Apple's cloud (Private Cloud Compute, PCC). The records never leave the phone.
Over about three months of building it, these are the five things that were not in Apple's documentation and that I ended up measuring on a real phone.
1. In the background, CPU work runs on slow cores. The Neural Engine is unaffected.
A stage that takes 33 seconds in the foreground had not finished after 5 minutes in the background (BGContinuedProcessingTask). Apple's developer forums have a thread about the same thing: "4–5× slower in the background, cause unknown". I also assumed the Neural Engine was slow in the background. When I measured it, the Neural Engine stayed within 2×; it was the CPU path that was slow. Same job (10 minutes of Japanese speech to text) under different conditions, iPhone 16, iOS 27, median of two runs:
| Compute units | Foreground | Background (continued processing) | Background, while recording |
|---|---|---|---|
| Neural Engine | 2.2 s | 4.3 s (2×) | 2.4 s |
| CPU only | 8.4 s | 102–174 s (12–21×) | 7.3 s |
How to read the table: in the foreground the job takes 2.2 s on the Neural Engine and 8.4 s on the CPU. Handed to a background continued-processing task, the Neural Engine still finishes in 4.3 s, but the CPU-only version takes 102–174 s. In the background while recording (microphone open), both run at foreground speed.
Why only the CPU? I read the process's CPU time and energy from task_info. CPU-only inference in the foreground: 8.7 s elapsed, 7.9 s of CPU time. In the background: 112 s elapsed, 86 s of CPU time, at one quarter of the energy. So the background process is still getting CPU time for almost the whole elapsed period, yet it needs 11× the CPU time to finish the same task. That rules out "the scheduler is starving it". The explanation that fits best is "it is running on the efficiency cores at a low clock" (I have not looked at core placement in Instruments, so treat it as an inference).
The audio is processed in segments of up to 20 s, so I also looked at the time per segment. In the background it is slow from the very first segment (1.7 s per segment against 0.14 s in the foreground), and some stretches run at about 8 s per segment. Apple's forums have an older report that "even in the foreground, after 30 s of sustained load the threads get moved to efficiency cores"; this is a different phenomenon, because it is slow from the start.
Three lessons. If you are slow in the background, suspect the stages that run on the CPU (preprocessing, clustering, post-processing), not the model. Do not call Core ML from a .background-priority Task: it is 3.6× slower even in the foreground, while the CPU path does not care about priority. While the microphone is open, the background runs at foreground speed, so finish the heavy work while recording. I changed the app accordingly, and a laughter recount that used to take 167 s after recording now takes 0 s.
2. Measured on read speech, the ranking of conversation recognizers flips
On read speech (FLEURS Japanese): Apple's recognizer 4.7%, Parakeet 5.3%. On 258 minutes of broadcast conversation: Parakeet 12.7%, Apple 23.0%. Apple returns nothing for some clips and inserts a "thank you very much" that nobody said. Had I chosen by read speech, I would have switched and regretted it.
One more thing to watch. When I added IBM's Granite to the comparison, whether or not 0.5 s of silence was padded around the audio moved the error rate by 3 points. With equal conditions Granite and Parakeet are the same, and on real recordings Parakeet drops fewer phrases, so the app stays on Parakeet.
3. Same character error rate, three times the errors in the summaries
I transcribed the same conversations with two recognizers, generated a summary from each transcript, and compared the summaries blind. Topics containing an error: 9 of 32 versus 3 of 29. The character error rates were the same. Users read the summary, so I had been using the wrong yardstick. The summarizer's prompt also worked backwards: telling it "only use numbers that appear in the transcript" made it copy misheard numbers verbatim ("two or three companies" became "23 companies") and increased the errors. The recognizer's per-word confidence is around 0.95 whether the word is right or wrong, so it cannot be used as a weight.
4. How often Apple's PCC refuses a production request as "inappropriate"
That Apple's guardrails refuse too many ordinary requests has been reported many times on the developer forums, for the on-device model. I could not find a rate for the cloud side (PCC) in production, so I counted 41 days of production logs from my own iPhone.
| Calls | Failed | Refused as "inappropriate" | Rate limited | Daily quota hit |
|---|---|---|---|---|
| 614 | 130 | 65 (10.6%) | 59 (9.6%) | 0 |
What gets refused is ordinary diary drafts and summaries of conversations; a recording of a technical meeting gets refused too. A refusal takes 7.6–12.7 s, longer than a success (1–3 s), because the response starts streaming before the filter stops it. Rate limiting kicks in at 8–10 calls per minute, which my morning batch hit every day. There is a daily quota too: on a development phone, one day of 1,158 calls had 689 fail with the quota error. To be honest, I had not read the API that returns resetDate, and I did not know the quota is per iCloud account and shared with other apps.
5. Whatever I fed it, my "copy" agreed with me only 62–67% of the time. What identifies a person is how they talk.
I had an LLM stand in for me and measured how often it made the same choices I would. The material: a one-line bio, 20,000 tokens of diary, 31 answers about my values, a 12-question interview. Whatever I gave it, agreement with my own choices was 0.62–0.67. The one-line bio was the strongest (0.72), and adding my own answers made it worse.
How I talk, on the other hand, clearly changes with the listener. Pauses between utterances: 1.6 s with coworkers, 2.2 s with my wife, 2.8 s with my daughter, 5.3 s with my boss. Sentence length: 20 characters with family, 32 with coworkers. The research points the same way: people can identify someone they know from the melody of the voice alone, even family members cannot tell a written impersonation from the real person, and the bereaved judge "that sounds like them" by tone rather than by whether the facts are right. What makes a person recognizable is how they say things, not what they say.
So the app keeps not only the polished text but the words as spoken, the voice numbers (pitch, rate, pauses, laughter) and who you were talking to, for your own segments only. Other people's voices are not kept, and the audio itself is deleted after 24 hours.
What is not there yet
iOS and watchOS only. The app is closed source; the measurement probe and its reports are on GitHub (yamannyama/granite-speech-coreml). Japanese first; English is not yet measured. Per-feature refusal rates and a full-day battery figure are being logged. The diary and export are free; the daily morning note, conversation records, pictures and music are Pro ($3.99/month).
Happy to answer questions about background processing, PCC in production, HealthKit quirks, comparing recognizers, and what did not work.