Amud HaYomi is a daily-page Talmud cycle: one side of one daf דף folio — one leaf of the Talmud each day, every day. A maggid shiur מגיד שיעור the one who delivers the lecture who records a shiur שיעור lecture in Yiddish on each day's daf, reached out and asked if we could turn his RSS feed into an app, and we did: a catalog organized the way the Talmud itself is organized, by maseches מסכתא tractate , daf דף page , and amud עמוד side of the page . Simple enough, right?
As we were building it, we kept looking for places where AI could genuinely add something. What we came up with is the feature this article is about.
A shiur is 15-40 minutes of audio walking through a dense page of a mixture between Hebrew and Aramaic. Listeners follow along in their own Gemara volume, and the perennial experience is losing your place — which line is he on? There's even a Yiddish word for the recommended method of keeping your place: teitel טייטל lit. "pointing" — keeping your place on the daf . We also wanted the app to work for listeners picking up a shiur wherever they are, even without a physical volume in hand. We wanted the app to answer that the way Spotify answers it for lyrics: the text follows the audio, the current passage lights up, the page marker moves as the listener learns.
Other than which amud each audio file covered, we had nothing to work with. No transcript, no timestamps. The audio is in Yiddish. Not textbook Yiddish, but the Chassidic register spoken in Brooklyn, quoting Aramaic mid-sentence. Here is what shipped, and then the road to it.
מֵאֵימָתַי קוֹרִין אֶת שְׁמַע בְּעַרְבִין
מִשָּׁעָה שֶׁהַכֹּהֲנִים נִכְנָסִים לֶאֱכֹל בִּתְרוּמָתָן
עַד סוֹף הָאַשְׁמוּרָה הָרִאשׁוֹנָה דִּבְרֵי רַבִּי אֱלִיעֶזֶר
וַחֲכָמִים אוֹמְרִים עַד חֲצוֹת
רַבָּן גַּמְלִיאֵל אוֹמֵר עַד שֶׁיַּעֲלֶה עַמּוּד הַשַּׁחַר
מַעֲשֶׂה וּבָאוּ בָנָיו מִבֵּית הַמִּשְׁתֶּה
אָמְרוּ לוֹ לֹא קָרִינוּ אֶת שְׁמַע
Step 1 — Getting the words out of the audio
Off-the-shelf speech models fall apart on this audio. Generic Whisper (OpenAI's state-of-the-art speech-to-text model) doesn't just make errors on Chassidic Yiddish — it hallucinates, confidently producing sentences that are either complete nonsense or completely coherent but entirely unrelated to what's recorded. The fix was an open-source fine-tune: yi-whisper-large-v3-turbo, a Whisper variant fine-tuned on Yiddish. On this material the difference is night and day — it even gets the embedded Aramaic quotations more or less right, which turned out to be the part we needed most.
On an M4 16-core Mac, Whisper runs at about realtime — a 20-minute recording takes about 20 minutes to transcribe. So we converted the model to Apple's MLX framework, which brought it to roughly five times realtime. That's fast — if you're processing one episode a day. For a backlog of 800-plus recordings, it was hopeless. So we ended up running the backfill on a rented H100 GPU in the cloud, at roughly 207× realtime: the entire catalog — over five hundred hours of audio — transcribed for about $9, in under three hours.
Step 2 — Pinning words to the page
A transcript with timestamps is half the problem. The other half: figuring out where, in those 40 minutes of Yiddish explanation, the maggid shiur is reading the actual page. When he reads, his words match the canonical text; when he explains, they don't. Those moments of reading are our synchronization points — we call them anchors.
The matcher works on character trigrams (every three-letter window of a word — so two words count as similar when they share most of their three-letter pieces, even if they don't match exactly), which means it forgives transcription noise and spelling variation. Every candidate match is weighted by how rare the phrase is — a phrase like אמר רבי appears forty times a page and pins nothing, while a distinctive four-word run pins its location exactly. Then the candidates go through a longest-monotonic-chain pass: audio moves forward, so the matched positions must too, and any match that would require the lecturer to jump backward through the page is discarded.
After experimenting with different algorithms for this, the design principle that worked best was precision over recall. Anchors are deliberately sparse — the UI highlights the span between anchors rather than faking word-by-word karaoke. A gap in the highlight is invisible; a wrong highlight teaches the listener to distrust the feature. We only ever chose the gap.
Step 3 — The shape of the page
Talmud study has a fixed visual geography — the Vilna printing's layout, with the text in the middle and the classic commentaries wrapping around it, unchanged since the 1880s. Serious learners don't want a text column; they want their page. This is commonly called tzuras hadaf צורת הדף the shape of the page . So beside the teleprompter view there's a facsimile of the actual printed daf, with a marker that tracks the audio across the physical page.
That requires knowing where every word sits on the page image. The PDFs we were working with were OCR'd, with the recognized text embedded invisibly inside the PDF itself — but extracting that text and tying each word back to its position on the page was archaeological work. Because of the unique design of the page — the Talmudic text in the center, surrounded by meforshim מפרשים the classic commentaries printed around the text on both sides — pinpointing which words are the Talmud itself and which belong to a mefaresh מפרש commentator took a bit of fiddling. In the end, every word got a bounding box: four coordinates marking where it starts and ends on the page. We matched the boxes to our canonical text, and could then anchor the audio to the page itself — under the same principle as the text anchors: a marker pointing at the wrong line is worse than no marker.
Step 4 — LLMs get a veto, not a pen
After the deterministic pipeline ran, about half the episodes cleared the quality gate and went live. The rest sat in a gray zone: real anchors, decent coverage, not quite enough confidence to publish. The tempting move was to hand the transcript and the page to a large language model and ask it to produce better timestamps. We ruled that out early — an LLM asked to author timestamps will produce confident, plausible, but frequently wrong ones. And we found a more interesting failure mode: because LLMs understand intent, they tend to anchor the text not to the moment the maggid shiur is literally quoting it, but to where he is explaining it — a semantically reasonable answer that is exactly the wrong one for a follow-along. We wanted wrong anchors ruled out as far as possible, so LLMs don't get to write them.
Instead the LLMs got a narrower job: adjudication. For each gray-zone episode, the judge sees every machine-made anchor with its transcript context and votes keep or drop — it can only remove, never invent. We auditioned two judges, Claude Sonnet 4.6 and Gemini 3.1 Pro, and the finding was unambiguous: Gemini is just markedly better at grokking Chassidish Yiddish — the register, the Aramaic code-switching, the way a quote dissolves into explanation — so Gemini got the job. Episodes where at least 70% of anchors survive get promoted to production with the surviving subset. We validated the judge first, on episodes with trusted alignments: its median disagreement was 0.0 seconds, and its errors ran in the safe direction — it drops good anchors more readily than it keeps bad ones. The verify pass rescued 191 episodes, with Claude standing in as the fallback judge when Gemini's quota runs dry. Nothing an LLM wrote ever reaches a listener.
Step 5 — Making it run itself
A backfill is a project; a pipeline is a product. New shiurim arrive daily through the podcast RSS feed, and a nightly job picks up whatever is new: transcribe on the local MLX model, align, gate, send the gray zone through the LLM verify pass, and push whatever qualifies to production. Hand-corrected data is never overwritten, below-gate episodes queue for human review rather than shipping — and by the time the morning's listeners open today's amud, yesterday's shiur follows along.
What made it work
None of the pieces here are exotic — an open-source speech model, fuzzy string matching, PDF parsing, a couple of LLM API calls. What made it work was where we drew the trust boundaries: deterministic code decides, statistical models propose, language models veto, and anything below the confidence bar degrades to a quieter experience instead of a wrong one. That's a pattern we keep reaching for in AI systems that face real users: AI is at its best when its real, unique strengths are deliberately put to work — not when it's blindly handed the reins.
We're proud of what we accomplished on this project — see it live at shasinyiddish.com.