Nimdzi LIVE!

The episode contains genuine practitioner-level findings - annotator bias toward AI output, asymmetric automated metric reliability, and annotation time variance - but these are diluted heavily by host summaries, off-topic tangents about vibe coding, and repetitive conversational padding. The useful signal-to-noise ratio is mediocre for a 61-minute runtime.

“I think the linguists, they just hate models. They somehow understood that that was a AI translation. So they mark every year because of. They just uh, don't like AI translations. I think they were biased. I think they were purely biased.”

“Some of them spent like half an hour for the, uh, whole annotation and some of them spent like around two hours. Uh, and it's a mystery for me why.”

There is one genuinely novel idea - a future positive-reinforcement dataset that teaches models to improve translations rather than just flag errors - and the asymmetric reliability of automated metrics is an underappreciated point. The rest of the episode recycles standard localization-industry talking points: AI won't fully replace translators yet, agentic future, human in the loop.

“I foresee and have an idea of uh, another kind of metric or the way uh, the language model is taught to translate... maybe there is a future world when we encourage machine to translate better and just give them positive feedback so that they not spot the euros but make the translation better.”

“going to production uh, without people is not yet, not yet there. So we are very excited sometimes about the quality of translation that AIs provide to us.”

Alexander Murauski is a 20-year LSP founder with genuine engineering depth who personally ran the study under discussion, giving him direct practitioner authority. He falls short of the top tier - some answers are uncertain or speculative, and the study itself is explicitly framed as exploratory rather than definitive - but he is not a recycled thought-leader.

“We are just running Lots of automated quality, uh benchmarking of uh different MTs between different language pairs and different domains. Every project, every production project starts with um every client just comes uh and asks well I need to translate everything by AI.”

“Our uh, benchmarking on production projects shows that it's, it's really good. So I would uh, place it uh, just uh, somewhere near Gemini.”

The episode is anchored by real numbers from an actual study: 45 linguists, 34 hours, 6 days, 16 language pairs (10 supported + 6 unsupported), a Japanese inter-annotator kappa of 0.4, annotation times ranging from 15 minutes (Portuguese) to 90 minutes (Hmong), and named tools (Metric X, COMET Kiwi, Vertex AI, Hugging Face). The study is small and exploratory, and some claims - like the Vertex AI anomaly - remain unexplained, which limits the score.

“45 different linguists spend a total of 34 different hours over six days to actually analyze the output”

“Japanese, uh the 0.4 is quite high agreement, it's moderate. So they are more or less agreeable.”

The host occasionally demonstrates good instincts - probing the 99% pass threshold, asking about linguist resistance to AI - but too often fills airtime reading from the report aloud, making extended analogies (plumber, calculator), and letting speculative claims about agentic localization go entirely unchallenged. The conversation is collegial rather than disciplined.

“I just want to read this inter annotator agreement, why multiple annotators matter and what low agreement actually tells us. With three evaluators per language working independently, we measured how often they agree and the answer is not very often.”

“Are linguists generally receptive to working on projects like this that are essentially like hybrids using technology but still wanting to enforce human quality evaluation?”