MLOps.community

There are a few genuinely useful technical ideas buried in a lot of filler, tangents, and demo fumbling - notably Dylan's cheap statistical routing signal using retrieval score spread, and the four-way memory taxonomy. Most of the runtime is throat-clearing, restating obvious points, or vague gestures toward papers without substantive unpacking.

“you can do some statistic based on like the spread of those scores or uh, how the difference between the, the top one and the top second results. And basically from here try to create a programmatic um, you know signals that will, that that will be able to tell you if uh, the quality of your retrieval was good or not without virtually any adding any compute to your pipeline”

“there is working one which is the context window... there is uh, semantic or factual memory... Then there is a procedure, procedural one. It's actually more like skills... And the fourth one would be episodic one. This perfect thing for the vector search.”

The statistical score-spread routing signal is a modestly fresh operational tip, and the pointer to RL-for-retrieval is timely, but virtually everything else - graph+vector complementarity, synthetic golden datasets, memory types, BM25 limitations - is standard circulating knowledge in the MLOps community with no novel framing.

“from here try to create a programmatic um, you know signals that will, that that will be able to tell you if uh, the quality of your retrieval was good or not without virtually any adding any compute to your pipeline”

“reinforcement learning, uh, for agentic retrieval is the way to get to that next five order of magnitude efficient um, agentic retrieval when the context window gets bigger”

All guests are Qdrant developer relations staff, not independent senior operators who have shipped agentic retrieval at scale; Dylan's background at Arize AI adds some practitioner credibility but the panel is fundamentally a vendor dev-rel roundtable, which limits the depth and independence of the insights offered.

“So Dylan, I'm going to hand to you to talk a little bit about what you found”

“I come from Arise AI, so I'm, I might, my opinion might be biased there”

A handful of concrete references appear - NDCG/MRR, BM25 being five orders of magnitude smaller in memory, Raspberry Pi deployment, the SARAH and SID1 papers, cogni and mem0 as named companies - but no actual benchmark numbers, latency figures, or customer results are cited, and paper references are described at a very high level without substantive data.

“early search primitives like the BM25s of the world are already five orders of magnitude smaller in terms of memory that they consume on the machine”

“this version can run on a Raspberry PI and not even like the top of the line newest one. I have one form like five years ago that runs it just fine”

The host moves the conversation along reasonably and occasionally surfaces good follow-up angles (prompt injection, RL for evals, forgetting mechanisms), but he frequently restates rather than probes, lets vague claims pass unchallenged, and loses the thread mid-conversation - reducing the episode's potential depth.

“okay, there's a lot to unpack with all three of you saying this stuff because basically let me just say what I caught real fast and then you can correct me if I'm wrong”

“now my memory is starting to fade. Uh, trying to hold all these things in my hand head. The super intelligent retrieval agents were doing what now?”