Awkward Silences

The episode delivers substantive, non-obvious insights about qualitative research and AI throughout. Sam Ladner clearly articulates the distinction between 'unriddling' (sense-making) and correlation-finding, explains why outliers require explanation in qual work (not dismissal), and provides concrete guidance on when to use AI (closed coding, transcription) versus when to reserve human judgment (emotional interpretation, storytelling). However, some sections drift into tangential personal anecdotes (dream analysis, exercise classes) that dilute density.

“You can't offload cognitive sense making to machines. Machines don't think and they certainly don't make intuitive leaps and they don't, um, have abductive logic. They're basically big correlation machines.”

“Qualitative research is not correlation hunting. That's not what it is and it never has been. It's more, it's closer to unriddling than it is to predicting what is going on here.”

Ladner offers a fresh framing of AI's role in qual research by emphasizing what it *cannot* do (think, feel, imagine) rather than chasing use cases. The 'unriddling' metaphor is memorable and contrarian to the prevailing 'AI will automate research' narrative. However, some core arguments - sentiment analysis's limitations, the value of outliers in qual research, the need for researcher judgment - are relatively established within qualitative methodology circles, not groundbreaking.

“AI is psychopathic. It really is like literally psychopathic. Has absolutely no interest whatsoever in serving human beings.”

“AI cannot imagine. It can only use what's known. It's a prediction machine. It uses past information. It can't come up with something entirely new.”

Sam Ladner is a genuinely credible practitioner: published author of two books (Practical Ethnography, Mixed Methods), writing a third, and working at Workday in a research capacity. She has direct, hands-on experience conducting qualitative research at scale and is clearly reflexive about her own practice (graduate training, evolution of methods, ongoing writing). This is not a career podcaster or pure theorist, but someone doing the work and willing to challenge hype.

“I had professors who were telling me, that's a really good thing. You'll learn the data really clearly and deeply, which was true.”

“So now that I'm not as busy coding, I can do that work. I've seen a lot of people start doing that now because they've had, you know, feedback from, you know, their stakeholders saying, yeah, well you know, what's the difference between you and AI?”

Ladner provides concrete tool recommendations (MaxQDA, Atlas Ti, Nvivo) and describes specific workflows (closed coding with predetermined code systems, using UX honeycomb frameworks). She gives a concrete example of analyzing an outlier (someone with 'trauma induced data fear'). However, she offers few hard metrics, timelines, or financial data; most examples are illustrative rather than backed by numbers or specific case studies from her own work.

“Like the user journey is a pretty good one. Uh, the UX honeycomb is also a really normal one. You know, you're going to see, oh, these are all the things where they've, you know, shown that we're not providing a good user experience because we haven't got accessibility.”

“I tell the AI this is my coding system. And you know, I've had, I've done this before many times. Like I've had a coding system that's a pre existing coding system.”

Hosts Erin May and Carol Guest ask sharp, follow-up questions and gently push Ladner on claims (e.g., 'But they're not doing it mathematically. If they're doing it right' prompt real clarification; 'Wait, okay. And so Chat GPT doesn't know what's up with all the folklore. So how are you AI ing it?' pushes on a half-explained tangent). However, the conversation allows some digressions (dream analysis, memoir recommendations) to consume airtime without tight redirection, and lacks truly adversarial moments testing core claims.

“So let me ask a related question, right? So totally understood on the mathematical nature of sentiment analysis and of asking AI to do it at scale. Compare that to a person, a human being, tagging what they're hearing in interviews.”

“Yeah. Well, I love what you're saying because it's something that I've thought about a lot doing this over the years. Right. Is the human part of qualitative research is very important and that's only coming, I think, more to the forefront as AI is more of a part of it.”