Software Testing Unleashed

The episode contains concrete, practical applications of LLMs in testing workflows (keyword documentation, code review with reasoning models, defect translation, accessibility testing), which are genuinely useful. However, the insights are scattered and somewhat iterative in presentation style; the host doesn't push for deeper analytical frameworks or challenge underlying assumptions, resulting in a conversational walkthrough rather than densely packed learning.

“we decided to add little AI functionality in the test management tool... you press a button it reads through the implementation of the keyword and it comes up with a header”

“we ended up with O1 and did it there. And it worked actually quite well. So it catches most of the more obvious stuff but you get it like this right within a second or two you do typically two or three cycles and then formally the keyword looks pretty good”

The guest discusses specific use cases (RAG-based test specs, reasoning models for reviews, alternative text generation for accessibility) that show thoughtful experimentation, but the underlying ideas - using LLMs for documentation, code review, and automation - are not novel in the broader AI-in-software engineering discourse. The framing around autonomous agents and testing AI is present but underdeveloped and somewhat reactive rather than offering fresh frameworks.

“we used an approach where we fed in additional project specific information. So it was a RAG approach, retrieved augmented retrieval augmented generation”

“if you look at the language models, you have to really be careful. about which year you're talking about because the reasoning models haven't been there from the beginning”

Klaudia Dussa Zieger is the ISDQB president and a consultant at IMUS (Imbus) working directly on AI testing implementations in a commercial test management tool. She brings hands-on practitioner experience with deployed AI features and candid assessment of where AI fails (reverse-engineering legacy code for unit tests). However, she is not a prominent founder or exec known for scaling a major testing company, which limits the caliber relative to top-tier guests.

“I'm speaking not as the ISDQB president, but as um a consultant coming from IMUS and we do have a test management tool”

“one of our X AI experts is working in these projects and I was asking him Do you think an AI could help... he says no. No. Because you need to look into the existing code”

The episode includes concrete tool features (keyword documentation buttons, O1-based reviews, defect translation to domain-user language, accessibility alt-text generation) and specific model choices (Llama 3.1 → Claude 3.5 for spec generation; O1 for reviews; standard models for docs). However, no quantitative results are provided - no metrics on defect-detection rates, time savings, accuracy improvements, or team adoption numbers. Claims like 'they were quite happy' and 'it worked really well' lack supporting data.

“we ended up with O1 and did it there. And it worked actually quite well. So it catches most of the more obvious stuff but you get it like this right within a second or two”

“He gets 95%. The last five percent he has to do by himself”

The host (Richie) asks open-ended questions and occasionally requests clarification, but rarely pushes back or probes disagreements. When the guest mentions self-healing test cases were abandoned at RoboCon, the host simply notes 'I need to dig a little deeper' rather than actually digging in the episode. There are few sharp follow-ups on limitations, ROI, or trade-offs; the conversation mostly flows as a guided tour of use cases rather than critical dialogue.

“when they tried to look at the usage of large language models for the digital accessibility tests. Because we have a group that just deals with digital accessibility testing... let me ask about high level of test automation. Where do you there see possibilities”

“When it came up last year we also had a discussion with our specialist... I need to dig a little deeper why they're not doing it”