What the verbatim pass reveals about AI interview synthesis
Fathom delivers clean interview summaries. A follow-up verbatim pass with a fixed slot structure still found two spots where the synthesis claimed more than was said. That is no argument against AI tools. It is an argument for a second layer.
We use Fathom for the user interviews. The tool transcribes, summarises, suggests themes. The results are good. With a sample this small, trusting them would be the obvious mistake.
What synthesis tools do well
They condense. Fifty minutes of conversation become four paragraphs a human can skim in ten. They structure by theme, they surface standout quotes, they write in full sentences. For a first orientation after the interview, that is worth gold.
What they do worse
They smooth things over. "Maybe I would need that if I planned more often" becomes "the tester would use the material list regularly". A conditional statement with a caveat turns into a confirmation. Whoever reads only the synthesis builds product decisions on a statement that was never made that way.
In interview four we caught it twice. Both times the synthesis had assigned a recommendation as a solid insight that, in the transcript, was a conditional half-sentence. An "if that happened automatically" became "the tester wants automation". The loss of meaning is real, even if at first glance it looks merely semantic.
Verbatim pass with a slot structure
The fix is mundane. After the synthesis, a second run goes through the transcript, guided by a prompt with eight fixed slots. Per slot it asks: is there a literal quote on this topic, and if so which one. No paraphrasing, no smoothing, only sentences the person actually said.
Interview four produced eleven verbatim quotes. Two of them contradicted the synthesis, nine confirmed it. The two corrections meant that a cluster previously treated as confirmed got downgraded to "mentioned spontaneously in two interviews, n=2". That is more honest and thinner. Both are good.
The methodology point
The verbatim pass is no argument against synthesis tools. It is the second layer every synthesis needs. A tool does the first layer faster than any human. The second layer you have to do yourself, because it undoes exactly the smoothing the tool is optimised for.
Whoever turns interview data into product decisions should not trust the readability. Trust the spot where the tester faltered.