Stop Building AI Tools to Avoid Talking to Your Users — The Solo Operator's Version

Ashley Rolfmore's post "Stop trying to engineer your way out of listening to people" is on the Hacker News front page today. It's aimed at product and design teams. I'm writing the solo operator version because I've done exactly what she's describing, and so has every indie builder I know who's been at it more than a year.

The pattern: you have a product. You have users. You need to understand what your users want, or why they churn, or why onboarding is broken. The uncomfortable path is to call five of them and listen for 20 minutes each. The comfortable path is to build something. A dashboard. A feedback widget. An AI-summarized NPS pipeline. A session replay tool. A Claude-powered transcript analyzer. The comfortable path looks productive. It isn't. It's elaborate avoidance.

I know this because I've built most of those things. I'm going to describe what happened, what I learned, and what I do now. It's the post I need to read regularly, and I figure if I need it then so does someone else.

The Tools I Built So I Wouldn't Have to Call Anyone

Over the last two years I've shipped the following, at various times and for various projects:

A feedback widget that captures sentiment and tags it automatically. It collected hundreds of submissions. The signal density was terrible. People who fill out feedback widgets are either furious or ecstatic, and the ecstatic ones don't tell you anything useful.

A dashboard that tracked where users dropped off in onboarding, down to the individual step. I could tell you that 34% of signups quit at step three. I could not tell you why. The dashboard told me where to look and refused to tell me what to look for.

A session replay integration. I watched probably forty hours of user sessions. I learned roughly one genuinely useful thing per ten hours of watching. Most of it was "yes, that button is confusing," which I already suspected. The cost-per-insight was abysmal.

An NPS pipeline that ran responses through Claude to produce "thematic summaries." The summaries were coherent. They were also generic. "Users appreciate the speed but find the pricing confusing" is the kind of insight you could have written before running the pipeline, because it describes every SaaS in existence.

A churn survey auto-email. 12% response rate. The responses skewed heavily toward one pattern — the dissatisfied vocal minority — and missed the silent majority whose reasons for churning were more diffuse.

None of this was useless. But if I total up the time spent building and maintaining these tools versus the decisions they actually influenced, the ratio is bad. The ratio would have been better if I'd spent that time on something else.

What Actually Worked

Three 20-minute calls per week. That's it. That's the whole methodology.

The format, specifically:

I don't prepare a list of questions. I don't send a survey ahead. I ask the user to walk me through the last time they tried to do the thing my product exists to help them do. Not the abstract version — the specific last time. "When was the last time you sat down to [whatever it is]? What happened? Walk me through it."

Then I shut up.

Most people will talk for fifteen minutes straight if you let them. In those fifteen minutes I learn more than I did in a week of dashboard watching. Not because the dashboard is lying, but because the dashboard doesn't know what to look for. The user does. They just needed me to ask the right question, which turns out to be "what did you actually do?" rather than "what would you like?"

I take notes during the call, not a transcript. A transcript is tempting — you can run it through Claude, extract themes, store it forever — but the act of typing a transcript makes you listen worse. You listen for what's easy to write down rather than what matters. Paper notes, with arrows and underlines, force you to synthesize in real time. That's what you're actually there for.

After the call I write three bullet points in my notebook. Not "the user said X, Y, and Z." The three most surprising things. If nothing was surprising, that's also a signal — either I'm talking to the wrong users, or I already know my product well enough, or (most often) I didn't listen hard enough.

Three calls a week is sixty minutes of calls, plus maybe forty-five minutes of prep and follow-up. An hour and forty-five minutes weekly. That's the budget. Every tool I built to replace this was, in retrospect, an attempt to avoid that hour and forty-five.

The AI-Research-Synthesis Trap Specifically

The 2026 version of this avoidance has a specific flavor. You run your call transcripts through Claude. You ask it to extract themes. You save the themes to a vector database. You build a dashboard of what your users are saying. It is very sophisticated. It is also, almost always, a layer of abstraction between you and the thing you needed to hear.

Claude is good at summarizing. Claude cannot tell you what your user didn't say — the pause before the word "easy," the grimace when you asked about pricing, the fact that they stopped using the product two weeks ago and are being polite about it now. Those are the parts of a user interview that matter. Those are not in the transcript.

This is not an argument against transcription. I transcribe my calls now, because Fireflies is running in the background and I'd be foolish not to have the record. It's an argument against using the transcript as the primary artifact. The primary artifact is the thing you wrote in your notebook during the call. The transcript is a safety net for when your memory fails.

The Counter-Argument, Steelmanned

There are cases where scaled, tool-based feedback is genuinely the right answer. Let me be specific about what they are.

Pricing page A/B tests. You don't need to call anyone. You need the conversion rate on two versions of the page. Run the test. The number tells you the answer.

Onboarding funnel optimization. Same thing. The funnel metric is a measurement problem with a clean answer. You don't need qualitative research for this; you need a bigger sample.

Bug report triage. At scale, automated categorization of bug reports is fine and probably necessary. You don't need to manually read every "the button is broken" report.

Product direction. Feature prioritization. Understanding why people churn. Figuring out what your next product should be. None of these are measurement problems. They are sensemaking problems. And sensemaking does not scale through tools. It scales through people. Three calls a week.

The mistake is applying tools to sensemaking problems because they worked on measurement problems. The tools work. They solve the wrong problem. The problem is "I don't know what to build next," and no dashboard in the world is going to answer that. Three calls this week will give you more signal than three quarters of instrumentation.

The Uncomfortable Core

The reason we build these tools is not that they work. It's that they feel productive and they don't require us to sit on a Zoom with someone who might say something we don't want to hear. That's the entire mechanic. Once you name it out loud, it's hard to keep pretending.

I've watched founders — myself included — spend months building internal tooling to avoid a week of user calls. The internal tooling feels like infrastructure. The user calls feel like emotional labor. But the calls are the actual work. The tooling is a way of looking busy while the business decision gets made without any new information.

The fix is not clever. Put three 20-minute calls on your calendar this week. Don't prepare. Ask one question. Listen. Write three bullet points after.

If you don't want to, ask yourself why. That answer — the real one — is usually where the useful thing is.