(2021-07-11) Chin Product Validation Frameworks Are Useless Without Taste

Cedric Chin: Product Validation Frameworks are Useless Without Taste. Product validation methodologies consist of two parts: an explicitly followable process, and a tacit component that may be best described as ‘having good product judgment’, or ‘product taste’. (see earlier (2021-03-21) Chin Product Development As Iterated Taste)

re explicit vs tacit pic.twitter.com/nhDjjpAMF5

— Bill Seitz of FluxGarden (@BillSeitz) February 21, 2022

I’ve had multiple casual conversations with entrepreneurs and product builders about Amazon’s intriguing PR/FAQ process, and what we might learn from it. I found myself saying that focusing on the trappings of the process alone isn’t going to do too much for you; you also have to acquire good product judgment, or run the process with individuals who possess good product judgment.

After graduation I was hired to turn a Vietnamese consulting company into a product business. Before we pivoted into Point of Sales systems, we built a mobile ad-network, three mobile games, and a CRM product for insurance agents, while running an app consulting business at the same time. I made sure to use the Lean Startup methodology for all of them. None of them succeeded.

If you zoom out a bit, I think we iterated through most of the failure modes possible when attempting to use the Lean Startup.

Of course, it’s not clear that these are the right lessons, since building products are a wicked learning domain, and it’s important to have a loose feedback loop when reflecting on past failures in such domains.

my boss called me on Skype and said “Hey, let’s do Point of Sales systems. I’ve bought the remains of a POS distributor

The rest, as they say, is history — within months we were making more than we had made in a full year of consulting work

All five products were more profitable than anything I’d ever built in the years before.

What changed? Was I a better product person?

What changed was the product domain. It was a simpler maze to navigate. As a business mentor put it, years later: “What you’re doing is easy, because you’re making Point of Sale systems for retail customers, who know exactly what they want. But if you try and build something completely new for them, you’ll be struggling with all the same problems, all over again.”

I’ve come to see customer obsession and taste as two parallel paths to product success. In some domains, you can get there through customer obsession alone — ala the Lean Startup. (customer-driven vs product-driven)

In other domains — when you’re building TikTok, say, or the first iPhone, the idea maze is a larger mess, so the process is more dependent on judgment and taste.

That taste, of course, is tacit in nature. But people don’t seem to want to talk about that. (tacit knowledge? mastery?)

The Four Product Methodologies

Once you have this lens of customer-obsession vs taste, the four product methodologies that I covered in my previous post begin to make more sense. You can’t copy what Apple does, or even what Amazon does with its PR/FAQ process, if you don’t already have people who have good product judgment.

To recap, those methodologies are:

• The Lean Startup
• Amazon’s PR/FAQ Process: You iterate on PR/FAQs, which are six-page documents that force team members to focus on the customer. Often, prototype iteration occurs concurrently with PR/FAQ iteration.
• Apple’s Creative Selection: You iterate on demos, and present these demos to a hierarchy of product leaders.
• Pixar’s Braintrust: Directors iterate on reels — that is, crudely drawn storyboards that are edited together with temporary voices and music — which are then presented to a core group of experienced storytellers.

When viewed through this lens, the majority of alternative product development methods seem to demand that you have taste in order to succeed. And so if you attempt to copy Amazon’s PR/FAQ process without having someone with product taste in the room, it’s likely that you would fail spectacularly — as Amazon did when they ran the process for the very first time.

In practice, many products require a mix of both taste and customer obsession.