FIGURE 6.1 Root Causes of Failed Product Efforts
It doesn’t matter how good your engineering team is if they are not given something worthwhile to build.
Generally have three stages of companies:
Started discussion on Top Reasons for Loss of Velocity
I believe this speaks more toward the lack of product strategy right - for all of cloud?
I would like to add to this list of top reasons for loss of velocity: From my perspective- another reason for the loss of velocity is the engagement with security. They are another partner here to the triad. So we do not have the ability to experiment as much because we are trying to build consensus across 4 pillars. If we have to embrace experimentation, we have to embrace risk.
Rally to objective process sounds similar to the what worked at interactive design and development agencies I worked at.
Pivoting to The Good Things we are doing to build a strong product culture…
FIGURE 6.1 Root Causes of Failed Product Efforts
The first truth is that at least half of our ideas are just not going to work.
The second inconvenient truth is that even with the ideas that do prove to have potential, it typically takes several iterations to get the implementation of this idea to the point where it delivers the necessary business value.
We call that time to money.
The biggest flaw of the old waterfall process has always been, and remains, that all the risk is at the end, which means that customer validation happens way too late.
Three overarching principles
In strong teams, product, design, and engineering work side by side
Conventional product roadmaps are all about output. Strong teams know it’s not only about implementing a solution. They must ensure that solution solves the underlying problem.
It’s about business results.
FIGURE 8.1 Continuous Discovery and Delivery
We are always working in parallel to both discover the necessary product to be built—which is primarily what the product manager and designer work on every day—while the engineers work to deliver production‐quality product.
The purpose of product discovery is to quickly separate the good ideas from the bad. The output of product discovery is a validated product backlog.
Will the user buy this (or choose to use it)? Can the user figure out how to use this? Can our engineers build this? Can our stakeholders support this?
Product discovery involves running a series of very quick experiments, and to do these experiments quickly and inexpensively,
The purpose of product delivery is to build and deliver these production‐quality technology products, something you can sell and run a business on.
Just because we’ve invested the time and effort to create a robust product does not mean anyone will want to buy it. So, in the product world, we strive to achieve product/market fit.
This refers to the longer‐term objective of this product, normally 2–10 years out. It is how we as a product organization intend to deliver on the company’s mission.
So, we use prototypes to conduct rapid experiments in product discovery, and then in delivery, we build and release products in hopes of achieving product/market fit, which is a key step on the way to delivering on the company’s product vision.
The MVP should be a prototype, not a product.
A product team is a group of people who bring together different specialized skills and responsibilities and feel real ownership for a product or at least a substantial piece of a larger product.
John Doerr, the famous Silicon Valley venture capitalist: “We need teams of missionaries, not teams of mercenaries.”
Mercenaries build whatever they’re told to build. Missionaries are true believers in the vision and are committed to solving problems for their customers.
They are empowered to figure out the best way to meet those objectives, and they are accountable for the results. This literally means product, design, and engineering working out solutions together.
Team Scope
It’s nearly impossible to have a team of missionaries when they’re pulled together for a project that lasts only a few months and is then disbanded.
Try to minimize dependencies between teams.
First, collaboration is built on relationships, and product teams—especially co‐located teams—are designed to nurture these relationships.
Second, to innovate you need expertise, and the durable nature of product teams lets people go deep enough to gain that expertise.
Third, instead of just building what others determine might be valuable, in the product team model the full team understands—needs to understand—the business objectives and context. And most important, the full team feels ownership and responsibility for the outcome.
If your company is not yet set up around dedicated product teams, this is probably the most important thing for you to fix. Everything else depends on this.
When a product succeeds, it’s because everyone on the team did what they needed to do. But when a product fails, it’s the product manager’s fault.
You need to become an acknowledged expert on the customer: their issues, pains, desires, how they think
A big part of knowing your customer is understanding what they’re doing with your product. Most product managers start their day with half an hour or so in the analytics tools, understanding what’s been happening in the past 24 hours. They’re looking at sales analytics and usage analytics. They’re looking at the results of A/B tests.
The third critical contribution—and the one that is often considered the most difficult by many product managers—is a deep understanding of your business and how it works, and the role your product plays in your business.
This means knowing who your various stakeholders are and especially learning the constraints they operate under.
Succeeding in the job of product means convincing each key stakeholder that you understand their constraints and that you are committed to only delivering solutions that you believe are consistent with those constraints.
The fourth critical contribution is deep knowledge of the market and industry in which you’re competing. This includes not only your competitors but also key trends in technology, customer behaviors and expectations, following the relevant industry analysts, and understanding the role of social media for your market and customers.
The successful product manager must be the very best versions of smart, creative, and persistent.
Start by becoming an expert in your users and customers. Share very openly what you learn, both the good and the bad. Become your team’s and your company’s go‐to person for understanding anything about your customer—quantitative and qualitative.
Work to establish a strong relationship with your key stakeholders and business partners. Convince them of two things: (1) You understand the constraints they operate under. (2) You will only bring to them solutions that you believe will work within those constraints.
Become an undisputed expert on your product and your industry. Again, share your knowledge openly and generously.
Finally, work very hard to build and nurture the strong collaborative relationship with your product team.
In product companies, it is critical that the product manager also be the product owner. If you split these roles into two people, some very common and predictable problems result—most commonly, the loss of your team’s ability to innovate and consistently create new value
There are two specific academic courses that every product manager should take:
You will need to understand how for‐profit companies work and the main business key performance indicators (KPIs) that are important to your business—
A good general marketing course will often cover these topics
Product managers who need to learn how to work effectively with designers.
Modern product designers are responsible for the following:
Modern product designers continuously collaborate with product managers and engineers—from discovery to delivery.
Rather than sitting with fellow designers, the modern product designer sits side by side with his or her product manager, a full partner with the product manager on product discovery.
Rather than being measured on the output of their design work, the product designer is measured on the success of the product.
User experience (UX) is much bigger than user interface (UI).
For modern products, this usually includes multiple different interfaces, as well as other customer touch points (e‐mail, marketing campaigns, sales process, customer support, and so forth).
With some products, UX also includes offline services, such as riding in a car summoned through Uber or staying in a house sourced through Airbnb.
The list of touch points could be very long, considering questions as: How will customers first learn about the product? How will we onboard a first‐time user and (perhaps gradually) reveal new functionality? How might users interact at different times during their day? What other things are competing for the user’s attention? How might things be different for a one‐month‐old customer versus a one‐year‐old customer? How will we motivate a user to a higher level of commitment to the product? How will we create moments of gratification? How will a user share his experience with others? How will customers receive an offline service? What is the perceived responsiveness of the product?
Good product designers use prototypes as their primary canvas for communicating ideas, both internally and externally.
Good product designers are constantly testing their ideas with real users and customers. They don’t just test when a prototype or idea is ready; they build testing into their weekly cadence, so they’re able to constantly validate and refine ideas as well as collect new insights
User testing is broader than usability testing. Product designers and their product teams utilize the opportunity to assess the value of their ideas.
Interaction design generally includes the underlying conceptual models
Task flows, and control layouts to manipulate those concepts.
Visual design includes composition, typography, and how the visual brand is expressed.
Incredibly common and serious problems:
Apple is one of the most valuable and design‐conscious companies on the planet; yet few tech companies understand the importance of design talent.
We need design—not just as a service to make our product beautiful—but to discover the right product.
Do whatever you need to do to have your designer sit next to you.
Include your designer from the very inception of every idea.
Include your designer in as many customer and user interactions as possible. Learn about the users and customers together.
Fight your temptation to provide your designer with your own design ideas. Give your designer as much room as possible
Encourage your designer to iterate early and often.
Not get all nitpicky about design details with the very early iterations.
Purpose of developing this programming literacy is not so you start telling your engineers how to do their job but, rather, to significantly improve your ability to engage with and collaborate
You must constantly demonstrate to your team that you’re open minded, you know how to listen, and you want and need their help in coming up with the right product.
While it’s good if you have a strong technology understanding, it’s not good if you use that knowledge to try to do their jobs for them.
Give your engineers as much latitude as possible in coming up with the best solution.
It is your job to make sure they feel like missionaries and not mercenaries. You do this by involving them deeply in the customer pain you are trying to solve and in the business problems you face.
Modern product marketing managers represent the market to the product team—the positioning, the messaging, and a winning go‐to‐market plan. They are deeply engaged with the sales channel and know their capabilities, limitations, and current competitive issues.
If your company has a sales organization, and you don’t have a product marketing partner, then this responsibility likely falls on you as product manager.
This can easily become a full‐time job. And given the cost of the sales organization, it’s really not an option to ignore them.
Make sure you have a product marketing manager to work with, and it’s absolutely worth your time to make sure you understand the market—
We are continuously doing two kinds of rapid learning and experimentation. One kind of learning is qualitative, and the other is quantitative.
User researchers are trained in this range of qualitative techniques
Data analysts help teams collect the right sort of analytics, manage data privacy constraints, analyze the data, plan live‐data tests, and understand and interpret the results.
Most companies have a blended approach in which the engineers write some of the automated tests (e.g., the unit level tests), and the test automation engineers write the higher‐level automated tests.
The level of test automation necessary to release with confidence is significant
The primary job of leadership in any technology organization is to recruit, to develop, and to retain strong talent. However, in a product company, the role goes beyond people development and into what we call holistic view of product.
The three distinct but critical elements to the holistic view of product are described next.
Either the leaders of the product management organization (VP product, directors of product), or a principal product manager.
This person should regularly review the work of the various product managers and product teams, identifying and helping to resolve conflicts.
People responsible for the holistic user experience.
Technology organization leader (often titled CTO or VP engineering).
Responsible for the holistic view of the system implementation. They should be reviewing the architecture and systems design of all the software—
The systems always seem to grow in complexity and size much faster than anyone can document, and with software, the definitive answer always lives in the source code itself (at least the current answer—not usually the rationale or the history).
These three holistic‐view leaders—the head of product, the head of design, and the head of technology—are obviously very valuable individually, but in combination you can see their real power. This is why my personal preference is to have these three people sit very close to one another,
VP product is your most senior product role in your company or business unit.
Strong in four key competencies:
Develop a strong team of product managers and designers.
Hire someone who has proven ability to develop others. They should have a track record of identifying and recruiting potential talent, and then working actively and continuously with those people to address their weaknesses and exploit their strengths.
Two very different types of product leaders needed for two very different situations: Where there is a CEO or a founder who is the clear product visionary Where there is no clear product visionary—usually in situations where the founder has moved on
VP product needs to complement the CEO. If you have a strong, visionary CEO, there may be some very strong VP product candidates that won’t want the position because they know that, in this company, their job is primarily to execute the vision of the CEO.
You need a product leader who knows how to get things done and has absolutely proved her ability to do so.
A strong product culture means that the team understands the importance of continuous and rapid testing and learning.
At a minimum, you are looking for someone with the combination of a strong technology background with an understanding of the economics and dynamics of your business and your market.
Your product leader must be able to work well on a personal level with the other key execs, especially the CEO and CTO.
This is the organization responsible for architecture, engineering, quality, site operations, site security, release management, and usually delivery management. This group is responsible for building and running the company’s products and services.
Titles vary but often include VP engineering, or chief technology officer (CTO).
The CIO role is very different from the CTO role. In fact, if your technology organization reports to the CIO, that is a warning flag for many of the pathologies discussed in Chapter 6,
Removing technology as a barrier, as well as broadening the art of the possible for business and product leaders, is the overarching objective.
Build an excellent organization with a strong management team committed to developing the skills of your employees.
Represent technology in the overall strategic direction and leadership of the company, working with other company executives to help inform direction, M&A activity, and build/buy/partner decisions.
Make sure this organization can rapidly, reliably, and repeatedly deliver quality product to market.
Make sure the company has an architecture capable of delivering the functionality, scalability, reliability, security and performance it needs to compete and thrive.
Make sure that members of the senior engineering staff are participating actively and contributing significantly throughout product discovery.
The CTO will serve as the company spokesperson for the engineering organization, demonstrating leadership in the community with developers, partners, and customers.
In growth‐stage and enterprise companies, many product managers complain that they have to spend far too much of their time doing project management activities.
Delivery managers are a special type of project manager whose mission is all about removing obstacles
These delivery managers are typically also the Scrum Masters
One of the most difficult issues facing every product organization at scale is just how to split up your product across your many product teams.
There are some critical core principles, and the key is to understand those principles and then weigh the options for your particular circumstances.
Phase out products that no longer carry their own weight, and we can often reduce the investments in our cash‐cow products so that we can invest more in future sources of revenue and growth.
Some people like the three horizons model, while others take more of a portfolio management approach.
While we can never entirely eliminate dependencies, we can work to reduce and minimize them.
A team should feel empowered, yet accountable for some significant part of the product offering.
We often find common needs and the increased importance of shared services.
Realize, however, that creating shared services also creates dependencies and can impinge on autonomy.
Many larger and older organizations no longer have a relevant vision and strategy, but this is key.
The minimum size for a product team is usually two engineers and a product manager,
For user‐facing technology, then a product designer
It’s really difficult for one product manager and product designer to keep more than about 10–12 engineers busy with good stuff
Start with the product vision, come up with an architectural approach to deliver on that vision, and then design the teams around that architecture.
While we’d love for every team to be a full stack team that can work on any layer of the architecture, in practice that’s often not an option.
Be sure to staff these common services teams with strong and highly technical product managers (often called platform product managers).
If, for example, your company provides a two‐sided marketplace with buyers on one side and sellers on the other, there are real advantages to having some teams focus on buyers and others focus on sellers.
While there are advantages to aligning with business units, this usually comes after the other factors in priority.
The organization’s needs should and will change over time.
One of the key themes of this book is focusing on outcome and not output.
The first inconvenient truth is that at least half of our product ideas are just not going to work.
The second inconvenient truth is that, even with the ideas that do prove to be valuable, usable, feasible, and viable, it typically takes several iterations to get the execution of this idea to the point where it delivers the expected business value
This is often referred to as time to money.
Weak teams just plod through the roadmap they’ve been assigned,
Strong product teams understand these truths and embrace them rather than deny them. They are very good at quickly tackling the risks (no matter where that idea originated) and are fast at iterating to an effective solution.
Anytime you put a list of ideas on a document entitled “roadmap,” no matter how many disclaimers you put on it, people across the company will interpret the items as a commitment.
Sometimes, we do need to commit to a delivery on a date. We try to minimize those cases, but there are always some. But we need to make what is called a high‐integrity commitment.
In the empowered product team model this book is predicated on, the teams are themselves equipped to figure out the best ways to solve the particular business problems assigned to them.
There are two main components that provide this business context:
Prioritizing business results, rather than product ideas.
The way we manage commitments is with a little bit of give and take.
We ask the executives and our other stakeholders to give us a little time in product discovery to investigate the necessary solution.
Once we have come up with a solution that works for our business, we now can make an informed and high‐integrity commitment
It’s mainly a persuasive piece that might be in the form of a storyboard, a narrative such as a white paper, or a special type of prototype referred to as a visiontype.
Its primary purpose is to communicate this vision and inspire the teams
The product strategy is our sequence of products or releases we plan to deliver on the path to realizing the product vision.
For most types of businesses, I encourage teams to construct product strategy around a series of product/market fits.
For business‐focused companies, you might have each product/market fit focus on a different vertical market (e.g., financial services, manufacturing, automotive).
For consumer‐focused companies, we often structure each product/market fit around a different customer or user persona. For example, an education‐related service might have a strategy that targets high school students first, college students next,
There’s no single approach to product strategy that is ideal for everyone,
The difference between vision and strategy is analogous to the difference between good leadership and good management. Leadership inspires and sets the direction, and management helps get us there.
There is no one right way to do this, but there are three critical inputs to your decision:
The first is market sizing, usually referred to as total addressable market (TAM). All things considered equal, we like big markets rather than small markets.
The second factor concerns distribution, usually referred to as go to market (GTM). Different markets may require different sales channels and go‐to‐market strategies.
The third factor is a (very rough) estimation of how long it will take, referred to as time to market (TTM).
10 key principles for coming up with an effective product vision.
Evangelize continuously and relentlessly. There is no such thing as over‐communicating when it comes to explaining and selling the vision.
Good strategies have these five principles in common:
Where the product vision describes the future you want to create, and the product strategy describes your path to achieving that vision, the product principles speak to the nature of the products you want to create.
As an example, early on at eBay we found we needed a product principle that spoke to the relationship between buyers and sellers. Most of the revenue came from sellers, so we had a strong incentive to find ways to please sellers, but we soon realized that the real reason sellers loved us was because we provided them with buyers. This realization led to a critical principle that stated, “In cases where the needs of the buyers and the sellers conflict, we will prioritize the needs of the buyer, because that’s actually the most important thing we can do for sellers.”
George Patton quote I mentioned earlier: “Never tell people how to do things. Tell them what to do, and they will surprise you with their ingenuity.”
Objectives should be qualitative; key results need to be quantitative/measurable.
Key results should be a measure of business results, not output or tasks.
The rest of the company will use OKRs a bit differently, but for the product management, design, and technology organization, focus on the organization’s objectives and the objectives for each product team,
Find a good cadence for your organization (typically, annually for an organization’s objectives and quarterly for a team’s objectives).
Keep the number of objectives and key results for the organization and for each team small (one to three objectives, with one to three key results each is typical).
It’s critical that every product team track their active progress against their objectives (which is typically weekly).
The objectives do not need to cover every little thing the team does, but they should cover what the team needs to accomplish.
It’s important that, one way or another, teams feel accountable to achieving their objectives. If they fail substantially, it’s worth having a post‐mortem/retrospective
Agree as an organization on how you will be evaluating or scoring your key results.
What’s important here is consistency across the organization,
It’s common to define a score of 0 (on a scale from 0 to 1.0) if you essentially make no progress, 0.3 if you just did the bare minimum—what you know you can achieve, 0.7 if you’ve accomplished more than the minimum and have really done what you’d hoped you would achieve, and 1.0 if you’ve really surprised yourselves and others with a truly exceptional result,
Establish very clear and consistent ways to indicate when a key result is in reality a high‐integrity commitment
For most key results, you may be shooting for that 0.7 score. But for a high‐integrity commitment, these are special, and it’s more binary.
Be very transparent (across the product and technology organization) on what objectives each product team is working on and their current progress.
Senior management (CEO and executive team) is responsible for the organization’s objectives and key results.
Imagine if the engineers were told to spend their time on re‐platforming, the designers on moving to a responsive design, and QA on retooling. While each of these may be worthy activities, the chances of solving the business problems that the cross‐functional teams were created to solve are not high.
If you deploy OKRs for your product organization, the key is to focus your OKRs at the product team level. This means don’t let functional team or individual person OKRs confuse the issue.
The key is that the cascading of OKRs in a product organization needs to be up from the cross‐functional product teams to the company or business‐unit level.
Focused here on growth stage or enterprise organizations.
With larger organizations, product teams need more help. The first help they need is a very clear understanding of the organization‐level objectives.
Leadership (especially the head of product, head of technology, and head of design) will need to discuss the company objectives and which teams are best suited to pursue each objective.
Moreover, at scale, it is very common to have some significant number of product teams that are there in support of the other product teams. These are often called platform product teams, or shared services product teams.
Leadership will need to help coordinate the objectives for these teams and make sure we coordinate the dependencies and align the interests.
Once you have your objectives, there is a very critical reconciliation process in which the leadership team looks at the proposed key results from the product teams and identifies gaps
Lean on management to help connect the dots between teams.
Delivery managers play a key role in tracking and managing these dependencies and our commitments.
In many enterprise scale organizations, there are essentially multiple business units, and in this case, we would expect that there are corporate level OKRs,
When using OKRs at scale, there’s a larger burden on leadership and management to ensure that the organization is truly aligned,
Product evangelism is, as Guy Kawasaki put it years ago, “selling the dream.”
Top‐10 pieces of advice for product managers to sell the dream:
Spend time with your team. If you’re not spending significant face time with your designer and every engineer on your team, then they can’t see the enthusiasm in your eyes.
Two very significant challenges to tackle.
We need to learn fast, yet also release with confidence.
The purpose of product discovery is to address these critical risks: Will the customer buy this, or choose to use it? (Value risk) Can the user figure out how to use it? (Usability risk) Can we build it? (Feasibility risk) Does this solution work for our business? (Business viability risk)
Set of core principles that drive how we work.
Marc Andreessen, “The most important thing is to know what you can’t know,”
We must validate our ideas on real users and customers.
One of the most common traps in product is to believe that we can anticipate our customer’s actual response
Our goal in discovery is to validate our ideas the fastest, cheapest way possible.
We need to validate the feasibility of our ideas during discovery, not after.
We need to validate the business viability of our ideas during discovery, not after.
It’s about shared learning. One of the keys to having a team of missionaries
We need to tease out the risks and determine where it makes sense to focus our time.
Two goals
An opportunity assessment is designed for the vast majority of product work, which ranges from a simple optimization to a feature to a medium‐sized project.
A customer letter is designed for larger projects or initiatives that often have multiple goals and a more complicated desired outcome.
A startup canvas for those times you’re creating an entirely new product line or a new business.
Answer four key questions about the discovery work you are about to undertake:
For smaller and more typically sized efforts, the opportunity assessment is usually sufficient.
But when embarking on a somewhat larger effort, there may in fact be multiple reasons, several customer problems to be solved, or business objectives to be tackled. To communicate the value effectively, it may take more than the four questions listed in the previous chapter.
How Amazon builds product, and one of them is referred to as the working backward process, where you start the effort with a pretend press release.
The idea is that the product manager frames the work ahead of the team by writing an imagined press release of what it would be like once this product launches.
The actual reader of this press release is the product team, related or impacted teams, and leadership.
Variation of this technique that was developed and refined at Nordstrom. The idea is that rather than communicate the benefits in a press release format, you describe them in the format of a customer letter written from the hypothetical perspective of one of your product’s well‐defined user or customer personas.
The letter—sent to the CEO from a very happy and impressed customer—explains why he or she is so happy and grateful for the new product or redesign. The customer describes how it has changed or improved his or her life. The letter also includes an imagined congratulatory response from the CEO to the product team explaining how this has helped the business.
An Introduction to Lean Canvas
This is an early stage startup, where you are trying to figure out a new product that can power a new business, or, for those that work at an enterprise size company, when you’re asked to tackle an all‐new business opportunity for the company.
In other words, you’re not being asked to improve an existing product, you’re being asked to invent an entirely new product.
These are two‐dimensional maps, in which major user activities are arrayed along the horizontal dimension, loosely ordered by time from left to right.
Along the vertical dimension, we have a progressive level of detail. As we flesh out each major activity into sets of user tasks, we add stories for each of those tasks. The critical tasks are higher vertically than the optional tasks.
We can use this story map to frame our prototypes, and then as we get feedback on our prototypes and learn how people interact with our product ideas, we can easily update the story map to serve as a living reflection
Without strong products, our marketing programs require customer acquisition costs that are too high; our sales organization is forced to get “creative,” which drives up cost of sales, lengthens the sales cycle, and puts downward pressure on price; and our customer success organization is forced to take it on the chin every day with frustrated customers.
This is a real customer (not friends or family), who is running your product in production (not a trial or prototype), who has paid real money for the product (it wasn’t given away to entice them to use it), and, most important, who is willing to tell others how much they love your product (voluntarily and sincerely).
Consider it the single best leading indicator of future product success.
We are looking to develop six reference customers in our specific target market or segment, so, the idea is to find six similar customers. If you end up targeting two or three customers from two or three different markets, this program will not give you the focus you want and need.
We need them to have people and time willing to work closely with us. They need to be willing to spend time with the product team, testing out early prototypes and helping the team ensure the product works well for them. If possible, we would like them to be well‐recognized marquee names, because that will be of the most value to the sales and marketing staff.
For developer products, the program is very much like the one for businesses, but the main difference is that we work with the development teams (engineers and product managers) that will use our APIs to get them successfully using our product.
The result of the program is a set of reference applications rather than reference customers.
For customer‐enabling tools, such as a new dashboard for your customer service agents, we pick six to eight well‐respected, influential internal users/employees—
Rather than focusing on six businesses to work closely with (where we have access to many different users at each customer), we instead focus on a somewhat larger number of consumers (on the order of 10–50) that we engage with to get them to the point that they are loving our product.
Here’s what I’m always trying to understand:
Here are some tips for getting the most out of these learning opportunities:
The idea is that we do the customer’s job for them—manually and personally. Just as if you went to a hotel concierge
With this technique, you become the concierge. You do what the user or customer needs done for them. You may have to ask them to train you first, but you are in their shoes doing the tasks they would do.
A concierge test requires going out to the actual users and customers and asking them to show you how they work so that you can learn how to do their job,
Historically, the two main approaches used by good teams to come up with product opportunities have been:
Third alternative is to allow, and even encourage, our customers to use our products to solve problems other than what we planned for and officially support.
Some product people can get upset when they find customers using their products for unintended use cases. This concern is usually tied to the support obligations. I’m suggesting, however, that this special case can be very strategic and well worth the investment to support.
I have been a long‐time fan of public APIs as a part of a company’s product strategy. I consider developers to be one of the consistently best sources of truly innovative product
In an undirected hack day, people can explore whatever product ideas they like, so long as it’s at least loosely related to the mission of the company.
In a directed hack day, there’s a customer problem (for example, something is really difficult to learn and use, or it takes too long to do) or business objective we’ve been assigned (for example, “Reduce the customer churn rate” or “Increase customer lifetime value”),
There are two major benefits to these directed hack days.
Five key principles behind their use.
There are several situations wherein your engineers may identify a significant feasibility risk involved in solving a particular problem
Common examples include:
A feasibility prototype is a long way from a commercially shippable product—the idea is to write just enough code to mitigate the feasibility risk.
Most of the time the feasibility prototype is intended to be throwaway code—
Simulation. Smoke and mirrors. It’s all a façade. There is nothing behind the curtain.
A low‐fidelity user prototype doesn’t look real—it is essentially an interactive wireframe.
High‐fidelity user prototype is still a simulation; however, now it looks and feels very real. In fact, with many good high‐fidelity user prototypes, you need to look close to see that it’s not real.
The big limitation of a user prototype is that it’s not good for proving anything—like whether or not your product will sell.
Some of my favorite examples of this are when applying game dynamics, search result relevance, many social features, and product funnel work.
The live‐data prototype is substantially smaller than the eventual product, and the bar is dramatically lower in terms of quality, performance, and functionality. It needs to run well enough to collect data for some very specific use cases, and that’s about it.
There are two big limitations
One of my favorite examples of a hybrid prototype—and an exceptionally powerful tool for learning quickly in product discovery—is today often referred to as a Wizard of Oz prototype. A Wizard of Oz prototype combines the front‐end user experience of a high‐fidelity user prototype but with an actual person behind the scenes performing manually what would ultimately be handled by automation.
Four types of questions we’re trying to answer during discovery:
We do usability testing in discovery—using prototypes, before we build the product—and not at the end,
Usability testing with a high‐fidelity user prototype.
Good product managers know they will get the product wrong initially and that nobody gets it right the first time.
We want to learn whether the user or customer really has the problems we think they have, and how they solve those problems today, and what it would take for them to switch.
Keep your users in use mode and out of critique mode. What matters is whether users can easily do the tasks they need to do.
Three important cases you’re looking for:
Avoid giving any help or leading the witness in any way.
After each test subject, or after each set of tests, someone—usually either the product manager or the designer—writes up a short summary e‐mail of key learnings and sends it out to the product team. But forget big reports that take a long time to write, are seldom read, and are obsolete by the time they’re delivered
Just because someone can use our product doesn’t mean they will choose to use our product.
The customer must perceive your product to be substantially better to motivate them to buy your product and then wade through the pain and obstacles of migrating from their old solution.
One of the biggest possible wastes of time and effort, and the reason for countless failed startups, is when a team designs and builds a product
When they finally release the product, they find that people won’t buy it.
When the user clicks that button, rather than taking the user to the new feature, it instead takes the user to a special page that explains that you are studying the possibility of adding this new feature, and you are seeking customers to talk to
User sees a page that explains that you are studying the possibility of adding this new offering, and you’d like to talk with them
Protect your revenue and brand, and protect your employees and customers.
Quantitative testing tells us what’s happening (or not), but it can’t tell us why,
I argue that qualitative testing of your product ideas with real users and customers is probably the single most important discovery activity for you and your product team.
Generally begin the user test with a short user interview where we try to make sure our user has the problems we think she has, how she solves these problems today, and what it would take for her to switch
Value test is always preceded by a usability test.
Test to see whether the user can figure out how to operate our product.
The magic happens when one of your engineers is right there watching the qualitative testing with you.
Can ask people if they will sign a “non‐binding letter of intent to buy” which is a good indicator that people are serious.
Qualitative testing is all about fast learning and big insights, quantitative techniques are all about collecting evidence.
Collect enough data that we have statistically significant results
The gold standard for this type of testing is an A/B test.
It’s also important for tech product managers to have a broad understanding of the types of analytics that are important to your product.
I still encounter too many product teams that either aren’t instrumenting their product to collect analytics, or they do it at such a minor level that they don’t know if and how their product is being used.
Answer several related questions:
If you put an engineer on the spot, without time to investigate and consider, you are very likely to get a conservative answer, partly designed to make you go away.
The solution must also work for your business.
If what you are proposing to build could impact the sales channel, the major marketing programs, or is potentially outside of the brand promise (the range of things your customers expect from your company), then you’ll want to discuss this with marketing and show them prototypes of what you are proposing before you consider building anything.
A direct sales channel is very expensive, and this requires a high‐value product and price point.
If what you are proposing would represent a departure from what the sales channel has proven their ability to sell, sit down with the sales leadership and show them what you are proposing before you build anything,
You need to understand what your company’s customer success strategy is, and you need to ensure that your products are aligned with that strategy.
If there are cost issues involved, sitting down with someone in finance and modeling the costs will be critical
Privacy concerns, compliance concerns, intellectual property, and competitive issues are all common constraints related to legal.
Most businesses have some number of close business relationships with partners of various types, usually with a contract
Sometimes these agreements can cripple your company’s ability to compete. Sometimes they are a huge win.
If you are proposing anything at all remotely related to security, you will probably want to bring your tech lead and sit down with the security leadership early
With every company there is some CEO or general manager that is responsible for the business unit.
A user test is when we test our product ideas on real users and customers.
A product demo is when you sell your product to prospective users and customers, or evangelize your product
A walkthrough is when you show your prototype to a stakeholder and you want to make sure they see and note absolutely everything that might be a concern.
Many inexperienced product managers do a walkthrough with a prospective customer when they should have prepared a product demo.
Be sure to be clear about whether you’re doing a user test, a product demo, or a walkthrough.
In 1999, a then very young Netflix
There wasn’t much of a reason to rent DVDs via the U.S. Postal Service when you could just stop by the local Blockbuster store on the way home from work.
One of many tests they tried was to move to a subscription service.
The good news was that, yes, this approach really did appeal to people.
The bad news is that the team created some real problems
Netflix customers wanted to rent mostly newly released feature films; yet, these were much more expensive
Needed to somehow get customers to want and ask for a blend of expensive and less expensive titles.
This is where Netflix’s queue, ratings system, and recommendation engine all came from.
Between working with the co‐founders on the strategy, validating concepts with the users, assessing the analytics, driving features and functionality with the team—and working with finance on the new business model, marketing on acquisition, and the warehouse on fulfillment—you can imagine the workload
The team got the new service up and running and used this to power and grow their business for another seven years, until they disrupted themselves again by moving aggressively to the streaming model.
When they were struggling for cash early on, they offered to sell themselves to Blockbuster for $50 million, and Blockbuster turned them down.
It’s hard because the changes are so often cultural.
A discovery sprint is a one‐week time box of product discovery work, designed to tackle a substantial problem or risk your product team is facing.
Always end your week by validating your potential solution with real users and customers.
The GV team decided to share their learnings in a book. The book is titled, Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days, by Jake Knapp, John Zeratsky, and Braden Kowitz.
Pilot teams allow the roll out of change to a limited part of the organization before implementing it more broadly.
Some people in your organization love change, some want to see someone else use it successfully first, some need more time to digest changes, and a few hate change and will only change if they’re forced to do it.
The goal is that over time, the organization moves its focus from specific features launching on specific dates to business results.
If the feature you’re working on is to add PayPal as a payment method, and the reason is to increase conversion, then be sure to always show the current conversion rate and the result you’re hoping to achieve.
If the impact was good, celebrate it. If the impact was not what was hoped, then emphasize to everyone that, while you did ship the feature, the result was not successful.
Two big reasons why stakeholders especially are so attracted to roadmaps:
Teams work on the prioritized business objectives determined by the leaders; we share our key results transparently; and we commit to high‐integrity commitments when critical delivery dates are needed.
It is all too easy to institute processes that govern how you produce products that can bring innovation to a grinding halt.
Agile methods are generally very conducive to consistent innovation.
Yet there are several process consultancies that specialize in “Agile at Scale,” which introduce methods and structures intended to scale to large numbers of engineers, yet which absolutely destroy any hope of innovation.
One practical test of whether a person is considered a stakeholder is whether or not they have veto power, or can otherwise prevent your work from launching.
This group of people typically includes: The executive team (CEO and leaders of marketing, sales, and technology) Business partners (to make sure the product and the business are aligned) Finance (to make sure the product fits within the financial parameters and model of the company) Legal (to make sure that what you propose is defensible) Compliance (to make sure what you propose complies with any relevant standards or policies) Business development (to make sure what you propose does not violate any existing contracts or relationships)
The product manager has the responsibility to understand the considerations and constraints of the various stakeholders, and to bring this knowledge into the product team.
The main way we demonstrate this knowledge to the organization is by sharing very openly what we learn.
The key technique is to spend one‐on‐one time with the key stakeholders. Sit down with them and listen. Explain that the better you understand their constraints, the better your solutions will be. Ask lots of questions. Be open and transparent.
Commit to previewing your solutions during discovery with the key stakeholders before you put this work on the product backlog.
Move the discussion from opinions to data. Share what you’re learning very openly.
Presentations are notoriously terrible for testing business viability. The reason is that they are far too ambiguous.
In contrast, high‐fidelity user prototypes are ideal
A group setting is not the forum for designing strong products. It results in design by committee, which yields mediocre results at best.
Sharing what we learn in a startup happens naturally because the product team and the company are pretty much the same thing. However, as companies scale, this becomes substantially more difficult;
A technique I love for helping with this is for the head of product, at a company all‐hands or similar meeting every week or two, to take 15 to 30 minutes to highlight what has been learned
Goot teams
Organizations that lose the ability to innovate at scale are inevitably missing one or more of the following attributes:
I think of product culture along two dimensions.
What does it really mean to have a strong innovation culture?
What does it really mean to have a strong execution culture?
