Transcript for:
Design Thinking and Ideation Workshop Insights

Hello to everyone. Welcome to the second workshop of part of MADAREC ideation program. Today we will be on ideation and design thinking workshop with Mr. Kevin who is a product design and research director in Barcelona. He has over 17 years of experience in the field of design. years of experience in user experience, product design and data visualization. He was the director and the head of design and research at Amantize. Also, Mr. Kevin has many high-performing design teams, launched innovative product features and implemented user-centric teams. user-centric design strategy. This is Mr. Kevin, who will be today our international expert to present or to introduce ideation and design thinking workshop. This is a handover, the mic with Mr. Kevin. We can start now. Okay, thank you. i think you can share your screen can you all see my screen yeah that's fine okay okay cool So let's begin. So yeah, so I am going to do a quick introduction about the topic, and then we'll dive into some examples and techniques, and then we can go into questions. A bit about myself. I am from the US, from Washington, DC, the capital. I've lived in Amsterdam and Sydney, Australia, and I now live in Barcelona, Spain. A lot of my work has taken me around the world, both for doing research and working for different companies in different regions. Started out as a web developer, web designer, built a lot of products myself, and then built a lot of teams that built really big products, sometimes for consulting firms like PwC and EY, other times directly for one client or for a company like Hilton, Gap, Booking.com, or Uber. So... just so that you know like i have both experience and kind of like public sector but also in the private sector so i can answer questions on their sides um so the last team that i managed uh was this team and we launched products in 30 different countries so just like a very large scope of work and so hopefully i can bring some of that experience to the table today For the learning goals and for the agenda for the session, we're going to go over what is design thinking. I'm going to really focus on the ideation stage, so where you are in your business right now, the idea you want to develop into a company, the customers you want to attract, the price you want to charge, and how do you kind of make smart decisions about each of those using the data of competitors, of the market, of what people are willing to spend. There's also a process we will go through to kind of hone our ideas from a simple idea to a validated concept to an MVP, like a version one, version two. We'll also talk about how to target your first customers, which we're going to call personas. These are just types of customers that you want to target. And then we're going to talk about how to remove yourself from the process to avoid bias and to actually involve real users, real customers to help you make the best decisions. I kind of call the first section designing business. So we're not talking about the product or the service yet. We're talking about the way the business functions, the way your startup, your company actually operates. So as an early stage idea stage startup, this is a company that's going to build a product or service. And so this is before you have an idea of the right product for the market. This is before you have that version one of the product or that MVP. And so a lot of what you're doing right now is trying to get confirmation that what you want to sell will match the demand in the market. So you're both doing research on the idea or the problem you have, but you also need to do lots of research and validation on whether people want that product and if they want that product or service in the way and at the price you're trying to sell it. So it's more of a combination or a package, not simply is it a good idea, yes or no. It's about the idea, the presentation, the price point, and the customer combination all working out. Now, there's multiple stages for a company. So when you're at the ideation stage, you're really talking about market needs. You're developing maybe like a unique product based on what else is available in the market. But then after you finish this stage. is when you start testing the business idea. Maybe you change the way you sell it. Maybe it's a one-off, maybe it's a subscription, maybe it's consulting. So the different models for the business can also still change after ideation. And then at the launch stage is when you've kind of finalized the business model, the product offering, the price point, the target customer, and now you're trying to, for the first time, actually sell that product to that target customer. And so this is how you establish your first market presence. This is when you go public, this is when maybe you do a soft launch, alpha, beta, and this is when you try to get real customers. This is leaving the friends and family and this is really going out to people who don't know you. At the growth stage, this is when you take a company that has real customers, recurring business, retained customers who stay within the business and buy, and let's say they are happy, loyal customers, and you try to increase that. You try to increase that as much as possible without essentially spending all your money on marketing. So this is effective or let's say efficient growth. And then the maturity stage is when you are going to, let's say, add a new business unit or a new product that has an extension or a complete complement or a new business attached to the core business. So we'll focus today on the ideation stage. Favorite two examples when it comes to ideation are Jeff Bezos and Steve Jobs. Both of them have had some really great books written about the way they run companies from the ideation stage to where they are now. The first book is called Working Backwards. It's from Colin Breyer and Bill Carr. These are the two longest executives at Amazon. And so they've been there for over 20 years. And they talk about how multiple ideas from the company went from ideation all the way to where they are today. And then Steve Jobs had a book written about him, a memoir by Walter Isaacson. He actually asked for this book to be written. This is the only kind of sanctioned memoir of his life. And Walter Isaacson is a famous reporter and journalist from CNN. Both of them highlight the same principle, which is you focus so much on the customer by starting with their problems, their needs, their complaints, their frustrations. And then the company and the business ideas and the features and the products will come to you. So if you start with the customer and work backwards, the business is organically already targeting a specific customer's needs. When we kind of translate that into a framework or a model, it's called the customer centricity model. So in the middle or in the center, you have the customer. What does that mean? Let's say one circle is feedback. Feedback is how you continuously understand the quality of your work, not just the quantity. But this also matters from the customer point of view. This matters from who leads the company. And this matters from who does customer service and who actually sells the product. Then you have the data, let's say, from the sales team. You also have the data from people using the product. And then you have the data on the business health. And then last but not least, you have to actually know who is coming to the product, who is signing up for sales calls, who's visiting the website, who's following on social media, who's your first customer, who are they referring, who's leaving, who's staying. And so really you want to get as much data and as much detail about each customer as possible. And this is where you get to understanding the customer. So we're going to break this down into each piece. So you see three circles here. So there are about three sections in this session that we'll go through step by step. As you go through custom centricity, you will start to develop more mature ways to execute each of these steps. Let's say at level one, like a very basic company, almost everyone does this. You're focused on delivering a good product, right? And so maybe you know the functionality you want or the features you want. You know your budget, you know how much time you have. But this isn't about the customer. This is about you, right? You're thinking about the features you want to build. And if you build them, you know, or you hope the customers will come. This is still very founder centric, owner centric. We're doing things that we want to do and we hope the customers also agree with us. Then you take your ideas, your model, your designs, your functionality, and you get feedback. So only at the very end of the process in level one are you getting customer feedback. So they're not at the center yet. Then you have level two. Level two is when you are actually deciding the functionality of the product, like how it works, what it does, you are letting the technology lead. So you say what is feasible, what is capable, what can it do, what can't it do. Maybe you get customer feedback on what's the most important thing to build first from the tech side. Maybe you do some early surveys on, you know, which three features must it have. So maybe you have like a must have list and a want to have list. And then this kind of customer feedback is maybe involved occasionally during your process, but not every single day. Level three. is when you really increase how much feedback customers have as you make very large strategic executive decisions. So let's say you shift from selling a product that you designed to you sell a solution to a problem. This is the difference between telling me the features and the price and your packages and your bundles and instead you tell me the outcomes I will have. Use our product because you will no longer have xyz problem. Use our product because X, Y, Z will be better. Or this problem that you have is going to go away or happen less frequently. So you switch the language of the business, switch the language of your marketing, of your sales, of your advertising to outcomes or results of the customer's life, not of your life. So this is less sales and more storytelling. So then you start to listen to your customers. You have to use the same language they use. So when you do a survey or you call them to sell and they say yes or they say no, which language are they repeating back to you? So you're using these drivers of choice, these things that they relate to, that they agree with, that they say to their friends, that they put in their referrals, app store reviews, any public or verbal validation of what they care about or liked from your pitch, from your sales demo. You use that in all your materials. And then you also have to use, let's say, any chance you get. to involve a customer. Let's say you do version 1.2, version 1.3. Between versions, you're asking for feedback. You're asking for a beta group or a focus group or a volunteer group to see the first version. So you always get feedback. You're always looking for someone or a group of people to represent the public. And so you really encourage people to email you questions. Oftentimes you see founders of certain companies like Notion. give out their email publicly and people just email them. Like people on, people email Elon Musk sometimes for Twitter and like he encourages public customer feedback. And so this is that kind of level three maturity. Level four would be kind of putting it in every single department in the business. You almost only develop and plan version two, three, and four of any product based on the largest complaints and the largest feedback from customers. So this is the opposite of treating your product managers as mini CEOs. You're actually treating everyone like customer service. You want everyone exposed to customer feedback, customer complaints, tickets, any pain points. You want to treat your largest customers, your most loyal customers, almost like business partners or consultants. And you have them directly involved and giving feedback on strategy, on roadmaps, things like this. And then you are agnostic or maybe you are non-biased about who decides what the next features are. It could be an intern. It could be an engineer. It could be customer service. It could be a salesperson. Anyone who has access and exposure to a customer essentially can decide what the business does next because you want as much feedback from customers as possible driving decisions. And this is level four. And this is where you would hope the biggest companies. let's say, hope to get to. It's very hard to stay level four. So you end up seeing companies kind of teeter back and forth between three and four. The bigger you are, the harder four gets to be. So the Amazon working back background method, if you boil it down into five simple steps, is you kind of think about what you hope customers will say, or what problems you hope customers will be happy to share with other people. are being resolved. So the way they describe it is as a press release. Imagine a reporter wrote an article about your product or a feature. What would the headline be? And what would be the image? And what would be the top three sentences that they would say? Then you look at the opportunity. So let's say you have a business proposition. You have a problem you think customers have. You want to build a product. You need to figure out how many customers have this problem. How many customers in your region, in your language, in your market, in your price point have this problem? And is it enough people? Is it enough people who have the budget, who have the problem? who speak the same language, who are in the same market as you, to actually be a good first round test, a good first customer base for the product. This is just gathering data. You're not yet judging or eliminating ideas. You're just gathering for each idea you have, how big of an opportunity is each idea. Then for step three. You're looking at possible solutions. This is more brainstorming. You're not investigating deeply. You're not engineering anything yet. You're just kind of, you want to involve your customers or your stakeholders, your shareholders, your partners, anyone who's an early adopter, maybe someone who wants to be like one of the first customers. You kind of treat them like a focus group and you just run some of these ideas by them and you see if they agree with the way you've done opportunity sizing and the ranking and prioritization of your ideas. Fourth would be you start to narrow down maybe the top three ideas you have and think about what the next six to 12 months would look like to actually execute that idea. So this is like a very high level roadmap. Maybe these are OKRs, maybe these are like quarterly milestones, themes you would need to hit, things you need to build, things that have to exist, either departments or functionalities or capabilities. And then once you have like one idea that you actually are executing. Then you need the full backlog of which engineers are going to work on what, do you need designers, do you need marketing, do you need sales, do you need enterprise sales. So you start to design the business and the roles and specific tasks that have to be assigned to each role to execute the grand vision. And then this is when you actually start kicking off the work. So this is the way their model works. For the model to work, there's a really basic requirement. they consider core is you have to really be testing usability at each of these stages. You're not testing whether people like your idea or not. You're testing whether they would use it, right? It isn't enough for them to say, cool. You want them to say, cool, I want it. Cool, how much is it? I would pay that price. It's a very different thing to like an idea than to buy a product and use it every day. So you're looking for a certain customer, a specific user in a certain context. So I have this problem on this recurring basis, which means I'm willing to solve this problem immediately or urgently because the problem is big enough for me. And when that problem is big enough, I turn to this product because it is effective, efficient, and I'm happy with the way it works. So that is how usability is defined. And that is what we consider a user. So People can have the same problem, but if your problem happens for you every two years, and it happens for me every day, we are different users. So it's really important to define that cadence and that size of the problem. Right, let's turn to how we turn this into an experiment. So you're going to start to think, okay, I need to build some of these things, but this is the most tricky and dangerous part. of any new business, any new functionality, is you want to discover and learn as much as possible without the most expensive part of the business, which is building, right? So you want to avoid hiring engineers, building things, hiring a sales team, paying out commissions and paying out salaries. You want to figure out as much as possible before you have to distribute labor and scale costs. And so this is going to be a series of experiments you can run from the idea stage all the way through scale. You can kind of treat each idea you have kind of like animals in the wild, like the most adaptable, fastest, smartest animals will win. This is how you have to treat your ideas. Sometimes your really good ideas just aren't fast enough. Sometimes some of the really good ideas just aren't tall enough, you know? So you really have to be very non-biased and non-emotional when you are kind of seeing which tests win. steve you have a question from mr malik yes mr kevin can you hear me clearly yes let me just open my video so more engaging So I just have a question on usability. It seems that there are multiple parameters, you know, to measure this usability. So are there any specific tools that we can use? Tools are actually quite flexible when it comes to usability. It's really about answering a question. So I'll give you three examples to talk about effectiveness, efficiency, and satisfaction. Effectiveness is kind of binary. So either it did or it did not help. So this could be a yes or no question. This is something you see oftentimes on online surveys. Would you recommend to a friend? Yes or no. Would you use this again? Yes or no. Was the experience good? Thumbs up, thumbs down, right? This is about effectiveness. Efficiency would be about the quality of the solution. So do you know of better solutions? Do you know of worse solutions? Would you have used this solution in the future? Or would you use someone else? And why, right? So you're looking for differentiation, gaps in quality. This is where you get kind of small improvements to the way things are done. Not if they are done, yes or no, but the format, the quality, the price, things like this. And then satisfaction is an overall review of happiness. And this is kind of like a personal benchmark for yourself. You can build the same products three different ways and have... the same effectiveness, same efficiency, same costs, but have different happiness, right? So the customers will never be able to indicate that happiness to you in a rating or in a payment. They will only be able to tell you this if you ask. And so this is a very classic issue. For example, Netflix had for a long time, they were growing, people were paying the monthly price, and then they lost some licenses for some movie content, and they lost customers. The price point didn't change, the efficiency, the effectiveness of the platform didn't change. What changed was how happy the customers were with the content. So you really have to specifically ask for happiness because it's very hard to track that any other way. I understand. So this is, does measuring usability comes after the MVP or can we do it before? Honestly, it could be way before. So let's say you have the idea and your idea includes that you want to sell something as a monthly payment. Maybe it includes a year long contract requirement. Maybe it includes an onboarding service or fee. Any of the things that your business requires to access the service or product can be tested and people can say, hey, I think that's ineffective. I think that's an inefficient use of my time to go through a sales manager. or I think I wouldn't be happy using this process because it's too labor intensive, too time intensive for my business. And so you can use these three questions as early as the idea stage. It'll just be, of course, depending on how detailed you are with the idea when you ask for feedback. So maybe after an initial business model drafting, then we will be able to draft some questions. I think so. Like a one pager, maybe a landing page. just like a little bit of articulation not too much doesn't have to be built but it just just needs to be explained all right thank you i think we can ask uh insta kevin after uh the after the decision end of the session at the end we have uh i have an hours or 20 minutes to ask it took you an answer okay i do recommend if you have questions throughout the session just put them in the chat and that way you don't forget what the question is and then we'll go through the questions at the end. And now it can be hard to remember all the questions as we go through. Okay, so let's talk about running these experiments. So as I said before, there's many ways to ask the questions. The tools do not matter as much as the actual question and who it's being asked to. So let's talk about that. So we're going to treat them like evolution in the sense that some elements of every idea are winners and some elements are not winners. And so sometimes you can try the same concept or idea in the same year sometimes and have different results because something has changed either in the market or the price of goods or inflation or anything, right? So one of the things I love talking about is how this applies from animals to companies. So if you look retroactively, data is very stagnant. If you look on a projective basis, so let's say, will this work in the future? then the data is very unhelpful because you don't have all the variables anymore. So let's say the last 15 years, most of the companies that go public have been around for about 15 years. But that's been changing rapidly. So the number of companies to go public every year has been changing. The speed with which companies are valued at a billion euros or a billion dollars has been changing very quickly. So the benchmark for quality has been changing. But we can't keep using the same standards every year. Same thing applies to inside product processes. Experiments used to be really small. Companies used to do a couple hundred experiments a year. Facebook is now at over 100,000 experiments a year, but the experiments of course go down in size. So an experiment could be this simple thing like changing the text on a button, it could be changing the color, it could be changing layout, it could be changing the price, it could be changing the platform, the speed. So if you really look at every single thing as a possible element to change or to experiment with, you can run lots and lots of experiments. Some of the largest companies and some of the really smallest companies all do experiments. It's just become a normal thing nowadays. It comes from manufacturing when it was very hard to do experiments. You can imagine like physical products like chocolate from Nestle or cars. Experiments are six month, 12 month, two year processes. If I'm going to produce two or three cars, maybe I sell the same car in India, the Middle East, the US, South America with different names and slightly different features. And so that's an experiment. right? Each version, each combination of features with different price points is its own experiment. But it's very slow because it's a manufacturer, actually, old cars. I can do concept cars, I can do testing, I can do market days with like convention shows of the car prototype or concept, but physical products are very hard to experiment with quickly. Digital products, of course, I can give every single person in this room today a different version of Facebook, or a different version of TikTok. or a different version of Twitter. And I can probably change it every hour and you wouldn't even notice. And they do this all the time. Even between the Google home screen and the Google results page, many of us will get very different actual experiences. Google allows themselves and their engineers to run experiments down to the phone type or age or years in the country or location or language. And so there's so many different combinations possible. So it's a very important way to run more and more experiments. Now, the most important thing here is about how you develop feedback loops. This is also the most stressful part of doing this early stage, is you can imagine this seems like a lot of work. If you're going to allow customers and early stage customers and potential business partners or customers to ask questions all the time or ask for feedback all the time, It can be a lot of work to respond to them and actually calculate all these results. So one of the biggest things is automating that feedback. So in the discovery phase, which is the majority of the ideation stage, but it happens multiple times. So let's say discovery can happen anytime you make a change to the product or anything you make a change to a feature or any of the elements. This is about when you are looking at the requirements of the product or feature. This is also about when you are getting feedback from customers. We call it field studies when you go out into the environment where the customers live or work. So let's say you're selling a business product, B2B. Let's say for people at airports, you would actually go and conduct surveys, observations, and actually see what it's like to work at the airport, see the problem, interview people or customers or users or employees in the actual context of the problem. Let's say you don't have access to that. This happens a lot in medical. This happens a lot in government. You can't just go physically watch them do their work. That's not how it works. So maybe you need to invite them into a conversation, a phone call, a dinner, a lunch, and get feedback either in a group or one-on-one. Maybe you are further down the process and you are actually ready to start doing some preliminary sales, some initial kind of service agreements. And so you actually are recording every single sales call, every single support call, and you're then analyzing either with AI or with a human, you know, what... what who in the sales team is having the best sales or the most success what are they saying which customers are the ones that are saying yes which customers are saying yes which questions are those customers asking and so you're really monitoring and looking for trends and patterns in the success of your sales and the success of your kind of early adopters to indicate what will work for more customers. Then you have in the exploration stage, let's say you've done discovery, you think you found something that's really insightful, very important, you can go build it, maybe you can go sell it. Before you go and build it, you need to make sure that no one else has built it. And if they have built it before you, did it work? This is the step I think most companies are really bad at, big and small. You see companies build things that you saw another company build two years ago and it did terribly. And then you watch them make the same mistake. It's as if they are not paying attention to the market and to the landscape and to their competitors. So competitive analysis is one aspect of it. You want to be sure that just because someone did it, you could probably still do it and have better success. You need to make sure that you're learning from the mistakes, that you're not making a duplicate of the mistake. Maybe there is some fundamental change that you made. Maybe you're targeting a different persona or different group of customers. So this is something that's also super important. Sometimes you see something work in the US, like an Uber kind of company. And then in Europe, it works fine. It's the same market, same consumer demand, same habits. And then other times you see things like Uber launch in China and it completely fails. And you have to think about the environment in which you are selling that product. It is not just the consumers are the same, but also the financial landscape, the government landscape, the access to capital, digital cards, all of these things impact the access to the service. So when you look at personal development, don't just look at who is the user of the product. Look at who is the enabler of the product. So you can't sell a car ride service if you don't have riders. And you can't sell car ride service if you don't have drivers, right? And you can't sell car ride service if you don't have cars. So the access to all three elements of the service dictate the ability for that service to scale into different markets. Another one that I think is quite simple to do, again, before the MVP, before you build anything, is you just look at every single process you're going to have in the company to be able to sell the product. So this is how do the customers find you? Is it going to be email, phone call, website? I'll give you a fun example. I've helped a couple companies scale from the US and from Europe into South America. And they essentially hoped that in South America, they would make all of their sales through emails and through calls. Big, big mistake because South America is a 95% WhatsApp market when it comes to sales. Even if you're selling fridges or enterprise B2B contracts, people communicate on WhatsApp. So even the airlines send boarding passes, the airlines say gate changes, flight cancellations via WhatsApp, not SMS, not email. Number one channel is WhatsApp. So it's super important that you look at the region, but also that you map every single step and then you compare that step to, let's say, the market. You want to do emails. Is anybody else doing emails? So when you do competitor analysis, it isn't just looking at the feature list is the same. You also want to make sure that you're using the same technology. If you're using the same contact methods, if you're using the same outreach communication tools to actually reach the customers for acquisition. And the rest of this is pretty simple from there. So you want to document your features, rank order them. But this. you'll kind of get feedback from customers. All right, and then second to last is the testing model. So this is, let's say, when you're ready to start building like a version one or some kind of prototype. This is when you go into research. If you're selling a product that has to be used, let's say, as an internal tool or B2B, like for other businesses, think about who the actual buyer is. separately than who the users are. The person who is the CFO buying financial tools for a company is rarely going to be the user because the finance department is full of people who do accounts payable and receivable, and those are the actual users. So even if HR or the CFO or a VP is the buyer for your product, make sure the buyer is not the only happy customer. Make sure the users are also happy or they will get complaints and they will be forced to actually switch products. So you're... not just thinking about the buyer persona, but whoever has to train and teach the product, and then whoever has to end up using it every day. And then this is really dependent on the model, or let's say whether your product is private or public or who the customer is. But if accessibility is a big requirement in government or international products, that's super important to check. And then in general, for B2C products, it's really good that you get some amount of customers to be kind of in an internal group that you can ask for feedback on an ongoing basis so that you don't have to always do these public requests for feedback. You can just go to this beta group or this test group, maybe to get a discount for giving feedback. Maybe they have some kind of special relationship and they get first access to features. Think of some group that you can use for reliable testing. And then listening is quite literally just listening. So not just collecting the data, but also how you use it. So let's say you have AI synthesize your notes. Maybe you have tons of phone calls from the sales team, but no one goes back and listens to hours and hours of sales calls. So how do you actually extract the insights and the data from those calls? Think about the ways you're going to actually store them and then how kind of constantly you're going to reflect on them, share them with the company and the team. So now let's talk about 2-2-2. what is design thinking, the actual process. So I'll go into kind of like the textbook definition of design thinking, and then I'll go into what it means in like a modern business. So design thinking is like a pretty old methodology. It's like from the 70s, but it's also based on a very old concept from 1956. But the definition is that it's used to solve complex problems, and it's a way of systemic reasoning and intuition to explore ideal future states. So it's really what I like to say as a joke is design thinking is the way you can tell the future using data. And so it's really about the end user or the customer and what their actions will be in the future. So as I said before, it comes from a very old concept, but it's been used for companies that were heavily on the manufacturing side. Then it became something that was used for like digital engineering. So kind of when we had the computer and GUIs, so 2001, it changed its definition a bit. And then it changed kind of like in two different ways. On the manufacturing side, you had people making the computers, like the physical hardware, and you had people on the other side making the software for the computers. And both of them had a very similar definition, but for very different purposes. Usable hardware had to do about with when customers around the world started to have disposable income to go and buy things that they didn't need, the things that they wanted, right? So you have so many choices at this point that you want a Mac, you want an Apple. It's more expensive. It made trade-offs. It wasn't every person's ideal computer. It was a certain customer's computer, but it had to be usable. So there are certain things that it had to do, that every computer has to do. And that was kind of what was defined in the 80s. And then since then, nothing really... super crazy has happened to computers. What we've done is we've made it personalized. So different combinations of features to work for agencies or creatives or law enforcement or for young people, just modified which features we bundled together. But the core usable part of the computer has almost never changed because it's what it does. Then we have like studies about how you make this process. These are not things I expect you to read. These are just screenshots. of like the science behind this, and then modern times. So McKinsey is a big consulting firm, and they have a division called McKinsey Digital, where they talk about how digital products are completely different than consulting and physical services. So when you're infusing your organization or your business with the design-driven culture that puts the customer first, it may provide not only real measurable results, but also a distinctive competitive advantage. This is when companies were competing with IBM, Disney, like companies that are legacy, right? Don't think digital first. Companies that are built in the last two, five, ten years, digital is the default, right? You do your marketing online, you do your sales online, you do your hiring online, you do all your acquisition mostly online. It's very different than companies that had to physically build dealerships and sales centers and that kind of thing. Look at an Amazon who has actually gone from the old school model to the new school model, and they have very different opinions about this. So at Amazon, the executive team is required to call in and actually visit call centers and listen to customers firsthand. This is a very rare tech. Customers oftentimes complain that the company doesn't know how they feel, that the executives of the company is out of touch with reality. But at the best companies, even the people at the top, the VPs, the CEO is actually in calls, listening to customer service, listening to sales. All those quality assurance recordings actually get used for quality assurance. And that is kind of like the core of the process. First, you empathize. You want to be as close as possible and relate as much as possible to the customers who use your product the most. Second is you want to define what is the success of your product. Third, you want to ideate on multiple ways you can execute that success, multiple ways to actually achieve that outcome. Fourth, you want to actually start designing some of those ideas. So how does it look? How does it work? The colors, the layout, the size, the order of information. Fifth, you want to actually build a functioning part of this, like a prototype or a test that actually works. And then sixth, you want to build something that not only works, but can scale. So it working for you and your computer and like one phone is a test. Maybe 10 phones, maybe 100 phones. It working for thousands or millions of people is scale, is implementing something to be used. by the wider public. So just as a quick summary with some funny gifs, when you empathize, you really are trying to understand and document the perspective of multiple people. So customers, non-customers, people who say yes in sales, people who say maybe, people who say no, you want to really document all of this and really understand why. So you're looking at what people say. You're quoting them, you're recording them, anything you can find about what the words they actually choose to say. You're trying to interpret how they feel, frustration, happiness. Maybe they feel things are too expensive or low quality. Any of their feelings, which you probably can't interpret directly by their words, maybe in their actions. And then what are their actual actions in terms of doing things? The call rate, the return rate, the frequency of purchase, things like this. In the definition stage, we're more so looking at the amount of data you're going to have at this point. It's going to be very confusing. So you need to actually group together what is good and what is bad. So you want to start to do a correlation between success or indications of success and indications of possible failure. Ideation is going to be, let's say, your most fun part. So this is what we call like divergent thinking. So these are ideas that can be wild, crazy, you know. infeasible. Think of things just because you thought of them, not because they're possible. And then during the prototype stage, this is when your engineers are going to yell at you and your sales people will say, hey, we can't do that. It's not possible. That won't work. It's illegal. Whatever it may be. And so when it comes down to actually executing the ideas, wait until you get to the execution stage before you start killing ideas. It's much better to have a thousand bad ideas. and find out that two of them are possible than to only have two ideas and find out both of them are not possible. And this is when you get to the modern definition of an MVP. An MVP, let's say pre-2010, would be it works. Customers used to be willing to try hideous, terribly priced things one time because it was functional. An MVP nowadays... does not work, does not get sold, does not get used if it's purely functional. It has to be functional. It has to be pretty good with reliability, with quality. It has to be something that's easy to learn how to use. It has to tell a story. It has to have a good branding. There needs to be like a very clear vision. They want to know who owns the company. They want to know the vision, the mission. They kind of want to know the whole future from day one. And it's a very different way to sell a product nowadays because customers, the consumers are a lot more demanding. Companies have been so transparent and vulnerable the last couple of years that there's a lot more scams. And so customers really care about investigating all the details. They're not just going to see a good product and purchase. They want to see a good product. They want to ask questions. They want to know who else has already used it before. They hate being first. So the earlier adopter group is getting smaller and smaller. And so you really have to explain the quality of the product in a lot more detail. And quality is not just the functionality. Quality is, do I perceive it to be good quality? Is the website good? Is it glitchy? Is it hard to get a human on the phone? Is the sales terms confusing? So everything that can lead to a perception of it being fraudulent or a scam or bad quality essentially equals bad quality. And then you get to the test. And so you take your same prototype you had in the last phase, you see the ones that did well, hopefully you have multiple prototypes and multiple ideas that you think could be viable. And then you just run it through the three questions I mentioned before. Do people find it efficient? Do they find it effective to solve their problems? And are they happy with that solution? And essentially, you're trying to take the one that has the highest scores on those three topics. It doesn't have to be a physical number, but you want to build products where there is a high level of... perception of efficiency, perception of effectiveness, and generally customers are happy when they go and choose it. They don't want to feel like they have no options and they're forced. You really want them to willingly choose your product. This is the difference between an Outlook and a Gmail. Most companies that use Microsoft, their employees are forced to use Outlook. It's the only option because the company has Microsoft. When a person in their personal life uses Gmail and at work they use Outlook, it's almost because they have no choice at work, not because they actively want to use that product. In the free market, in the public sector, private sector, I would say, sorry, you actually want customers to never be in a position where they feel they have no other option. You want them to willingly use your product and be happy telling other people that have that same problem, hey, I'm using the solution. It's amazing. You should use it. Here's my discount code. You know, there should be like a willingness or an incentive for them to share, but it should not be because there are no other options. And then let's say you have a test that is successful. Customers want to use it. The customers in the test are really hoping that it becomes a reliable, permanent, you know, not a beta, but like an actual permanent feature or product, and they want it to be used as something that they rely upon. either for their business or for something in their private life. So they want it to be a real product. And that is when you implement it and scale it. And hopefully you sell it to millions and millions of people. So boom, boom, boom, make it pretty, paint it, tighten, tighten the screws, and then get it ready for, you know, the launch day. So this is also very easy to say versus do. Let's talk about what that means in terms of data. You're going to get two types of data in this process. You're going to get very simple. I love it. I hate it. You're going to get yes, no data. You're going to get, I want four of them. And so you need to be sure you don't mix these types of data together. Your quantitative is going to be, you know, how many sales do you have? How many calls do you have? How many downloads? How many views? How many signups? You know, this is like hard facts, but this does not tell you why, right? It tells you what happened. It does not tell you if they were unhappy when they did it, it doesn't tell you if they were confused, it doesn't tell you if they felt frustrated in the process, it just tells you that they actually completed the action. It's going to be way more important during the early ideation stage that you focus on qualitative data. You want to know how easy it was to check out, not the number of checkouts. If you have a thousand checkouts, but everyone was super unhappy, you probably could have had four thousand checkouts. So it's super important to understand the quality of each thing that you build and offer to the public, not just that you built it. Building it is only half the battle. Building it well is your job. Let's see. So there's probably a thousand different versions of the persona canvas online. So there's no particular preference of mine on how you format this. But you kind of want to get away from thinking of your customers as the customers. You want to start specifically mentioning or thinking about a type of customer. And we are much better as humans when we personalize or empathize with a person. So if you can figure out like a fake name for your customer, like, oh, this product is for Emma's or this product is for David's. Like you start to really think about a real human when you actually build product. Would David use this? Like think about your earliest customer. Would that customer actually use this product? So it gives you like a real person's opinion to think about as you make hard choices. So think about their needs and their goals. Think about where they're going to discover this. Maybe it's a magazine, maybe it's a forum, maybe it's an event. Think about how they make decisions, their personality type. Think about their behaviors, if they're conservative, if they're super exploratory, if they are people who are tech savvy or budget conscious. Think of like actual character traits. And it could end up looking something like this. So at the end of the day, you could say, you know, here is our typical entry-level customer. You know, she's young. She lives in Europe. She has a college degree on average. She uses other platforms that are similar to us, like these two products. She has some very clear needs. She has her needs not being met by the current products in the market because those current products don't do some things. How does she discover new solutions or new products? She goes to these three outlets. She's written feedback because we've heard feedback from customers during surveys or during sales demos or during trade shows. And this is what you gather as your persona. And this could be your entry-level, mid-tier, or your super user, your advanced customer. But you start to give that person a face and a name. And it's much easier to think of them when you're actually making decisions about the product. Now... This is different than your marketing personas. Who will buy and use the product is not always the same as who will join a sales demo. Obviously, the demo to conversion rate tends to be a pretty large gap. So think about who actually is retained as a happy, loyal customer, because the more you focus on those people, the less you worry about the people who don't say yes. And if you do this correctly, and you put most of your research in the discovery phase. A lot of science and research says that you essentially avoid the risk that comes with building by 700%. The earlier you put validation, the earlier you put research with customers in the process, the less risk you have of throwing things away and having to rebuild them. And also that means you save money. So faster deadlines and less waste. Now, when it comes down to doing these tests, it's very, very common. People want to do very large scale tests. I will tell you, it is very weird to say, and it's a very weird feeling as a founder. I've done this twice before myself. When you say we don't need to talk to 100 customers, we need to talk to five. Every single founder, every single person wants to talk to 50 or 100 or do some huge survey. But usually it isn't that much better in terms of return on investment. If you can get five actual... real people who would use the product, like actual people you would sell to. Five of those people, not five friends, not five people who are biased towards you. Five genuine strangers who are actually target customers and use these questions with them. You only need five of them. There will always be at least one person who will complain about something. And that complaint represents hundreds of thousands of people. And so listen to that complaint and solve the problem. And you should try to get to about 15% of complaints resolved, and you should be able to move forward. So ultimately, you're going to get data at the beginning, tons of random information. You don't know what to do with it. Maybe ChatGPT or AI can help you because now we have these beautiful tools. Then you're going to get information. So you're going to see a repeat of data. So when data starts to repeat itself, you're going to see a trend. You're going to see what you call evidence or proof that you're correct. You're still not there yet. Third, you're going to get knowledge, right? You're going to say, hey, the data is matching my beliefs. My beliefs are matching the customer feedback. I know this to be true. And then you're going to get wisdom, which is when you know something to be true and the data reflects that and the customer behavior reflects that. And it's been years of seeing that. It becomes wisdom. It becomes experience, right? It becomes expertise. And if you want to go and make this jump from not having to get your own data, then proof, then knowledge, then wisdom, which you can go do so you can get an industry leader who has the wisdom and you can bring them into the company as an advisor, as an early stage tester, things like this. Be very mindful that this is still going to be a hunch, as in this is still going to be based on, you know, personal experience, anecdotal feedback. A person who has done something for 20, 30 years also is biased by things that happened 20 to 30 years ago. So you still want to get people that are fresh, that are directly unbiased by the legacy of things, especially if your market has changed. And the B2C world, a lot of my clients are acting very recklessly using data and research from 10 years ago. And I have to remind them, hey, guys, remember that COVID has changed the market. We're in a second recession. We have AI now, so maybe we shouldn't be using research that came from 2018. Things have changed significantly. Maybe we should rerun that test. Maybe we should rerun that research. So the wisdom is valuable, but do not rely solely on wisdom. And then in practice, this is what the entire process looks like. So this is your discovery phase. So this is your problem discovery and problem definition. So you're going to see what I call the diverge-converge behavior a lot. When you generate tons of ideas, you're diverging. When you are narrowing down to a few ideas, you are converging. So when you're looking at the problem, you're going to discover hundreds and hundreds of versions of the problem. Then you're going to have to pick the one you care about, right? So you're going to have tons of ideas. Then you're going to have a couple ideas. Then when you come to a couple ideas, when it comes down to how to build them, you're going to hear from engineers and from salespeople, hundreds of ways to sell or engineer that idea. So then you're going to have tons of ideas again. We call the solution discovery. And then you're going to have tons of ideas die because You will not be able to execute them or build them or afford them. And then this is when you get what we call a concept validation. So you will only ever build a couple concepts from all of your hundreds of ideas. If you can do the research and the discovery and the validation as early in this process as possible, you will be very great, very happy, because you will avoid doing it during development. During development, you are hoping to find things that are infeasible, as in it's not possible to build. You don't want ideas that people don't want in the development process. You want ideas people want, but let's say no one can build it or it's too expensive to build. You want things that are maybe the technology doesn't exist yet, or you want things where maybe people's personal devices can't handle it. You want there to be a roadblock that is not yours, right? Something that is in the market that blocks you, not yourself. In this process, you will find people will find tons of problems with like loading time or legal frameworks or access to data, things like this. And this is where you narrow down the idea, you get validation, and then whatever is actually passes all of these checks, these milestones is what goes out to customers. But in the idea stage and in most of the next, let's say, first year of any idea stage business, you are focusing on the discovery phase. And you don't want to jump. to development too quickly. You kind of want to get things that are perceived as developed. So prototypes, concepts, landing pages, sales demos, beta tests out as soon as possible and get them out in high quantity. And then you want to make iterations before you build it in a way that you can't fix or change quickly. I'm seeing some questions pile up. This is great. Yeah, these lines on the screen, I also don't know how to get them off. Let's see. Success. Done. All right. Boom, boom, boom. Back to normal. All right. So next we have use cases. So I'll give you some examples. Feel free to ask questions in the chat about any of these. I can go into examples about any of them. This is a ton of them. But let's say this is 90% of what's used. today. Let's say you are very, very early stage in your business and you're still questioning the business model. Let's say you're still questioning the type of customer to go after. This is the assessment phase. So you're looking at your goals, you're looking at your business model, you're looking at what will be your costs and expenses, you're looking at your assumptions, you're looking at who is a good stakeholder or partner or like a vendor you need to work with. to execute the idea. This whole stage, you can still do research on, but this is going to be assessment stage. These are going to be really big changes to the product and the business. Second stage is like fact finding. So this is when you've done some research, but you don't know if it's true, right? Is it only true in one country? Is it only true in one market? Is it only true today because of the recession? Is it only true because of the currency fluctuation? So you want to know how... factual are these beliefs or these findings. And so you want to either scale the confirmation by talking to 20 people, 30 people, doing a survey, buying data online, buying a report, finding statistics, things like this. Things that confirm or validate with a large quantity, with a statistically significant number of people that what you found out from a small group, like your five-person test group, is in fact true or can be found to be true. of a larger population. Then it comes down to like the actual test itself. This is where I spend most of my time. This is the work that I do the most. Some of my favorite tests are called fake door tests. We'll talk about that in a second. It's also like dogfooding. Dogfooding is when you get people at your company to use your product. For example, at an Uber or a Glovo or Amazon, this is when they are forced to use their own product and then they have to actually go do one of the services. So imagine an Amazon worker who works in sales or marketing actually being forced to go and deliver products using a truck. Or imagine an Uber Eats employee from the finance team actually having to get a scooter and go and deliver orders. And so this is what they call dogfooding. So you're kind of eating the food you give to your pets. If you think about your customers as your pets, this is that idea. Then you have your labs idea. Labs are where you essentially build a separate product. It's like older companies or bigger companies that have one core product want to expand. They generate some really crazy idea that is relevant in terms of the industry, but completely different in terms of the actual offering. This is like Google getting YouTube. Right. It's like, cool. Google does cloud storage and they do search and discovery. But videos are very different than internet search results. So it's a similar functional experience for them to build, but it's a very different offer to consumers. So this is what would be like a labs or an innovation team. Early adopter programs are just like focus groups. Smoke tests are my favorite. This is very similar to what I mentioned earlier as a fake door, which I will go into in one second. But like a fake door or a smoke test is. I don't know, the most fun thing you can do, I think, is when you build a product only aesthetically, like a landing page or a sales lead form where you do advertising for a product that doesn't exist yet. And you actually see in the market who actually genuinely reacts or asks for a demo or pays or makes an account or signs up. And then you have to tell them, hey, actually, you are an early adopter. This product is coming out soon. It doesn't exist yet. Thanks for sending up. And this is exactly what Tesla made very, very famous. So people were buying Teslas one, two years before the car was ever produced. They only ever saw a clay model or like a plastic car that had like a little RC engine or like a robotic, like a little fake engine that was pushing the car in showrooms and in demos. or they had built one version of a car by hand. They'd never actually built the manufacturing center yet. And so this was a big smoke test. It's like, do you want this vehicle? Do you want what we say it can do? You don't know if we can do it yet. You've never seen it. You've never touched it. It isn't in a dealership. You're seeing smoke. You're seeing smoke, right? And I'm selling you smoke. I'm selling you this vision, this thing that is kind of foggy in the cloud. And so this can be extremely valuable if you have a very complicated product to build. This happens a lot in super high-tech, hard tech, like computer chips, car manufacturing, boats, airplane, fuel. They talk about the future of battery technology. You can't actually physically experience it. They have to sell it to you as a vision, as a concept, as outcomes, as results of what it could be able to do. And they ask for your money or they ask for your partnership today to build something in the future. This can be very, very valuable if you have a very complex or a very slow development time in the product space you're working in. So let's talk about how this works. And this is my favorite one. This is the fake door test. On the outside, it's a real door. You open it and there's no hole. It's a wall. Right. So this is going to be how you test some of your best ideas without spending any money on engineering and actually building things. Could be as simple as like an email. or a signup page or a pop-up pre-announcing a feature or pre-announcing a tool or a new tier or a new business plan, and then asking for email, asking for a password, asking for a down payment, whatever it may be. It can be more complicated. A company like Buffer that competes with Jira, this is like a task management platform, and it also kind of competes with Notion in terms of it's like documenting, like work documentation. They were going to sell this tool. with like a free plan at zero dollars one at five one at 20 and then when you went and you actually went from the you know the initial offer to the pricing page to actually go and buying one Only at the buying stage would you realize, hey, we actually haven't built this yet, but thanks for your interest. Give us your email and you'll be the first one on the list to get access. Now, you can do this multiple ways. You can do this in two ways. One of the ways that Buffer did it is that they did it before the product was built at all. So they actually just gained interest and they used that interest to get funding. So they said 100,000 people want this product, right? And they want it at this price point. This validates not only the product, the functionality, but it also validates the price, which is a very important thing to do. The other thing that you can do is you can do it as a gatekeeper for scale. Look at, this is how Pinterest launched, this is how Twitter launched. They couldn't, and Facebook launched the same way. You couldn't, they couldn't afford to have the product to be used by billions of people day one. They could only afford for there to be maybe a thousand customers. So then they created FOMO, and then everyone who wanted to use it because their friends were using it had to wait in the wait list. And so they knew that they should open up in Canada or in Spain next because of how many people were in the waitlist in that region. And so they actually used the waitlist to prioritize feature development and they used the waitlist to prioritize expansion of the business. And so you can actually have the product built, functional, working fine, but you don't know which markets to go to next. You don't know which features to build next. And you use this kind of smoke door, fake door test. to indicate interest, and then you change the roadmap based on where the interest is. So you kind of move the supply to match the demand. Again, in a product that already exists, you could have a very basic version of the product and you could start teasing the next version of the product. And you could say, sign up now to be first in line to access the pro version or to access the AI feature. And then all you have to do is register and track. people who actually asked. And so the waitlist can also be for future development of an existing product. Go through a couple more examples and then we have plenty of time for Q&A. I think it should wrap in 15 minutes. All right, so let's go through a couple of different real company examples that are not all in tech. So In the software as a service, in the B2B space, you have Vanta and Cocoon. These are smaller stage companies. Feel free to look them up. They went into a marketplace. They built a space first and looked for a problem. So these are the companies that you hear raising tons of money. Maybe they're backed by like a public fund and they're kind of looking for a problem to solve. But the market is so big that no one has really solved the problem before. So maybe there's just too little knowledge of what the space is. Imagine AI or imagine when Uber was first to car share, there was so little data that they really didn't know if they should do high end, low end, if they should do scooters or bikes. Now they've expanded into all these different categories and it seems obvious now, but at the beginning, there was so little data to go off of that they were just opening the industry. They were just the first one there and they were pretty much just looking for a problem to solve. Then you look at the second tier. This is when you have intuition. Let's say that there's been a very long industry existing, but no one has offered a product in a certain price point or a certain business model. And so this is really big in the consumer and marketplace space. This is like someone disrupting a booking.com. This is someone disrupting like an Airbnb. This is someone disrupting like a class pass. Like gyms have already always existed. But gyms like to lock you in to like a six-month contract. If you could pay 30 bucks a month and use any gym in the city, that's a very big disruption to the model of gyms, right? It's the same space, the same product. It's a very different payment method. It's a very different subscription and commitment model. And so they're not changing the business. They're changing the access to the product. Look at third. This is mostly infrastructure developer tools. This is really about how expensive it is to experiment. So these companies are not only taking on the risk of building something very complicated that they can't change very quickly, but they're doing something that they know that their competitors also can't compete very quickly against. They built something super technical in a certain way that no one else did. It would take months or years for their competitors that are much bigger than them, even with the cash they have to build it that way, because they built something 10, 20, 30 years ago. The industry around cargo shipping is so old, they don't think anyone else has a household name in tech besides Flexport. They're the only one that have actually made cargo shipping and international shipping a new household name in the last 20 years. Like FedEx, DHL, there's a list of companies you know in that industry, and then there's been no one new for a very long time. And then insert Flexport. And they essentially built a consumer-grade international shipping business out of nowhere. And so it's really not a problem with the industry or the product. It's really about how accessible it is and the experience of using it. So they've made a consumer-grade B2B shipping platform. And so it's a very risky bet because if it doesn't work, it was very expensive to build. When it works, it works. So the four elements of running a test is you want to make sure that the test is easy, that it's timely, There is some level of it looks really good. So you want it to be like a teaser of what the real product will look like. But then you also want it to be social. If there's any way to get validation in your product or in your feature or in anything you're building and you know people will talk about it, give them the ability to share it. So this is kind of like, you remember the days when everyone had a promo code or a referral code for Uber or for Netflix or for any platform? Use my code. I get 10 bucks off. Use my code. You get 10 bucks off. a discount something, you want there to be like an incentive for you to spend zero dollars on marketing. And for your customers who are happy, your early adopters that are happy to bring you more free customers, like free in terms of marketing. And then of course, they that they pay. There's a couple different models that I'll explain really quickly. There's about 10 of them, and each of them has lots of benefits. You may have heard of the flywheel effect, the flywheel effect. is essentially the momentum of a business. There's usually some action in the business that has a outsized impact. So maybe you buy something that's great from a store and then you come back next time and you buy it again. There's no growth there, right? You're going to buy apples or groceries at the same rate you always buy apples or groceries. Now, if I open a new grocery store and I steal customers away, And you come and buy groceries from my store, but then now you tell other friends and now they come to my store. I spent $1 on marketing to get you and I'm getting 15 customers. So it's going to be that referral process that actually is the flywheel that actually generates more and it returns more than I invested. The same thing happens for products. So let's say you can actually get customers to use the product. So they come in, they make an investment. They sign up, they make an account, and maybe they fall off. Maybe they leave or they abandon the product. This happens in shopping carts. This happens in enterprise sales. It happens everywhere. If you can figure out what gets them back into the product, and then if you can get them excited and loyal and to convert, and they reach a milestone in their personal life or in the life of the business, then you create a routine. And then that customer is a loyal customer. And anyone else who ever asked that customer, hey, what do you use for email marketing? What do you use for your shipping? What do you use for XYZ? They will always refer to your company because you are now a lifecycle product for them. You are now a heart. You are a pillar of how they do business or how they do operations. And that is the kind of customer engagement you want, especially if you are a company that needs long-term customers. Most companies need between 18 to 24 months to make the money that they spent on marketing a customer to get paid back. Some companies need a couple months, some companies need years. So like Netflix charges you $15 a month. Let's say the licensing costs for Netflix, if you watch every single movie on Netflix, it's probably a couple hundred dollars. So they need you to be a customer for at least six to eight months to make back the money they spent. to get a license for you. That's fine, right? Imagine if you're in medical. Medical, you're spending 10 years building one drug, one medicine. You need hundreds of countries to be customers for probably 10, 20 years. So you're really looking for like that routine, that like longevity, and you want to defend your space. So you're not looking for a product that is 1% better. You're looking to build a product that is 10 to 20% better, but you can hold the customer is longer. And so this cycle works in any industry. Look for where the investment comes from, who it comes from, what time of the year, why. Look at how much progress and how much time it takes to actually be onboarded. Look at how many milestones you have to trigger. And then what actually generates the motivation for when the customer decides to stay or become loyal or to commit. And that action and reward need to be very close to each other. So if they, let's say enterprise companies. Need to onboard not just the buyer, like let's say the person in finance, they need to onboard the staff. So as fast as possible between them paying for the contract and them onboarding their staff and their staff making accounts, it needs to be the very tight, short process. You want customers to feel satisfaction and success as close as possible to payment. So if I've paid. I need to get as quickly as possible through onboarding and through sign up and through training as possible. So I feel the value of having paid for this product. Now, step by step, really quickly, we have a couple examples from Starbucks, Facebook, and Duolingo. These are kind of like the darlings of engagement. They do really, really great job, sometimes too good of a job. Duolingo has some awards for how aggressive their engagement tactics are. They ran a campaign, I think earlier this year, where the app icon for the Duolingo language app changed if you didn't use the app. So let's say you're trying to learn a new language and you haven't logged in in four months. He starts crying. He looks sad. And so you get these kind of daily reminders that you're not an engaged user of the product you're paying for. Very, very smart. It was very effective. I think they won a couple of Webby Awards for this. It was super interesting. And so you're looking for either external or internal triggers to get a customer to do what you want. These are external because they're not even in the product, right? They're using email, they're using SMS, they're using text, they're using the App Store icon to get you to open the app. These are external triggers to get you to go back to the job you were trying to do. Let's look at motivation and ability. Companies are similar people, so I will use the language equally here. They don't want to do things unless they feel that they need to do it. If you want them to do something, it's on you. If they feel like they're missing out, they're wasting money, they're being inefficient, then it becomes a personal problem of theirs. Then it becomes a self-motivation. And so there's a certain amount of actions you can get someone to take before they feel passionate or committed. And then once they're passionate or committed to an outcome themselves, they will take a lot more actions. This is where you get the business model for freemium. So you give away something for free, a company or a customer gets used to a habit or a routine or a pattern of behavior with the free thing, and then they max out the value they can get for free. So now they have to commit to a paid plan or to a higher tier or to a subscription or to a contract to get higher value. And once they do that, they are committed because they don't want to waste the money they have now invested. And so they will now take more actions. than they would have taken previously on the free tier. On the free tier, they waste money all the time. This is kind of like your typical gym person. The gym price point is low enough that if I don't go a couple of months in the year, it's still worth it because I go a lot in summertime. They don't want you to go. Gym business models are based on you not coming. If every person who has an account at the gym came to the gym, the gym would be unusable because it would be full. So they are relying on about 40% of people who go and buy a gym membership to not ever come in because they actually could not service that many customers. So this is the opposite of this model, but it's all based on motivation and ability. So what you're able to do and what you want to have happen in the platform you have control over. So you need to decide how much action you want and at what time. Then it's the actual action. This applies to everyone, whether you're B2C or B2B. This can be purchase. This can be bookmark. This can be liking. This can be commenting. This can be sharing. This could be swipes. You need to figure out what the ideal number of actions are. And you need to make anything below that number feel like it's not enough. And you want to make that number feel like a big milestone when it's been reached. So Tinder and their free plan. only lets you have a couple of dates with a couple of people. And then it says come back tomorrow because they want a certain number of daily active users. So they limit how many actions you can take per day to make you, to force you to spread out actions over time. TikTok, Instagram, same way. You can do unlimited likes, video watches, plays, comments. They love this, right? So if you don't do any for a while, they're going to find ways to make it feel like you have failed. So when someone posts for the first time in six months, they tell all your friends, Jacob has posted a video for the first time in a long time. Please go watch. Because the watching incentivizes the customer, the user to do more of that action. So they generally create a bunch of environmental triggers to reward behavior that they want to see more of. So they kind of incentivize the action that they want. This is very similar, but a little different. This is called a reward. Reward is oftentimes linked to Facebook. Facebook has, let's say, mastered the manipulation of reward. So variable reward is you can do the same thing in a product multiple times and you don't get the same feeling. The product doesn't reward you or celebrate the action in the same way. And so let's say you come in as a new user on social media and you're setting up your account. You add a profile image, you add your friends, you add your mom, you add your brother, whatever. The first day, they're happy. They want you to add 30 friends. The next day, they don't care if you have 30 friends. They don't want someone who just constantly adds friends. I'll give you a better example. LinkedIn. LinkedIn at the beginning loves when you add all your coworkers, all your network. But if you add a thousand people on LinkedIn a day, LinkedIn now sees you as a spammer. LinkedIn now sees you as someone who is maybe using their network for marketing and they're kind of abusing the platform to do sales. And they don't want you using LinkedIn for spamming people. They want you to pay for the marketing suite they have. They have enterprise software. at LinkedIn. And so they change the goal every day. At the beginning, as a new customer, a business wants you to do certain things, set up, sign up, add new users, add your team, things like that. After that phase is over, the goal is different. They want you to be engaged. They want you to use new features. They want you to discover new things. They want you to upgrade your account. So the reward or the incentivized action changes based on the stage and the business or the stage in the relationship. between you and the customer. Second to last, second to last here is we have investment. Investment could be money. This could be upgrading your account. It could be paying for a premium feature like a blue checkmark. It could be paying for like a higher end tier like a family plan on Spotify. It could be any of these things that require some kind of upgrade. And so the company really wants you to know at all times that you're not on the highest tier. They will kind of make you jealous or envious of anyone who has a higher tier, or they will show you things that are higher access that you don't have access to all the time, just so that they remind you that there's one more level to pay for. And they kind of use this as an incentive to get you to invest more. And then the last one is progression and milestone. This is big both in B2B and in B2C. I think B2C is where it started. It's the most common method for high-scale growth in 2008 companies, is they would make everything multiple steps. So you're at level one or level two, or you have 15,000 airline miles, or you have a thousand credit card points, or you have a certain interest rate on a loan at the bank. So anything that implies that like... better behavior or continued behavior or action will get you a better deal or a better rate or more access. This is progression and milestone triggers. So you're going to use level of effort and milestones to incentivize you to keep a behavior that they want you to keep. This is important for you as a business owner, because let's say you need to have recurring users for an ice cream shop or for a business that has lower priced goods. You need to have something that encourages people to come back every day. Like restaurants have the punch cards. So if you buy three meals in a month, you get a fourth meal free. If you come back 15 times to buy coffee, we buy your next coffee. You buy three tires, you get one free tire, whatever it may be, right? This is where that comes from. The other thing could be stage. So let's say you fly 20 flights a year and now you have platinum access. But... You can't keep that forever, right? So we have to tell you that the cutoff for platinum access is January, and that next January it resets back to zero because you need to get platinum next year again. And so you have to maintain this stage. And so they want recurring, sustained behavior. And so they create these, let's say, variable rewards with a very long progression bar. So they have very high, very hard to reach milestones. So just to summarize how this relates to the first section of the session, you figure out who you're selling your product to. Then you actually need to go and involve them. You need to reach out to them, your target group, and talk to them. This can be as little as five people. Then you need to analyze what are these five people using in comparison to you. So what are they using today because your product doesn't exist? What would they use because your product isn't good enough? What would they use because it's cheaper or faster or easier to learn? Think of any version of better, not just better overall. Number four is think about how you sell or pitch or tell people about your idea or product. You want to get feedback on the way you pitch, on the way you sell, on the way you describe it. And then you want to combine the idea you have with what you learn in these first four steps. Because the idea you had from day zero should be changing as you get influenced by the feedback you get during these steps. Then you can apply what we just talked about. Then you start to do the tests, the experiments, the fake door tests. Then you get early adopters, early users, early customers. And then you start to build prototypes that are being used by a small group before you make them, you know. legal compliant and scalable and international and all the things you need to actually build it for the public market and to go to a wider audience. But you never let go of the step of keeping user feedback whenever you make a change to the product. So this is the full eight steps. And then this might be a good one to screenshot for anyone. When you're thinking about what does that mean? What do I need to do in terms of getting evidence? There's a bunch of there's levels of quality for getting evidence or running a test. You can always ask people what they do. People are not very good at memorizing every single thing that they do in their lives. They're going to get a very general summary. You can ask what they've done in the past. People have a better memory of what they've done in the past than their ability to predict the future. Or I would do this or I will do this. You can ask them about specific scenarios or stories. So this will give them a lot more context and you'll get a lot better answers. But it's always better to watch someone actually do something live. So to watch them have the problem or to see them use a new technology or a new product that they've never used for the very first time is a very insightful experience. The number one thing would be to get it to happen in a real life scenario, not a fake scenario. So if you can watch Patience. hospital, if you can watch people at an airport, if you can watch customers at a real bank, that's always going to be a lot more insightful than interviewing customers at a bank, than interviewing staff at a bank. You want to see them actually behave in their normal environment without knowing they're being watched, because that is going to be the realist thing to life. So if you can ever get that, that's going to be your best level of research. up here. I still have some time, but there's going to be some themes you realize in your research, in your notes, in your feedback from customers, and there's going to be four separate themes. There's going to be reasons why people do or do not want something, and this is really purely about like what's cool, what's popular, what is important to them, what they value. This is not about the quality. This is really about preference, right? Then you have viability. This is going to be about the cost. This is going to be about the frequency of payment, the payment method, things like this. Feasibility will be about whether it's possible to actually do it. So feasibility in terms of time they have or engineering or capacity, things like this. And then strategic fit will be things like vision, mission, values, alignment. This is really a lot more about ethics or ethos. So things that they care about or don't care about. So all of these are going to be super important to look for, but it's important that you realize that there are four separate families of risk. You can generally ignore some of these risks depending on the industry you work in. If you're working in ecotech and your company, I don't know, eats hamburgers at every marketing event and you drive diesel engine trucks everywhere, maybe your ethics are going to impact your business and your sales. Right. So that's going to be more important in that business. Maybe for selling a financial product and there's only one option to pay upfront for a year in advance and your target audience is small businesses who have cash flow issues. That's going to be a feasible or viability issue because the cost of your product doesn't match the audience you're selling it to. So just think about how important each family of risk is for your business. It's very different per business. And then if you're like, OK, cool, that's great. Which questions should I look for or ask? You can also use these questions. These are the most used questions for determining desirability, viability, feasibility, and then strategic fit. These are questions I actually use for work. You can use them as is. You can also, I would recommend changing our company to like the actual name. So I've seen people use them without changing them and they look very robotic. So don't just verbatim use them, like think about making them more human. And then there's a little section we have on persona development, but this is something that you will need to do using your research and your feedback from your customers. This is just an example of one. Everyone knows booking hotels and this new concept of booking home vacation rentals from Airbnb. My funny little example is that before, at the very beginning of time, you just had people who would travel and write little books and they would say, go here, do this thing, it's great. And then we had photographers and videographers and media companies who would say, hey, we went here, we did a documentary, we interviewed the locals, we translated. the native speakers, whatever it may be, go there, do this thing. It's great. Then you had travel agents whose whole jobs it was to actually go and actually see them for themselves. And you would trust that person individually. And so you trusted the individual to represent your tastes, your likes, your desires. Then we get this beautiful online experience where we all felt like we could go anywhere in the world in two seconds using our computers. And then this kind of replaced travel agents. And you didn't have to trust a single person anymore. You could trust thousands of people. And then we pretty much got Airbnb, where you have individual reviews from individual people on every single hotel and every single property in the entire world. So no longer do I have to trust a guide or a curator. I can now trust individuals that match me, that match my values, that match my needs, my demographics, but also my psychographics. Not just what I can afford and who I am and what I am, but also what I prefer, what I like, what I need, what I believe in, what I care for. And this is going to be super important for sales nowadays. It's very, very common in marketing and sales to look at, oh, our customer is single, makes this much money, lives in these places, has access to this, and is around this age. That is less an indicator now of commonality than it used to be. People of the same age group, same demographic region, same location, same language will still have very different preferences, very different belief systems, religions, very different thresholds for what is expensive or not expensive, very different priorities. And so when you're doing your research, don't just ask questions like, hey, which gender are you and where do you live? Ask them about, hey, how did you hear about us and why did you use the product today? Like, why are you looking for this solution? It's going to be super important to understand not only the action they take, but the reason why they took that action. I think that's my last reminder. I'll send the deck to everyone so you'll have some of these examples. I have another framework called Scamper, which is just about how to look at some of the results you get. So if you have an idea, you think it's good, but it doesn't work, think about how you can replace one element. Think about how you can combine ideas. Think about how you can adapt or modify, or you can remove an element, or you can actually reverse engineer an example that exists in the market. So Scamper is a methodology used by larger companies when they are being disrupted by smaller new companies to actually compete. And then there's a bunch of frameworks that are in here, but I'll send them all to you until you'll have all of them. There's also a free customer engagement book from Intercom. The link is in here. You can download it. And it also gives some additional examples of how to do some of these tests. So I'm going to skip now to the Q&A. And we have exactly 20 minutes for Q&A. Thank you, Mr. Kevin, for your interesting workshop. Then now we'll start with the questions. iPad 4, who has... Oh, firstly, Udaya. Yeah. Hi everyone, hi Kevin, thank you for the very entertaining and very informative session. I have a quick question, we are building a platform, we are quite advanced in the process, so we're getting close to the MVP, and our platform requires an enabler, because we need the retailers, basically something for the customers and retailers to negotiate and agree on a price. So it should be used by the customer. Yet we need the retailers to download it and put it on their website. And we have to get them to buy the idea first. We haven't been having so much luck so far because, you know, as an early adopter, we couldn't convince some retailers to work with us that bad. And we are left to decide on the MVP ourselves. What are your thoughts about building an MVP that should be centric? you know, customer centric, yet, you know, we don't have access to those early adopters just yet. Okay, this is great. So this is not just common in the retail space for let's say, like your normal store, but this is also a really big part of CPMG. So let's say you're trying to get a product like physical food into a store, the store does not want to give new makers of food retail space in the grocery store, right? They want to see that you already have some original base or there's some demand, but you're going to have trouble confirming that demand exists without being in the place where they shop for that thing, that product space. So in this case, you need to do brand marketing. You need to essentially sell what makes you special or makes the technology unique and see if you can get a large enough interest group. This is going to be a great example for like a fake door test. So if you can do regional marketing and specifically get enough customers in the same region to indicate interest or want to use this, you essentially create a pressure on the retailer. This is something you see very similarly for food products. Like when vegan food wasn't available globally, you would see companies getting petitions like 100,000 people want vegan food on this airline. Brazil has a very funny story that you don't used to not have anything besides steak on their flights because it's it's common enough to have really good quality meat and not to be desirable so there weren't really high requests for vegan food and so to get that to happen they essentially offered vegan as an option and the number of requests for vegan that were so high that they said okay let's do one flight a week where we have the option and if we hit a certain sales milestone we will expand that further This is now the general model for how grocery stores and most retailers allow third-party sellers in their platforms. They'll give you one shelf at the bottom of like row two, aisle two, and they say if you sell 50 in three months, then we'll give you a higher shelf or more shelf space. So see if you can get a level of indication, like a signup list or a wait list, something that's a bit more specific to, I want this in my region, my region is this location, and use that. as an indicator to appease or reduce the risk assessment for the retailer. Wonderful, thank you. Very, very, very informative. I have more questions, but I would gladly yield the stage to the colleagues and I might, if we have time, I might come back for more questions later. Very interesting. So I think, Kevin, if we can answer maybe the questions in the chat first. Yeah, I think there's a couple. Yeah, I'll go based on the timestamp. So I saw one from from sayida what are the best ways to audit user experience of our platform by ourselves this is a really great question um if you already have something built and there's already a user experience to audit or if you're auditing let's say a competitor and you want to know to kind of compare multiple competitors and see who has the best user experience you can actually use any platform the ones i recommend are usertesting.com It's probably the best one, but you can also look up any competitors to them. Like Usability Hub is another one. I'll write this in the chat as well. These are going to be the platforms where you can actually send a screen recording, an image, a design mock-up, and you can actually ask customers. And they will actually find customers for you. So you can say, I want 1,000 customers on a mobile device in Oman to take. this test. Now, a thousand might be super high. Maybe you just do like five or ten. And they will actually go and conduct the study. They have a certain price point where they pay people to take tests. The more number of people on the planet who meet the criteria, like English speaker versus Spanish versus whatever, the cheaper the test costs. You can get very niche. There's no problem. I've done really specific tests. I want to talk to people who work at a bank in Brazil. who speak fluent English and get a test back within the next three days. And we do it. So it's a very good platform to test other people's products because you don't have to do any work. All you have to do is submit the product, like a link or a screenshot or something, and the question you want answered, and they will conduct it for you. Okay, let me go to Mohanad. This is exactly the customer file. Let's just make this. Okay, cool. So this is a question about customer profiles and whether they are assumed or not. Customer profile is definitely one of your assumptions as a business. You have two things you probably assume, and you don't have to always assume both. One of them is you assume a problem exists because you have the problem, and then you need to discover who else has that same problem that you have. The other... thing could be is that you know an audience or an industry very well because you work in that industry or you know people in that industry or you just have knowledge of it, but you don't know who in that industry has the problem. So you might be needing to do a wide survey or lots of promotion or marketing to find out who in that industry or space has the same problem you do. In any case, this is going to be probably a survey or a poll, something very, very simple that you can do. And you can do it via LinkedIn, you can do it via groups, you can do it via associations or institutions or organizations where people from that industry or with that job title or in that space work. And you just try to organically network. Again, you're looking for five to 10 people, not hundreds, to be able to use as kind of like a test group or a focus group to give you feedback. If they work in that industry and they actually do sales or they do or they're an expert. they should be able to tell you the kind of segmentation of users or customers in that industry. So you can confirm your assumptions about who the customer is with someone who possibly is in the segment you think wants it. And they can correct you and say, hey, in my experience, actually, the person who buys this product isn't me, it's my boss, or it's the procurement team, or the finance team. So it's just really important to kind of get that early group. So you can kind of not assume all the answers to these questions and you can get other people's feedback and opinions it can still be wrong but it's much better and much lower risk then butter asks about the red lines and we get that fixed so i'll skip that one uh then we have adrian uh how would you suggest getting exposure to landing pages for pre-developed products so this one is great this is not an easy answer but it's a very important thing for all businesses at this point. Your social media and your general marketing, like online marketing strategy, is going to be the biggest driver of first-time user adoption. So let's say you have friends and family, let's say you have connections or networks or people you know in the industry your product is being sold to. You want to go through those channels, because it's free. People are going through referrals, they trust you, they trust your friends or family, your co-workers, your business partners, your friends. And then you can make referrals to you whether they want to join the feedback group I mentioned, or they want to join a survey or a test you're doing. That's one thing. To actually drive it to the landing page, it's mostly just going to be advertising. You can have people do, you know, referrals and posts and groups on your behalf, but to drive a large number of traffic, You need to segment and pick a demographic, so age, location, language, sometimes gender or job title or industry that you want ads to go to. And if you're choosing an audience that is an actual match to the problem you're trying to solve, they should be converting between 4% and let's say 6%. On the high level, 15%. On the low level, 15%. I have done businesses where we do Instagram advertising because it's some of the cheapest. Instagram, Facebook are some of the cheapest advertising on the internet. You can do 15 bucks and get 3,000 page traffic within five days. And that kind of thing is really helpful if it's an organically digital product. If it's a physical product, you need to do sales. Then you need to do cold calling and things like this. But for landing pages, I recommend just doing direct-to-consumer ads if it's a public product. If it's to get sales leads, then you can do a landing page that has like a signup form. And that signup form can directly kind of put them in a queue to be called by you or a sales team. But you essentially just need to figure out who your audience is and do a mixture of advertising and SEO so that people can organically find the page or they can non-organically be paid to drive traffic to that page. Awesome. Thanks a lot. I assume it's the same for like early, early launch of products, not necessarily pre-developed ones. Yes, it'll be the same problem, let's say. There's different products I recommend if you're earlier or late stage. Like if you're later stage and you're close to launching, there's things like Product Hunt. And there's like, you can even look up online startup launch landing page platforms. There's things like Product Hunt where there's companies that organize lists of recently launched startups. And if your product is launching within the next 90 days, they will add you to that list or newsletter and they will drive traffic for you. Otherwise, you have to do it yourself. Awesome. Thanks a lot. Ronald, how do you handle situations where user feedback? during tests contradicts the original problem definition or insights gathered in the empathize phase. This is why I have job security. So customers and people generally contradict themselves on a regular basis. And this is also why I mentioned that five or 10 person rule. There's generally going to be in a group of five, at least one. So in a group of 10, probably at least two people who have very different opinions than everyone else. this is how you get all of those research studies on like toothpaste or like skincare products they're like nine in ten dermatologists recommend because that one person said no this is a terrible product so you're kind of you're you want to look for those one or two people who very much disagree or don't align to everyone else those are ideal people to have in your test groups because they are vocalizing uh a problem that many hundreds of thousands of people will also have So you want to understand why they have this problem. What is the solution? Are they just not the right customer for you? Or are they part of the customer group that you need to build a separate product for, separate business plan, a higher tier? So it's really important to investigate where this kind of contradiction comes from. If it's in the way the question was asked, the way the product was designed, then you solve it. If it's a genuine difference in preference, opinion, or need, then maybe they are a different persona and they actually need a different product that answers your question i think for me or say we need to brief the answer because we have uh so we can go to a may's question next in which phase should early customers be involved Okay, for sure. So I'll actually go back to one of the slides I have because I think it's helpful for a visual. Okay, so keeping in mind that you want to involve customers at all stages. So if you're able to, if you're working in a product that's not private or sensitive and your customer isn't super private, like a government or something like this, you want to involve them as often as possible in the process, sometimes multiple times. Sometimes what I've done is I've built customer user groups and we actually use the same user group at multiple stages. So we say, hey, today we're doing a study on the problem space we're working on. We want to get your idea as an. an expert or a member of this industry or a person who is like a senior in this industry, can you answer a survey? Then in that survey, we'll say, can we include you in future surveys? Are you open for us calling you to ask you your opinions and go deeper on your questions? And so anything like that would allow them to be contacted multiple times in multiple phases. Maybe what this happens, this does is we give that person a discount if they become one of our first customers. Or maybe they get a recurring discount for every single month that they participate. They get a lower price that month. Things like this. But it's actually a really, really big part of Airbnb, Microsoft, Tesla, a lot of the larger companies that release annual products do this. Because they need the same questions asked every quarter or every year. Because they have long-term products that they end up releasing new versions of on a regular basis. So instead of going back to doing these large surveys with random new people, they just keep a repeated group of people involved in the development process. So that's one kind of answer for the first question. Second question is you're selling a service that's not a product. So doing the testing, you need to test the service against real customers. You're wondering if you should service to them a normal price, reduced price, or totally free. This is a great question. So for service-based testing, it is much harder than product testing. In a service test, you have hard costs and you have soft costs. So if you can ever give something away cost, a service like a workshop, like consulting, whatever it may be, you probably are going to try to get it to be as close to at cost as possible so that you're not losing any money. But if you have a very high value, high cost product or service, say service, you might want to go negative at the beginning in terms of like. spending money or doing things that don't scale, as they say, or giving services away for free to a very valuable or very important partner or possible first customer as a way to build goodwill in a relationship because you're actually investing in the future very large sale that comes from that. It's up to you to determine kind of like how much expense and loss you can take when it comes to how many of these you do. But this is super common, let's say, in medicine. Like in medicine, they lose money for the first eight years. For a single product to be built, they do hundreds of trials and they fail trials and they fail all these different tests just to get one new drug. Same thing happens for cars. They may have three or four car ideas. They have concepts. They manufacture a couple of demos, functioning cars. They only ever build one of them. And so there's a lot of waste, but they give those cars away. or they give like the older version of cars away to like government, things like this. So there's ways to kind of recoup the cost. Law firms, consulting firms oftentimes have reduced prices for a long-term customer or when a product or service is new and they don't know yet the value the market will spend on that new service, they give it away at a discount as an introductory price. And then anyone who is in that kind of Founders Club, First Time Users Club, Early Adopters Club gets maybe grandfathered into that early adopter price, but then they increase the public retail price once they have validated. So it's just a matter of timing for you. If you can get the validation with one or two customers, maybe you secretly offer a lower price. If you need to get many customers in, maybe you publicly offer a lower price, but with a very obvious window of time. limited time offering or first six months free, something that actually communicates that you're doing this. But it could be a great way to get lots of customers in, get good feedback at a lower entry level. So maybe it's easier to get validation faster at a lower price. Thank you, Kevin. Great. So Mohammed, you mentioned the example of Tesla selling a prototype in the market with a buffer of two years until the actual product to be released. Would this approach be applicable for an app? This approach could be applicable for an app. It depends on the service that the app is. So I would say things that are, I think of an example of a real product. Okay, this is great. Apple has Siri and all of these AI tools are like a great example of this. A lot of these products have been in beta for multiple years. So Apple has been unhappy with Siri for about five years. I don't really know how long or why it took them this long to build a new version of Siri. They have essentially given it away for free in every single product because they don't know. They know for a fact that they couldn't sell it for money. So they couldn't charge people a monthly subscription to have access to it. So it was a free product. Now, OpenAI has been building their version of like an outsourceable chat GPT for about two years. They knew for the longest time the customer was going to be Apple and companies like Apple. They did a bidding war on this product. What they did is they made the product free with a threshold. It's very possible to do this for apps, but there needs to be some level of teaser or like beta experience because they need to imagine what the product would be like a physical product. It's easy to tell someone to wait one or two years, manufacturing, shipping, legal, trademark. But for digital products, the threshold is much lower. So two years would be a very, very long time in tech to wait for a product. I would say you could probably get away with six months, especially if you're early stage, you're building. This is the same model you will see on websites that sell, what's the word I'm looking for? Like. like Kickstarter, like a Kickstarter program. It's like crowdfunding programs. You'll see people spend money today, maybe 80 to 100 euros, 200 euros on a product, even digital that won't come out until next year. And this is kickstarter.com. Kickstarter.com's whole platform is this and apps are totally okay with this. But I think you have to be very transparent about the progress and the milestones you're making as you go from idea. to development the same way car companies do. During the two years, they didn't just take money and disappear. They were showing prototypes, they were showing improvement, they were showing development of the product. And so you have to do the same thing for the app. You have to be able to release an alpha, a beta, maybe some demos, maybe do some webinars and show the product and answer questions to the public. So don't just use the development process as a stealth mode. If you're going to... pre-take payment and pre-take account setups, make sure that you're definitely engaging with those people more than the public because now they have some expectations. Okay so I think we'll take the last question now from Paris. Is it better for B2B organizations such as solution-propped platforms to start with chunks of service and lose some competitive edge or weight? and test it all. So I think the question is like whether like to start small with some features or like go with the full solution from day one. Yeah so this is it's very specific to the product. I'm going to do a screen share again so I can show you. So all MVPs have this problem, like the idea that a customer has in their head of their problem. is very large, but it usually isn't as large as the founder of a company. If you think about what Airbnb does today, or what Uber does today, or what Amazon does today, no one would have ever imagined that was going to happen. So the MVP is always a micro or teeny version of what the vision or the mission is. So I would not wait until everything is built to release. I would think of When do you think you have released a single chunk of value? So can you check off a job to be done or a problem statement? Can you make one thing better? Once that one thing can be better, sell it. Wonderful. Thank you, Kevin. That was a very enriching session today. And thanks for answering all the questions. very enriching and hopefully we will get in touch very soon in other sessions as well and till we meet next thank you all and for the rest who couldn't answer your questions we will be covering keep those hold of these questions we will answer them during the mentoring session thank you all take care goodbye thank you thank you thank you all thank you bye Bye. Grace to you, Kevin. Many thanks.