one of the toughest type of pm interview questions are trade-off questions like how would you decide between launching x versus why in a previous video i came up with a framework to help you stand out in these type of questions and so many of you asked for more examples and you know i always deliver so today we're gonna go through another example question hey guys i'm diana and i'm a senior product manager at a tech company in silicon valley and i bring you the best tips to get into product management and teach you how to succeed once you've made it so the framework i'm going to go through today is an even updated version from the one i shared last time it's a six part step starting with first the mission the goal and the hypothesis of the feature we're talking about and here you want to talk about how the goal of the product is interconnected with the mission of the company and you also want to share the hypothesis of why you think we're considering between these hypotheses in helping the product accomplish its goal second you want to share some success metrics defining this product and some of these will be used in your analysis for the a b test third you want to talk about the hypothesized impact that x versus y is going to have on your metrics number four you want to talk about other options that you might consider testing in the a b test number five then you want to design your a b test and number six you want to analyze the metrics to help you come up with a ship no ship retest decision if you want an in-depth explanation of this framework take a look at the last video where i went deeper into the framework to explain each section so with that let's get started on today's example how would you decide between showing instagram stories for 24 hours or 48 hours so let's jump into the framework so first the mission goal and hypothesis the mission of instagram is to capture and share the world's moments and stories lowers the barrier to sharing the world's moments because stories lives on the feed temporarily and hence people feel less pressure to post high fidelity content now my hypothesis on why we're considering 48 hours because today instagram stories lives on the feed for 24 hours is the longer content stays on the platform the more time it has to allow for someone to engage with it hence this might increase the engagement per story and that might possibly get content creators to actually create more content if they feel like their content is light and because stories live on the platform for longer there's more supply in stories which might cause people to spend more time on instagram stories next we're going to go into metrics so what are some top line metrics we can think that are important for stories my number one metric is the number of stories that have at least one engagement per week and for stories an engagement counts as a reaction or a dm and this metric is important because it represents the connection that happens from the content this is the intersection when content creators and consumers intersect and connect another one of my top line metrics for stories is the average time spent on stories per weekly active user and that's to represent that people are spending a good chunk of their time on stories when we think about metrics for stories we want to think about both sides of the ecosystem which means content creators and consumers one of the first creator side metrics i want to measure is the total number of weekly active craters on stories per week over the number of weekly active users on stories because that represents the participation rate of creators on the platform and we might want to see if this will go up given the 48 hours versus 24 hours a second metric on the creation side is the average number of stories posted per week because in order to grow the supply side we can either increase the number of people posting stories or the number of stories each of them post and again this might go up if content creators feel like their stories are getting a ton of engagement now on the consumer side what would you measure again if i wanted to increase my top line goal which was the number of stories that have at least one engagement the levers that i could pull and i'm thinking about the funnel is first the number of stories impressions which tells me how many stories people are seeing available on their home screen so imagine i can actually make the stories each take up less space so people would see that there's more story supply and that might lead to more clicks on stories another metric is a number of stories that are clicked on or you might even measure the click-through rate to tell you how relevant the stories seem to the user the third thing under consumer metrics is a number of stories watched and i might define watched as something seen for more than three seconds and then even further down the funnel is a number of stories that have at least one engagement which i mentioned is our top line metric already now we also want to check whether this new considerate feature of having instagram stories for 48 hours is going to have a good impact on the instagram ecosystem overall so some things i might want to measure to learn if that's the case is count the average time spent per weekly active user and i might want to measure the number of weekly active users on the platform have you learned something new so far if so like the video and subscribe to the channel because next week we're gonna have even more videos telling you how to ace the product manager interview now let's dive into our third step which is the hypothesized impact on these metrics first i want to narrow down the most important metrics that are going to impact my decision on a ship no ship for this new feature we're considering so the four would be number one or top line metric which is the number of stories that have at least one engagement per week secondly is the average time spent on stories per weekly active user thirdly i also want to see the average time spent per weekly active instagram user is going up and fourth i want to measure the number of weekly active users now let's jump into the hypothesized impact on some of the metrics we talked about so i'm only going to do this for the key metrics that i think we're going to move with 48 hours versus 24 hours so the first one being the number of stories with at least one engagement and i hypothesize that this metric is going to go up because the longer content stays on the platform the more opportunity it has to get engagement number three i think average time spent per weekly active story user is also going to go up because again more supply and more content on the platform means there'll be more stories for people to view and engage with and this might lead to an increase in my fourth metric which is the average time spent on instagram per weekly active user because if they're spending more time on stories per week then as long as that doesn't cannibalize other features they're using time spent overall for instagram would go up we could also see an increase in weekly active users on stories now let's talk about variations that i might consider testing beyond the 24 versus 48 hours i may consider testing another option where users can actually choose between 24 versus 48 hours versus us choosing for them one versus the other and this option might still default to 48 hours but allows users to choose 24 hours if they prefer that and this might be important to help us cater for those power users who are already posting frequently and they don't want their content to stay on the platform for too long because they want to show people fresh content and we want to make sure we keep these power users because they drive a huge percentage of the content so with that let's go into designing our a b test it's important to test our hypotheses and an a b test is a great way to do that and a b test can help us measure the incremental impact and use data to decide what to ship versus not so in this case the control would be the 24 hours which is what stories is today option number one is to show stories for 48 hours and option two is the default stories two four to eight hours but give users the option to choose 24 hours and how i would make sure this a b test is successful is i would work with our data scientists and our engineers to first make sure we log all the metrics for unbiased results i want to make sure there's enough randomization and for statistical significance we'll want to make sure there's enough people in the test to power the test and i would ask my data scientists to do a power calculation beforehand to make sure we have enough people in the test and run it for a long enough time to make sure it's that sig i'd monitor the results and wait to see the impact level off before i make a conclusive analysis then sure my test isn't being impacted by novelty effects okay lastly i want to go through the different scenarios where i would ship no ship or retest so let's say we get the results and which options should be shipped i would look for the option where we're seeing the highest incremental impact on the key metrics that identified and make sure there's no major regressions on other metrics in a no ship decision that would mean my test options actually show a negative impact on my key metrics say for some reason the 48 hour option actually led to less engagement per story because maybe people are overwhelmed by the number of stories i would also not ship any of the options if the test options led to a regression on some really important metric maybe something like ads views or if it significantly cannibalized time spent on instagram feed i would re-run the test if we didn't get statistically significant results or if we got neutral results on some of the key metrics but saw that for the other metrics they were directionally positive okay guys and that's how i would end a trade-off question so again here's a summary of the framework that i use to answer these type of questions and we just went through an example question some bonus tips for you is to prioritize the key metrics that are going to impact your decision and the second tip is to show that you can design an a b test without it being biased or not statistically significant so we just ran through another example of a trade-off question if you're wondering how to answer questions like how do i set goals for x product or what would you do if you saw why percent decrease in this metric take a look at these two other videos that tackle this type of question and if you want to make sure you don't miss any of these videos to help you ace the product manager interview you'll want to subscribe to the channel and make sure to like this video thanks guys