Good afternoon everybody. Today, we have Alex Schultz, VP of growth at Facebook, and probably the world expert on growth, as he can vouch from this weekend, the person I go to when I have a really hard growth question. Many people who work at Facebook, many of these founders at Facebook in fact, and very early employees have said hiring Alex has been one of the secret successes of Facebook that is not well understood. You joined almost ten years ago, and I think at that point Facebook had less than 100 million users? >> Yeah. >> Way less than 100 million users. Facebook now has 1.6 billion and Alex has been after the importance of having a great product, maybe the second most important driver of that. >> Having Naomi, these very important people. >> And the team, anyway, thank you very much for coming to speak to us. >> Thank you. I did not pay Sam for that introduction and I would note that Havi is my boss so, hi Havi. But no, Havi and Naomi, Danny, they are these people who've been in the growth team for four years and then people like Blake, and Martin, and Ray and James Wang who set the team up. And it's very nice, but I am a VP on growth, I am not the VP of growth. So you've had an amazing lecture series. I just watched all of them yesterday to make sure I didn't say anything without actually having seen what everybody else said. And it was totally awesome. I loved what D'Angelo had to say, so that lecture, if you weren't here, if you didn't watch it, I strongly recommend you go back and see what Adam D'Angelo said, because I could not emphasize his points on metrics more than enough. And so what I'm going to try and do for the first half is go over some of the points that have been made already and emphasize what I think is most important when you're thinking about growth and what my take is and our take is on the Facebook growth team on those. And then in the second half, try and talk to you a little bit more about tactics that have been particularly useful. The caveat that I think you should take onboard is what Stewart from Slack and Flickr said. Which is to some extent, you do have 100 million monkeys bashing at typewriters, and I am very, very lucky to be on the growth team at Facebook to get to work on a product like Facebook that has the retention that Facebook does. That has everyone in the world wanting to use the product and to say that, I drove it or any of us drove it beyond their being great product market fit is perhaps not true. I can't tell you what the AB test is if we didn't even have a growth team at Facebook. So what matters most to growth? The single most important thing to growth is retention. Adam commented on this in his talk, and he showed you cohorts from some real company. But here is a stylized cohort of how you should look at retention. The number of days from acquisition is on the x axis. On the y axis is the percent of people who has still monthly active. I care a lot about using monthly active, not daily active, not weekly active. There are really good arguments for using daily active that people say all the time. Because it's about intensity of usage. But what really, really matters is that you got skin in the game and people come back to your product at least once a month. And then you build on top of that engagement revenue, whatever your key metrics are which we'll get to in a minute, but firstly, I believe monthly active is the most important thing. The second thing is how do you rally calculate it? So I've got a real graph for revenue in a few slides from Facebook from the first 250 days of our ads products, which I'll show you. But the thing that's really, really important is to say that, it's very easy to calculate this curve even if you have 1,000 or 100 users. And what you do is you look at all of your users on the first day after they signed up. Were they monthly active? Yes, clearly, right? The first 30 days everyone's going to be monthly active. I'd prefer if you defined a month as 28 days, by the way, because multiples of seven is really, really good for these kind of metrics because of weeks. But for the first 30 days, everyone has been active on their first day, on their second day, on their third day as a monthly active user, whether they came back or not. And then what you do is you look at day 31. And obviously, only some of your users will have been active for 31 days. Everyone who signed up a day ago was not active for 31 days. So what percentage of them were active on day 31, what percentage of them were active on day 32? And as you go further and further out in the number of days since registration, the number of people who have been on your service that long declines. Which means that your metric is going to get noisier, right, as you get out to the far right hand side of that curve. But that doesn't mean you can't get a handle of where things flatten out very, very early on with very small numbers of users. The number one criticism people give me when I talk about retention as a metric. Or even when you look at what Adam presented in his presentation earlier is you need lots and lots of users. So you need millions of users to calculate that, that is not true. We did it with thousands of users in the early days of advertisers for Facebook for example. Or you need to have had all of those users on for a very long time. Look at Adam's graph it was multiple years with months, and months, and months. No, you can do it day by day, and you can get to a really good place super, super fast, using this methodology. So what you're looking for in a retention curve? You're looking for it to flat line above zero, not flat line like a heart monitor. You want it to flat line at some line that asymptotes with the x-axis. There are three types of these curves that I've seen. The first type is where asymptotes to the x-axis. Like really, really close, goes down to almost zero. You hear about a lot of those companies. You think about most games, most games are smash hits. How many people use FarmVille now versus how many people used FarmVille at its absolute, please tell me you know what FarmVille is. [LAUGH] Yeah, okay, all right. How many people used farm, I'm feeling old. I had hair when I first did one of these. How many people use FarmVille now versus how many people used to use it? FarmVille was a smash hit, then there's other ones which look like this, where they stabilize out. So you think about Minecraft, right, or World of Warcraft. These games that somehow have network effects and people stay in them for a long time. The other version for this curve I've seen is the ones where they go down, they look like they're going to hit the x-axis and then they re-accelerate and they go up. There's a few different things that can cause that right, so you have this, this overwhelming thing that is driving you down to have zero retention and somehow the numbers go up again. So things that drive that for businesses, and you actually I think mentioned this in your intro. One, network effects, so when all your friends get on Facebook, you're much more likely to use Facebook. When all your friends get on Twitter, you'll more likely use the Twitter, WhatsApp, Messenger. Whichever social service you're talking about, there are network effects that really matter. Network effects also count in marketplaces. When you have enough sellers, you can have enough buyers. When you have enough traffic, you get more sellers. And so as marketplaces go, often what you find is these old cohorts that signed up resurrect and come back to the service. So network effects can drive the curve up after it's flatlined somewhere or is heading towards zero. The second thing that can do this is opening up a new interface. So think about it. A lot of people release their apps first on iOS. I actually loved Diane's talk last week for this, by the point that he made of starting with Nokia, focusing on Android because it's what most of the world use, but in Silicon Valley we typically build our apps first on iOS more often than not. And then when someone releases an Android application, and this was something I saw with Instagram years and years ago. But when you release an Android application, people might have tried it on an iPhone And then traded their iPhone for an Android and no longer be able to access your service. People might've tried your service on a friend's iPhone and then when you add an Android they come back and use your service. So that's another thing that can drive you up again. And then the third thing that I've seen on this curves that drive this curves up again is adding categories, so think about Amazon. There was a really good discussion, was that Stuart earlier or? Someone earlier in this talked about Amazon, and how Amazon started with books because there was this large inventory available of books. But then they added in another category. And then they added in another category over time. And imagine, there will be people who came to Amazon to buy that one book, but then two years down the line Amazon suddenly had kitchenware, I don't know, or fashion. And they came back and then they bought fashion and they brought them more use cases to come back more monthly active. This was true with eBay. We added motors. I remember being at eBay and we had to do all of our reporting on gross merchandise volume with and without motors. And no one had ever believed that you could buy a car online, and eBay did it. And it created a whole new use case for people, many of whom were existing eBay customers, who now had one more category they could shop in. So you want to curve the asymptotes to a line parallel to the x-axis. You can do it with far more limited amounts of data than most people think you can by using the technique I outlined earlier, and I'll show you a real world example in a few slides. And there are three types of curves that I've seen. Curves that asymptote like this, awesome, your company will survive. And how big will it get? We have another question. Curves to go straight to zero. That's okay, you could be in games and your job is to produce one hit after another after another. EA doesn't expect whatever fee for 2012, they still have lots of users now if they've released FIFA 27. So it's okay, there are businesses where hitting the x axis is okay. But you need to know you're in one of those businesses. And then the third that I've seen is the curve going up. And that can be driven by network effects. That can be driven by being available on more platforms. That can be driven by adding categories to your marketplace or service. From this falls out a brilliant way to look at growth, and this guy Danny Ferrante who was at Yahoo. Who was one of the first members of the growth team runs core data science at Facebook, he's totally awesome. He's really the brains behind the operation I'm the mouth. Danny came up or brought this to Facebook, and it's a simple way to look at growth. We were looking at growth as how many users were we growing by week on week? We looked at it on an absolute basis. And looking at it like that we were like okay so we're signing up whatever it was a million a week and we're growing by whatever it was 800,000 a week I don't know. Cool, we're churning 200,000 people. And we never actually looked at it in this way. When we change the way we look at things to say well okay, the net growth is 800,000 a week, the new users are 1 million a week. But what is churn and resurrection? And what we actually found at the time, and I can't remember the absolute numbers so I'll just make some up. But imagine you're churning 1.3 million. So not 200,000, 1.3 million. You're resurrecting 1.1 million. So plus 1 minus 1.3 plus 1.1, you've got net 800,000 in growth. But that 800,000 in growth is being driven, I think I did the math wrong, but 800,000 in growth is being driven by a different order of magnitude from churn and resurrection than you realized. And so that helps you focus on how do I stop people churning? How do I get people to come back to my service who are not coming back to my service right now? So once you have your retention curves, and once you start to have a business with retention. You need to look at your growth accounting, not just how many users are we acquiring. Make sense? Overall the number one thing I focus on for growth, and the one thing that we focus on for growth is retention. So what gets you retained in any given service? Magic Moment. What is the moment when you used Airbnb? What is the moment when used Door Dash, when you used Slack, when you used Facebook, LinkedIn, WhatsApp? Whatever the product, what is that moment when you went, yes, I've got it. Right, on Facebook it's friends. You see that first friend, that person who I've got completely the wrong crowd for this but the person from high school who you lost contact with as you grow older and moved away. But you see that person and you go, wow. I haven't talked to them in years. Let's see what's going on in their life. And they've had a kid and lost their hair too, and you feel better about yourself. That is a magic moment on Facebook. So how quickly can you drive the user to that magic moment. You look at Airbnb, right? They talked about the delight that they give people. They talked about these different screenshots of the stages of the user journey that they've created as a map of the user journey. Getting that first person reserve your house. Staying at that first property, those are our magic moments. eBay, buying your first item from a stranger online. Nowadays that doesn't seem crazy but back when I was working at eBay in the UK, that was revolutionary. And people really, their eyes lit up when you talk to them about it. In Germany they had this great ad, tights by minds that showed the build up of when you are waiting for the auction to finish and view to make your first purchase. That was such an adrenaline endorphin moment. But users loved it. Nowadays, it seems a long time ago. Those magic moments are what you need to look for. So how do you find them? Well you can find them in the non scalable way which is don't they just sound relatively obvious? You can actually think through what is the experience on my product? What do I want people to do? You can ask your first users, okay, you're keeping using my product. Why are you keeping using my, what is the thing that makes you happy to use my product? What was that first moment? It's non scalable but it generally works. When you ask people what is the best thing about Facebook? The first thing they say is connecting with friends and family. In all the research, why do you like Facebook? It helps me connect to my friends and family. So you can do it the non-scalable way or you can do it the scalable way. Look for correlations. So Marcus talked at startup school himself and he said that we focused on getting people to ten friends in 14 days. It's a smooth curve, right? Number of friends on the x axis and percent retained on the y axis. It's not like there's a step change at ten, if you get nine you don't retain. If you get 11 you do and it all changes at ten. No, it's clearly a smooth curve and we picked a point on the curve. But there's a strong correlation between number of friends you've got in the first 14 days and whether you retain or not. There's a correlation between number of friends in the first day but there are correlations for any product like did you place a bid, did you do a buy it now or did you list an item on eBay? Was the thing that was completely correlated with retention for eBay. You can imagine, I don't know, I mean DoorDash was founded at Stanford, right? You could imagine for DoorDash. You've placed your first order and that order arrives to your door. And whether you place an order or not and how many orders you place you place in the first week, probably they are actually correlated with whether you retain or not. So you need to look for the magic moment of your product by talking to users and then try and validate that with correlations. Makes sense? Cool, I love this. I put it in every presentation I could. So I'm lucky enough to have been trained in physics at Cambridge in the UK. And my favorite and my favorite lecture series, not my favorite lecture but my favorite lecture series was in this thing called dimensional reasoning. And so it's easy for me to stand up here and say, okay, you need to get flat retention and then you have a viable business. That's not actually true that's kind of a glib statement. If you're producing a social network and you have 5% retention you're not going to long term have a tremendous social network. Or a messaging app like what use would WhatsApp be if only 5% of your friends actually used WhatsApp who signed up for it. I mean it would be useless, right? And there's no way we could release 1 billion plus numbers of active users if that was true for us. On the other hand, if you have a marketplace that is selling fashion. And you have 5 or 10% retention but those people spend a lot of money you have a totally viable business. And so the level of retention you need varies dramatically by the category that you're in. Obviously, the higher the better is true in everything. If you can have an Ebay type business model or Amazon type business model with 90% retention. You probably end up bigger than Amazon and that would be incredible. But I think the point I like to make is, you can figure out and I love this as an inspirational story. So Geoffrey Taylor was this British physicist, he was on the edge of the Manhattan Project, he wasn't in the corp, he ended up winning a Noble Prize later for a different area. And the US and Russian governments were releasing these images to say this is how big our bomb is. But they weren't releasing the actual power because that was a closely guarded state secret. And so what Geoffrey Taylor did he used this technique called dimensional reasoning where you look at the units in the metric you were trying to achieve. So joules, right, for energy, it's joules. Kilograms times meters squared over seconds squared, that's what joules decomposes into. In this case, kilograms, what are you going to use? The volume of the air times the density of the air, which handily, is one. So you've now got meters cubed times meters squared over seconds squared. So meters [INAUDIBLE] over seconds times seconds minus 2. The time is on here, the diameter, the radius of the sphere is on here. You just plug the radius in as m, you plug the diameter of the time in as seconds, and you end up getting 25 kilotons as the output. You get 10 to the 14, I think is what it works out as. And it turns out the bomb was 21 kilotons as measured by the US government. He could take one photo and figure out the most closely guarded state secret in the United States at the time. You can figure out what retention you need to have if he can figure that out. >> [LAUGH] >> So here is what I promised, this is an actual curve. I presented a few different events, so I taken off the numbers because I'm worried they're State secrets. But this is from 2007 to 2008. I joined the week we started our ads business at Facebook, our self-service ad business by us. The pink speaks to the right-hand axis, the blue speaks to the left-hand axis. What the blue is is how many advertisers had been in the system at least one day, two days, three days because the x-axis is number of days from acquisition make sense. So the thing has only been around 250 days, so 0 people have been advertisers 251 days ago. And when they turned 51st date and all the people have been active on day 0. And then each of those pink data points is the total revenue achieved from every user who had registered. Who was using the service on their 30th day, or their 31st day divided by how many users signed up 30 or 31 days ago. Not how many were active, right? How many signed up. So total in the cohort, who signed up 30 or 31 days ago. You divide the revenue they made in the day by that number of users. And that gives you that shape of curve that's up there. I stumbled over my words a bit there. I promise you if you look over it again it makes perfect sense. I did this with Danny back in 2008. The really important things I want you to take away from looking at this curve is, look how flat the pink line is. Like I've been talking about curves flattening out, this is an actual curve flattening out. And look how quickly it's flattened out. By 100 days you can pretty much predict what the line is going to be at 200 days. I've seen this over and over. I've seen this with engagement. So if you divide time spent by the same denominator. I've seen it actually for sentiment. If you have large enough sentiment survey and you divide it by that same denominator. I have seen it for posts. I have seen it for photos uploaded, messages sent. You can use the same techniques that I talked about for determining how much retention you have and whether your business is viable, to figure out what is the lifetime value of one of your customers. If I was to acquire the total addressable market for my business today, what revenue will I be making in 100 days? That's what this curve a allows you to do. You can predict it for posts. You can predict it for page views. Time spent, whatever you want. The same methodology works for engagement, that works for retention. And, actually, retention is often the biggest driver of this metric. And we did this with very little data. Later on, there's a stat we are okay releasing, which was end of 2012, we had about 700,000 active advertisers. Beginning we had about 300,000 active advertisers. And this was five years before that. So you can imagine there was a very small data set that was behind calculating this. The last thing is I said things get noisy the further up you go. You can literally see that on this graph, right? The variance in the numbers the further to the right you go on the graph. So now, how do you operate for growth? Most important thing is have a clear goal. Brian, who is awesome on growth marketing on Facebook, great guy, he always says, if you are a growth team your product is a number. One of the most brilliant things I think that Mark did in the early days of Facebook was he aligned the entire company on monthly active people as our goal. Now why does this matter? It matters for a bunch of reasons, but the biggest and most important reason is I think it aligns everyone. When you have a company of more than two or three people, or by the way, a team of more than two or three people, certainly when it gets to the hundreds. You can't control what anyone does. I mean, you could probably control what one person does, but then the other 199 would do whatever the hell they wanted. So what do you want to do? You want to make sure everyone is pointing in the same direction, and that direction is generally right. So let's think back to the early days of any company. Whether it's Facebook, whether it's Air BnB, LinkedIn, whatever you want to look at. It would be completely reasonable to have a goal of revenue, right? You want your company to stay alive, you want to make enough money that your company can still be viable, that you can employ people, that valuation goes up. So a reasonable person could say, I think we need to drive towards revenue. At the same time, you could have a metric that was like, if you're Air BnB number of hosts. If you're Facebook, number of posts. If your DoorDash number of orders. I don't even know what this company's metrics are beyond Facebook's. You could easily have a production metric or a consumption metric. You could have a time spent in our service metric, and every one of those metrics would be a completely reasonable metric to drive for. So you've got ten reasonable people, maybe five of them think you should have monthly active people. One is really focused on revenue, two are looking at time spent and two are looking at number of posts. Everyone is pushing in a reasonable direction that is vaguely aligned but probably will have conflict. If you want to get ad in front of users as quickly as you possibly can you may not necessarily want to put friends in front of them immediately. If you want to get them to post you may dial up the composer and push down the ads. I don't know. These are trade offs that you could be making. And if the whole team isn't clear what the goal is, they will be in conflict. So having a clear goal that your company looks to drive is really, really important, and having that goal be MAP in my opinion is the best metric that you can drive for. Because it is that once a month visitation, that base level you can drive off of. If you're going to do a consumer internet company. But fundamentally, the most important thing is pick a goal, your team's product is a goal, focus on that goal. Align the company around that goal and then you are going to operate correctly for growth. The other thing I always say is you don't need a growth team. If you're a small company, if you're even series A, don't have a growth team. Don't hire someone like me. The whole company's job should be growth. I'm very happy at Facebook, so I don't want a job offer. But the whole company's goal should be growing. The purpose of a start up is to grow. So, what is the number you want to grow? And align the whole company against growing that number. Does that make sense? Now, Naomi is amazing and Naomi is someone people don't know that well out of Facebook, but she runs product and engineering for growth. She's brilliant, she's been at Facebook 12 years. So, when you say I'm original, I feel not original everyday. She sits like three feet from me, she's amazing. Anyway, she has this approach that we use to iterate through our growth planning cycles, which I think works really well. Understand, first of all understand what is going on. Look at all the data you have, talk to your users. Ask your users what they think. Understand everything that is going on with your product that you can possibly understand. What matters? Retention. Are you churning too many? Do you have the retention? Is the revenue going up on a per user basis? What are the acquisition channels you're getting? Which channel are people resurrecting through? Are they resurrecting through email? Is it push notifications? What's going on? Understand what's going on for your system. Have that all tracked. Then identify, look through all of those different metrics, all of that different research you get in, and identify where there are opportunities. Where do you see big drop-offs? Do you have a lot of people with push notifications turned on who are not receiving push notifications? Do you have a lot of people who are receiving push notifications who are not clicking on them? Do you have loads and loads of social media traffic, but no search traffic? Are your email viral invites working but your SMS ones not, and what is the difference between those? Go and look at the data and identify where there are gaps, and then try and double down on fixing those gaps in the execution phase. Now to this day, this is still how we run growth at Facebook and you can even see a a talk Naomi and I gave online nine years ago I think at FA. Eight or nine years ago, where we talked about using this approach and we have kept it going. Understand what's going on, identify the opportunities in that data you've understood, and then execute against it. Another really interesting thing I think was said earlier is data debt sucks and you're going to regret it later. And your intuition is going to run out at some point and then you're going to want data. In 2009, we actually shut down all of the work the growth team was doing during January. And we only spent the time logging, and making sure every critical flow on Facebook. Registration, at the time we didn't log clicks on emails. So, click some emails, it seems obvious now but we didn't have it. Clicks on the emails, did you then actually log in or did you fail in the password entry. All of these different things, we logged them so that we could understand what was going on. And that was super critical because actually in March that year, we had a massive growth slow down. And we would not have been able to understand what was going on if we hadn't done the logging in January. Because what went on was actually super new ones and without that logging in place, we wouldn't have known. Get the data logged so you can run through this process. So speaking of data, I am very, very lucky at Facebook. I do growth marketing, I've run the awesome internationalization team. But I also run the analytics team for the company and that's been a couple years now. When I talk about data to people internally, I like to say these three points. There's a meme in the valley that you operate either for user experience or for metrics. And I really want to bust that meme. Data when used in the right way gives you empathy about your users. One example I've talked about in the past is we launched a product on Facebook that you may have seen on your big 42 inch monitors. No one has a big 42 inch monitor in here, but I'm sure some of you do at home. That has on the right-hand side all the friends you can chat with on Facebook, but at the top it has this list of stories that's ticking over called ticker. When you're on a small screen, that product used to collapse into the right-hand column of Facebook and push things down so that you could still see these stories ticking over. That product worked great for all of us who had huge numbers of friends who are creating a lot of content on big screens. But if you went to the Philippines where there was huge usage in Internet cafes and people run narrow screens, the majority that amended up having that in the right-hand column. And the majority of them were low engagement with Facebook and couldn't see people you may know below the fold. And so, we actually lost friending amongst users who are very different to ourselves. And we only figured out what was happening by looking at the data and saying, why is this bad on small screens and awesome on big screens? Why is it not so great for low engagement users and super for power users? It was because we used the data to understand these users who are different to us here in Silicon Valley. Now think about it, I presume I don't know. I'm seeing a lot of Apple products here. I have a Lenovo at the moment. But most of you probably have a iPhones. Who has a iPhone? Who has an Android? You're the best. Most people in the world are using Androids. But everyone in this room has an iPhone, first and foremost. You don't understand. I'm actually forgetting how to use iOS now, which actually is a problem in my job. But I've got an Android because the majority, maybe not the majority, a vast number of our users use Android, as Jan said in the last talk. And I need to understand what the Android operating system offers. But I can't understand Android and iOS well as a user at the same time. So, I used data to understand what's going on on iOS. You need to gain empathy for your users. And research is great, and talking to your customers is great. But if you listen to the Twitch guy, Emmit, talk to your users then go away and build the product to serve them. And look at the data, in my opinion, would be the next thing to validate if you built the right thing. If you're doing the right thing, if your research actually ties up with what your users have said to you. So, data gives you empathy. It's not a trade-off between metrics and users. If you build the right metrics, you are optimizing for the users. Two, data predicts the future. Over and over and over again, I love this quote. It's a science fiction outright thing. But like the future is here, it's just not evenly distributed yet. So, those curves that show you the retention. When I said you can look at your total addressable market and say, if I sign them all up, how much revenue would I get? Or you just look at who you signed up today. Let's say back in those days, we signed up a thousand advertisers in a day. I could tell you how much revenue they would contribute hundred days out because of that curve. Now, also you're users are going to ramp up. So, a user who shops in one category if you're eBay or if you're Amazon and then buys in a second category and a third category is clearly going to get more valuable, right? Shopping the shop, driving up same store sales is a common technique in e-commerce. Okay, so how many of your users are increasing the number of categories they're shopping in? You see someone sign up. What likelihood is it that they're going to add extra categories, and what is your revenue going to be in the future? You can predict that with data. AB tests are another great thing. See all these people walking in the science march believing in science. And then when you apply the scientific method to their product and run an AB test, they ship the thing that looked bad in the AB test because they didn't believe the AB test. I see that with all kinds of startups I talk to. AB tests tell you what's going to happen when you ship them to 100%, use them it works. Finally, and this is something Mark actually added recently for me. Data helps you make decisions faster. If you can not as the CEO, have to have a conversation over something where you could just run a test, your company is going to be able to move quicker, because every decision won't have to come to you. People on the team, if they're clear on what that goal is, that north star metric is. They can run the test, see if they moved the metric and then only come to you when they know if they've moved the metric or not. Decisions like that help you move faster. Knowing what your most important priority is because you've looked at the data. That helps you move faster. Make sense? Cool. So now you want me to push as much of the time as possible so I'm pushing for as much as time as possible. So let's talk about tactics. First 100 users. Sam talked about this brilliantly. If you actually listen I thought Tracy in the previous conversation was just fantastic. When she talked about getting 30 of her friends in the construction industry to try Plangrid, and then asking them to pay and 29 of them kept paying. Like ask your friends first for your first 100 customers. Get people who will give you honest feedback, and I love the point about make them pay, if it's a paid product. Second, research and then reach out to who you'd like to have as your users. Totally makes sense, right? Like, you can do how Oscar was discovered by the Lob Team right. Just email a bunch of people, don't get caught in a spam filter. Don't just auto generate emails or you'll definitely get caught in a spam filter. But send personal emails, get introductions, see which friend you have in common. I emphasize by the way, I have never start up, so I might be entirely the wrong person to tell you about first 100 users, but Sam asked me to. Third, social media and PR, PR is unscalable, but it can give you amazing bumps. Amazing bumps. And in the early days I think PR is a good growth leader. Later on, I think companies don't realize when PRs stop being a growth lever, because they have these other metrics. But you can get these great one-time bumps. Just know you can't get them every single week, and sometimes they are a one off. And finally, by ads. I am not as against ads as Sam is, as a great method for growing users. I think that buying ads can be absolutely fantastic. I built my career buying ads, two of the most valuable internet companies, Google and Facebook make all their revenue from selling ads like ads are actually a good thing. If you get the targeting right and we're going to talk about that in a second. If you're having a growth team, if you're focused on growth tactics what people talk about is growth hacking, though I hate the term If you pour fuel. This is a picture from a guy called Nick on my team, Nick. I did not tell him to hold his hand into the flames, which you'll see in a second. If you pour lighter fluid onto dead coals with no spark, you get nothing. And if there's one thing to take away from this lecture, you need to have product market fit to drive growth. You need retention to drive growth. Otherwise, every growth tactic, every acquisition tactic you could possibly run doesn't matter. But if you have a spark there. If you have a glowing ember you can pour fuel on the fire of your product and that is what growth tactics does. So I look at things based on channel. Targeting creative and then conversion. So you think about channel, I'm going to do some greatest hits here. If ou look at SEO, you think, I have to think about keyword research. Which keywords do I want to show up for on Google on the other search engines? And make sure you have content targeted against those keywords. You need to get links, links from the internet to your site. If you're a Silicon Valley startup you technically get a lot of links off high value sites so you usually don't have to worry about that. But then you need to link inside your site. Many people will tell you links don't matter anymore, in my experience that's not true. I don't know why it's not true, but it's not been true in my experience. And then finally, there are a bunch of basics out there on SEO, you can look them up. There are a load of websites that talk about them. STO book, MOOs, there are a bunch of websites out there you can read about SCO tactics. Four page search. The biggest thing I see people do wrong is they dont think about incrementality. They by page search and they don't necessarily say, well, would I have got those customers anyway? So look really hard. If you're going to buy page search as a startup you should see a step change in your line. Your marketing team, if you've reached the stage where you have a marketing guy should show you a line that is flat and then they turn on page search and it goes up. That's the same of any ads if you are a small startup. If your a big company incrementality analysis there's this great Berkeley professor Steve Tedelis whose done a deep study in it with Ebay that I love and I think he's really smart. Big companies, you have to be very, very smart about getting incrementally. You're a small startup. Show me a step change. Do think about marginal not absolute ROI. So people always look at I spent this much I got this much. How valuable is it? That's not what you want to look at. You want to look at the last dollar you spent. How much did you get for that? Not overall does that make sense? Not as many knots. I'm going to blast through that. Look at margin ROI, look at diminishing return curves and economics. And then again, you need to do keyword research. So email and SMS push notifications. This is something every startup gets wrong. We always optimize for ourselves. Have you seen the Instagram thing on the internet where someone shows a phone of an Instagram user with a million followers when they posted something and they have push notifications on. And the string just goes [SOUND] and all the notifications are coming in at high speed? That's what we're like for our own products. For our products we are the power users and by nature every start up I have gone and talked to has always optimized for themselves. Has always optimized for the power user. And so by optimizing for the power user, what you do? Well, we shouldn't put down by default because it's going to completely expand my whole screen. Power users are smart. If you give them the options, they know how to turn off notifications. It's your marginal users you should care about with notifications. That person for whom they received their first Like on Instagram or the host that gets their first notification on Airbnb. Or the seller that gets their first bid on Ebay or the buyer who's just been out bid in their first item. That's the person that needs to get the notification. And needs to respond to it, and you need to optimize for in your head. Marginal user is the number one thing in email, SMS, push, notifications, and even offline mail which Rob was talking about and is still very important for many companies. People get wrong, they optimize for themselves. Beyond that you need to make sure it gets delivered, opened, clicked on, etc. There's so much more here and if I push through all of it I'm not going to get to any questions. Do you want me to get to questions? >> This is pretty important. How about this and no questions. >> This and no questions, is everybody okay with that? Everybody for next week. >> So think about on site merchandising. You own a lot of media. There's this idea of earned, bought, and owned media. Does that make sense? So if you post on Facebook and it gets loads of likes you've earned that media, right. If you bought an ad on Facebook or google or somewhere, you've bought that media. And then if they're on your own website, you own that media. People underestimate the value of owned media. If you have a lot of traffic on your site, ask them to invite their friends. Ask them to complete that next action in your product. Think about your on-site merchandising as an extension of your product, but understand that on-site merchandising is a marketing problem. So how can you trigger the right thing? Someone has just bought an item for you, immediately like Amazon does. Trigger for them a set of suggestions like why don't you buy this and this, so the customers who bought this also bought this, works incredibly well. What about creative? Creative is really important. It should be personalized, because they're on your site, you know their name. Hey Alex, why don't you buy these items that other similar customers have bought. Performs way better than buy these other items that similar customers have bought. That's one simple personalization. But instead of just saying this customer bought this item, you should do a prediction. This person, all the items they've ever bought let me predict for them in a personalized way what item they should buy. Look at Facebook and how often we say five of your friends have done x. Social context, that's personalization. Use people's own profile picture. We have a great personalization example where when you're on a Facebook page, we show you what your ad would look like if you bought it, with a Buy This button underneath it. It could not be more personalized. On-site merchandising gives you so many opportunities to deeply, deeply personalize the creative. And finally, paid and organic social. I do work for Facebook, we do make all of are money from ads. And I personally think that I am not the best person to speak about buying ads on Facebook, because I'm the guy who buys ads off Facebook to drive traffic to Facebook. But what I'll say is if I left Facebook today, and I was a start-up I would be crazy not to use our custom audiences and look-alike targeting. So I highly, highly encourage you if you think about nothing else on Facebook ads to look into those. It's a great way for you to say these are my best customers. Upload the list to Facebook. How do I find more customers like those? It works incredibly well for a lot of companies. And the other thing across all social media is to a great extent you're dealing with interesting squared. Because the click-through rate on your thing, the like rate or whatever on your thing will be related to how interesting it is, and also the distribution, whether it's being shared via a messaging app, reshared on Twitter, or on Facebook, will be driven by whether people found it interesting. Make sense? Cool. Okay, targeting. Behavioral is most important. So what you did Is much more important than who you are. There are 50 year old, 60 year old parents and grandparents who don't need two friends to be active on Facebook. And there are some who need 500. And there are teenagers who need 500 friends and some who need two, and you're much better to figure out like what is the behavior that tells you that. Then say on a demographic basis, I'm going to do this versus this. People who buying Nike's, Men's Nike trainers typically are men, but sometimes people buy for their boyfriend, or their husband, or their dad or whatever. And so if you just target women with women's trainers and men with men's trainers you are making a mistake. What you should do is look at what signals you to show the right ad or the right content to the right person at the right time. Targeting matters. I've talked about this before. Okay, creative. Creative is important but it's nowhere near as important as the above two unless you have a 10x creative. If you produce the Dollar Shave Club video, probably it's very very worth it but most people don't produce a piece of creative that amazing that changes the world. Typically I find targeting comes before creative in every single way, and channel comes before that. Does that make sense? So creative is awesome and really important and can give you a big uplift, but think about your channel first and your targeting second. If you have creative though, you should put it in context, you should personalize it, you give it a call to action. On call to action, this is a great example. Where on every ad back in the day on Facebook we had a little link at the top that said advertise. We translated it wrong by mistake in French. We found this with the data in French, we translated to clear an ad. Create an ad. And we had significantly more, actually 40% more acquisition from that channel that was at the time our single most important channel for advertiser acquisition. We took the French translation and we reverse translated it and changed the whole world. Instead of saying the kind of passive advertise to the active create an ad. And we achieved a 40% bump in that most important channel for us for advertiser acquisition. That is the call to action, like what you say matters. So when I say creative doesn't matter as much as channel, and targeting, it still matters a lot. And the words you use matter a lot. This is the example I said about in context with intent. If you are on your own page, this used to be ad we would show to you, that say hey this is what your ad would look like elsewhere on the site. And you can see that by hour this was the hour that we doubled advertiser acquisition for Facebook with the weekly push. If you want to see more about this, Brian Hale does a great talk about this at F8, and it's online. And then the last thing, yes. The last thing is conversion. Conversion is really really important, but it's not important until you've thought about all the other things first. So things to think about with conversion. The landing page people land on will determine what your conversion rate is going to be. If you can, don't have one button on it that they then have to click to go to the registration form. Put the registration form in the landing page. It's important to know what your most important metric is though. Back at eBay, what we had was this original metric confirmed registered users, and a new metric activated confirmed registered users. And an activated confirmed registered user was someone who bought an item, bid on an item, listed an item. And a confirmed registered user was someone who registered and confirmed and that was it. To get confirmed registered users we wanted to pump all of our search traffic, this was back in 2004 we were spending huge amounts on search. We wanted to pump all of our search traffic at the registration page. To get activated confirmed registered users we actually wanted to land it on the search term that you were looking for. So if you were looking for a trampoline, show you the trampolines you could buy on eBay. That change, and we changed all of our advertising to change the landing page, dropped confirmed registered users significantly that we were acquiring for eBay. But it increased activated confirmed registered users. It increased revenue and lifetime value per registered user. And it increased total revenue per day from the people we were acquiring. So although I think you can focus on the registration form really hard. Any commas, sometimes it's really, really important to land people on what they are searching for, so that they will activate and they will buy on your service. So it's not as simple as just put the registration form online. But we did do that on Facebook. So the final thing I like to say is Facebook moves fast. And move fast is an absolute core value for us as a company. Like core, core, core value and we still do bold things like taking messenger out of Facebook, that was challenging at the time. But I think now people are seeing how valuable a decision that was. And that was a project that I was deeply involved with and ran. We're willing to move fast, we're willing to be bold because fundamentally a good plan violently executed on today is better than a perfect plan next week. Data helps you move fast. These growth tactics help you move fast. Best of talks about making decisions with 70% of the data not 90% of the data. And I deeply believe in that. So move fast, focus on retention and have one key metric that you care about for your company's growth. Thank you. >> [APPLAUSE] >> Okay, apparently we have time for one question. I'm sorry. Go for it. >> So you mentioned that [INAUDIBLE]. How do you think that creating the. >> So the question was for the magic moments, you can kind of tell how you can figure out went they happened and that how you can create them. I mean honestly if you look at LinkedIn, Facebook, Twitter every social network in the known universe. What we do is, we immediately ask you to import your contacts. So once we know that friends is the thing that we need to do for you, we focus on what is the way to remove friction from getting you friends. If you look at the example I used with eBay, we hadn't really figured out that activating you was important to generating the revenue long term. So we shifted the landing page and the registration form to the search results. So we got you to the item you wanted to buy as quickly as possible. And that caused you to activate and get to that magic moment faster. So those are the kind of things once you identify it I actually think the tactics fall out very logically. My boss Javi likes to say that common sense is the least common of all senses and I agree with him. But if you step back and just look okay, if I want more people to buy an item on eBay should I land them on the registration page or the search results page? You'd probably land them on the search result page and that would get them to that magic moment faster. >> Thank you. >> Thank you, okay. >> [APPLAUSE]