[Music] in ancient Greece when anyone from slaves soldiers poets and politicians needed to make a big decision on life's most important questions like should I get married or should we embark on this Vo voage or should our army Advance into this territory they all consulted the Oracle so this is how it worked you would bring her a question and you would get on your knees and then she would go into this trance it would take a couple of days and then eventually she would come out of it giving you her predictions as your answer from the Oracle bones of ancient China to ancient Greece to Mayan calendars people have craved for prophecy in order to find out what's going to happen next and that's because we all want to make the right decision we don't want to miss something the future is scary so it's much nicer knowing that we can make a decision with some Assurance of the outcome well we have a new Oracle and its name is Big Data or we call it Watson or deep learning or neuron Nets and these are the kind of questions we ask of our Oracle now like uh you know what what's the most efficient way to ship these phones from China to Sweden or what are the odds of my child being born with a genetic disorder or what are the sales volume You can predict for this product I have a dog her name is l and she hates the rain and I have tried everything to untrain her but because I have failed at this I also have to consult an oracle called Dark SK every time before we go on a walk for very accurate weather predictions in the next 10 minutes so sweet so because of all of this our Oracle is a $122 billion industry now despite the size of this industry the returns are surprisingly low investing in Big Data is easy but using it is hard over 73% of Big Data projects aren't even profitable this is a humongous number when I first found out I was I was really shocked and paler one of the largest Big Data companies their clients like MX Koke and nasc they're not renewing their contracts because they're saying hey we're not seeing enough results and I have Executives coming up to me saying we're experiencing the same thing we invested in some big data system and our employees aren't making better decisions and they're certainly not coming up with more breakthrough ideas so this is all really interesting to me because I'm a technology ethnographer I study and I advise companies on the patterns of how people use technology and one of my interest areas is data data are to me what stars are to astronauts I mean data is practically its own language we communicate through data we make decisions with data and that's why people say we live in the age of datadriven decisionmaking but it is pretty cool that we live in an age where we can get all this feedback from our activity trackers who affordable genetic testing so why is having more data not helping us make better decisions especially for companies who have all these resources to invest in these Big Data Systems why isn't it getting any easier for them so I've witnessed the struggle firsthand in 2009 I started a research position with Nokia and at the time Nokia was one of thear largest cell phone companies in the world dominating Emerging Markets like China Mexico and India all places where I had done a lot of research on how low-income people use technology and I spent a lot of extra time in China getting to know the informal economy so I did things like working as a street vendor selling dumplings to construction workers or I did field work spending nights and days in internet cafes hanging out with Chinese youth so I can understand how they were using game games and mobile phones and using it between moving from the rural areas to the cities and through all of this qualitative evidence that I was gathering I was starting to see so clearly that a big change was about to happen among low-income Chinese people even though they were surrounded by advertisements for luxury products like fancy toilets who wouldn't want one and apartments and cars through my conversations with them I found out that the ads that actually enticed them the most were the ones for iPhones promising them this entry into this high-tech life and even when I was living with them in urban slums like this one I saw people investing over half of their monthly income into buying a phone and increasingly they were shanai which were our affordable knockoffs of iPhones and other brands they're very usable does the job and after years of living with migrants and working with them and just you know really doing everything that they were doing I started piecing all these data points together you know from the things that were seem like random like me selling dumplings to the things that were more obvious like you know tracking how much they were spending on their cell phone bills and I was able to create this much more holistic picture of what was happening and that's when I started to realize that even the poorest in China would want a smartphone and that they would do almost anything to get their hands on one and you have to keep in mind um iPhones had just come out it was 2009 so this was like eight years ago and Androids had just started looking like iPhones and a lot of very smart and realistic people said those smartphones that's just a fad right who wants to carry around these heavy things where batteries drain quickly and they break every time you drop them but I had a lot of data and I was very confident about my insights so I was very excited to share them with Nokia but Nokia was not convinced because it wasn't Big Data they said we have millions of data points and we don't see any indicators of anyone wanting to buy a smartphone and your data set of 100 oh as diverse as it is it's too weak for us to even take seriously and I said NOK you're right of course you wouldn't see this because you know you're sending out surveys assuming that people don't know what a smartphone is so of course you're not going to get any data back about people wanting to buy a smartphone in two years your surveys your methods have been designed to optimize an existing business model and I'm looking at these emergent human dynamics that haven't happened yet we're you know looking outside of market dynamics so that we can get ahead of it well you know what happens to Nokia their business fell off a cliff this this is the cost of missing something noia was so busy looking for the right data to fit their models that they never even bothered asking the right questions not everything valuable is measurable and for Nokia it was just it was inconceivable to them that people who had always paid a certain price point for something could just all of a sudden changed their behavior it was it was unfathomable but nokki is not alone I see organizations throwing out data all the time because it didn't come from a Quant model or it didn't doesn't fit in one but it's not big data's fault it's the way we use Big Data it's our responsibility big data's reputation for Success comes from quantifying very specific environments like uh electricity power grids or delivery Logistics or genetic code when we're quantifying in systems that are more or less contained big data does a very good job at giving us a very accurate view of the world enough for us to be able to make good predictions based off of it like once you know how much electricity a factory is consuming you can make projections off of that but not all systems are as neatly contained when you're quantifying in systems that are more dynamic especially systems that involve human beings forces are complex and unpredictable and these are things that we don't know how to model so well now this distinction between quantifying in contained systems versus Dynamic systems it's really important to understand and this Nuance is captured very well in this tweet by my favorite astrophysicist or I think he's everyone's favorite astrophysicist Neil degrass Tyson where he says in science when human behavior and the equation things go nonlinear and that's why physics is easy and sociology is hard it's true so that's why quantification and dynamic systems creates this really super interesting Paradox Big Data doesn't just create more knowledge it also creates more unknowns once you predict something about human behavior new factors emerge because conditions are constantly changing that's why it's a never ending cycle you think you know something and then something unknown enters the picture and that's why just relying on Big Data alone increases the chance that we'll miss something while giving us this illusion that we already know everything and what makes it really hard to see this Paradox and even like wrap our brains around it is that we have this thing that I call the quantification bias which is the unconscious belief of valuing the measurable over the immeasurable and we often maybe experience this at our work or you know we maybe work alongside colleagues who are like this or even our whole entire company may be like this where people become so fixated on that number that they can't see anything outside of it even when you present them evidence right in front of their face and it becomes even harder to see this bias because it's reinforced by Big Data companies that say Hey you know we can quantify everything important for you so that you can find that needle in the hay stack so you can find that single source of Truth your one true answer to everything and this is a metaphor that they literally use in their sales and marketing pitch checks this Hy stack and needle and this is a very appealing message because there's nothing wrong with quantifying it's actually very satisfying I mean I get a great sense of comfort from looking at an Excel spreadsheet you've even very simple ones it's just kind of like yes the formula worked it's all okay everything's under control but the problem is that quantifying is addictive and when we forget that and when we don't have something to kind of keep that in check it's very easy to just throw out data because it can't be expressed as a numerical value it's very easy just to slip into Silver Bullet thinking as if some simple solution existed because this is a great moment of danger for any organization because often times the future we need to predict it isn't in that hay stack but it's that tornado that's bearing down on us outside of the barn there is no greater risk than being blind to the unknown it can cause you to make the wrong decisions it can cause you to miss something big but we don't have to go down this path it turns out that the Oracle of ancient Greece holds the secret key that shows us the path forward now recent geological research has shown that the Temple of Apollo where the most famous Oracle sat was actually built over two earthquake faults and these faults would release these petrochemical fumes from underneath the Earth's crust and the Oracle literally sat right above these faults inhaling enormous amounts of ethylene gas these fissures true it's all true and that's what made her Babble and hallucinate and go into Trans likee State she was high as a kite so how did anyone how did anyone get any useful advice out of her in this state well you see those people surrounding the Oracle you see those people holding her up cuz she's like a little woozy and you see that guy in your left hand side holding the orange notebook well those were the temple guides and they worked hand inand with the Oracle when inquisitors would come and get on their knees that's when the temple gu would get to work because then after they asked their questions they would observe their emotional state and then they would ask them follow-up questions like hey why do you want to know this prophecy who are you what are you going to do with this information and then the temple guides would take this more ethnographic this more qualitative information and interpret the oracle's babblings so the Oracle didn't stand alone and neither should our Big Data Systems now to be clear I'm not saying that Big Data Systems are you know huffing ethyling gas or that they're even giving invalid predictions the total opposite but what I am saying is that in the same way that the Oracle needed her Temple guides Our Big Data Systems need them too they need people like ethnographers and user researchers who can gather what I call thick data this is precious data from like stories emotions and interactions that cannot be Quantified it's the kind of data that I collected for Nokia that comes in in the form of a very small sample size but delivers incredible depth of meaning and what makes it so thick and mey is the experience of understanding the human narrative and that's what helps us see what's missing in our models thick data grounds our business questions and human questions and that's why integrating big and thick data forms a more complete picture big data is able to offer insights at scale and leverage a best of machine intelligence whereas thick data can help us rescue the context loss that comes from making big data usable and they Leverage The Best of human intelligence and when you actually integrate the two that's when things get really fun because then you're no longer just working with data you've already collected you get to also work with data that hasn't been collected you get to ask questions about why why is this happening now when Netflix did this they unlocked a whole new way to transform their business now Netflix is known for their really great recommendation algorithm and they have this $1 million prize for anyone who could improve it and there were winners but Netflix discovered that the improvements were only incremental so to really find out what was going on they hired an ethnographer Grant McCracken to gather thick Data Insights and what he discovered was something that they hadn't seen initially in the quantitative data he discovered that people loved binge watching in fact people didn't even feel guilty about it they enjoyed it so Netflix was like oh this is a new insight so they went to their data science team and they were able to scale the thck data insight into with their quantitative data and once they verified it and validated it Netflix decided to to do something very simple but impactful they said instead of offering the same show from different genres or more of the different shows from similar users will just offer more of the same show we'll just make it easier for you to pinch watch and they didn't stop there they re they did all these things to redesign their entire viewer experience to really encourage bench watching it's why people and Friends disappear for a whole weekends at a time catching up on shows like masters of none this is where I will be this weekend so Netflix by integrating big data and thick data they not only improved their business but they transformed how we consume media and now their stocks are projected to double in the next few years but this isn't just about you know watching more videos or selling more smartphones for some integrating thick Data Insights into the algorithm could mean life or death especially for the marginalized all around the country police departments are using big data for predictive policing to set bond amounts and sentencing recommendations in ways that reinforce existing biases nsa's Skynet machine learning algorithm has possibly aided in the deaths of thousands of civilians in Pakistan from misreading cellular device Med metadata as all of our Lives become more automated from automobil to health insurance or to employment it is likely that all of us will be impacted by the quantification bias now the good news is that we've come a long way from huffing ethylene gas to make predictions we have better tools so let's just use them better let's integrate the Big Data with the thick data let's bring our Temple guides with the oracles and whether the work happens in companies or nonprofits or government or even in the software all of it matters because that means we're collectively committed to making better data better algorithms better outputs and better decisions this is how we'll avoid missing that something [Applause] [Music] [Music] [Applause]