[Music] when I was growing up Baba would tell bedtime stories to my sister and I before going to sleep some nights he would read us a story book or one of Aesop's fables some nights Baba would ask my sister and I to come up with our own stories to share with each other and on other nights my sister and I would insist that Baba tell us some of his own childhood stories from when he was growing up in India and on one of these nights baba recited a nursery rhyme to us in Bengali now in Bengali the nursery rhymes often impart some history or collective wisdom and he told us a myth hon the doula bond which literally translates to an entire sentence in English a good crop of mangoes is a signal for a good yield of rice that's just the first two words are meant on one abundance of tamarind is a warning for an impending flood that's the last two words did the live on now my grandfather was a professor and my grandparents parents were in business in medicine so if you're wondering why my father accounted the phrase to me as a young child nestled in suburban Austin Texas I was wondering the same thing but it turns out that those four words encapsulate a whole history past Don passed down from generation to generation that farming in a rich River Basin my ancestors knew very well the difference between a good monsoon in a disastrous rain and had come up with a heuristic which is essentially a simple empirical rule for making forecasts now today we may call this approach a superstition inconsiderate to be flawed for making decisions today standing up the cusp of the fourth Industrial Revolution in an age of increased automation in big data we may put our faith in New Age tools like machine learning in artificial intelligence to make decisions but I'm not so sure we should so let's take a step back my ancestors form those couplets according to a scientific method they utilize the power of observation in a few parameters to make generalizations they recorded observations on harvest yields and water elevation levels year after year developed hypotheses and tested them allowing for broad deductions now these couplets these heuristics have been passed down in the family for centuries making statistical predictions long before the world knew a subject called artificial intelligence but before we get too far into AI let's take another step back how did we come to make decisions in the way we do today so let's travel back 2 million years almost 2 million years ago our ancient ancestors made a pivotal development in human evolution they began turning their predators into prey by inventing a set of strategies and tactics known as hunting now hunter required our early ancestors the vigilant of their surroundings to utilize the terrain to their advantage it required our early ancestors be agile to adapt quickly and it also required our early ancestors understand the behavior of their prey to recognize their strategies to overcome them this met skills like memory and cognition so it was through their ability to detect patterns that humans realized they didn't need to outrun their prey if they dug a hole big enough and covered it with shrubbery they could trap their prey without running the risk of a counter strike and so the power of observation allowed humans to develop strategies for survival test them out in the wild and sometimes die in the process but nonetheless study their effectiveness hunting led to constant apt Asian and that was ultimately transformative in human development the decisions that worked were those that led to survival and the decisions that didn't were those that caused death humans learned to make decisions that reduced harm today our decisions are increasingly being outsourced to computer algorithms machine learning is heralded as the ultimate tool for disrupting and transforming entire industries but recognize that machine learning also forms generalizations based on observations we call data and light superstitions in other ancient heuristics it forms generalizations based on the date of the algorithm is trained on so if the data is tainted then the learning is flawed and recommendations may be problematic a hunter who gets a heuristic wrong may bear the cost only to him or herself but given the skill of machine learning applications in medical diagnosis financial decisions agricultural planning employment screening the consequences of any single blip are immense Dan Mirvish Huffington Post blogger analyzed stock market data back to 2008 and he found a funny trend whenever Anne Hathaway the actor was in the news Warren Buffett's Berkshire Hathaway stock went up how is this even possible well the culprit as many have identified may be machine learning you see sentiment analysis using AI is a big industry you identify some typical words embedded in textual writing and label the sentences to be either positive or negative and after some training voila AI starts assigning sentiment scores to sentences today's automated stock trading scans through thousands of news articles in a fraction of seconds distilling key sentiment information to make buy versus sell decisions and that's how what's good for Anne Hathaway may end up being good for Berkshire Hathaway though the Hathaway mix-up appears to be innocuous in reality it's highly problematic Anne Hathaway's name was associated with a positive sentiment score thereby producing a positive effect on Berkshire Hathaway stock but on the flip side there are names linked to race or ethnicity that have negative sentiment scores in a Boston University study a I trained on Google News data learned to associate words like boss architect financier with men while words like nurse and receptionist were associated with women built in gender bias effects job applicants and certain names can shut out people from a range of opportunities in 2016 ProPublica reported that a computer program used by US courts called Krampus or correctional offender management profiling for alternative sanctions wrongly Flags black defendants as likely to become repeat offenders at nearly twice the rate as white defendants that's forty five percent to twenty three percent and these are the percentages among both black and white defendants who did not reoffending a two-year period compus systematically predicts black defendants to be riskier than white defendants by almost a factor of two and Krampus makes its decisions based on artificial intelligence which in turn looks at the correlation between attributes and outcomes and makes the best judgments now it's obvious that Krampus is prejudiced but Krampus is also a computer program making decisions based on convict data it has about outcomes in this case compasses data is biased and the algorithm simply reproduces the same prejudice as it claims to be immune from now it's clear that if machines trained on data where the data exhibits bias then machines will also make biased predictions and while these New Age tools of artificial intelligence and machine learning are used to guide our decision-making processes today they're imperfect approximate and mere representations of reality that focus solely on some aspect of a complex world to make reductions statistical correctness alone is not a good measure for guiding decisions it's one thing to say something like eating fish may help reduce your cholesterol we're even if the benefit of reducing cholesterol isn't realized an individual is not irreparably harmed by eating fish it's quite another thing to say I think you should go to prison because we know other people who look just like you who have committed crimes where the harm by association is simply irreparable children and two-parent households are statistically more successful than children from single-parent households yet this statistical measure should not be a basis for job discrimination even though models of association mate may be statistically correct and perhaps more accurate than anything else faced with uncertainty Association alone should not be the basis for making societal decisions there are other principles in play such as innocent until proven guilty which values the protection of individuals over collective ethics should be an integral part of the conversation of any guided decision-making that holds decisions accountable to a standard higher than statistical accuracy AI should not get a pass from ethics simply because it's based on math otherwise AI without ethics can justify discriminating job candidates based on the candidates or parents marital status simply because it's innards are inscrutable the inscrutability of data-driven decisions should not be an excuse for recycling discriminatory practices that society has rejected as unfair every decision must stand up to ethics consider an entirely different decision-making approach that's been a for centuries as an anchor for ethics and morality religion religion service well in many regards however in some instances when religiously guided decisions have needlessly harmed individuals religion has lost its moral anchor and claims to being an effective framework for making decisions so what does this tell us well it suggests that no system of reasoning can survive in the long term without a fundamental concern for fairness and equitability the decision-making frameworks that are most effective are those that pass the test of time and adhere to the shared virtues of mutual trust and harm reduction today's AI systems have much to learn about harm mitigation and ethics if they're to remain for the long term there's no single correct decision-making framework for making choices any decision-making framework whether it be artificial intelligence religion or superstition seems to work as long as it passes the test of time and is beneficial fair and equitable regardless of the logic behind it and that's why couplets in Bengali predicting the intensity of monsoons or phrases in Swedish such as pip are pit bar I apologise for my pronunciation have survived past the Renaissance the Reformation Scientific Revolution industrialization and secularization data-driven decisions at their core are designed to bring efficiency deliver the greatest goods to the greatest numbers at the lowest cost but those decisions also have to serve individuals in a society individuals cannot simply be discarded because they cost too much or rewarded because they cost less data-driven decisions must serve people at their core not simply sub select a group of individuals for endowing privileges and that's why as we examine new decision-making frameworks it's important more than ever that ethics should remain at its core if ethics is ignored then these new age tools of artificial intelligence and data-driven decisions will not survive in the long term in a way that many superstitions and traditional wisdom have there's a proverb in Bengali which translates to have all the facts but also have the wisdom to know which ones to apply the problem isn't with the fact that we don't have enough tools in the toolbox it's knowing which ones we should use thank you [Applause]