Annie Duke on “right” decisions, when to quit, and hedging your bets.
Annie Duke is an author, corporate speaker, and consultant in the decision-making space. Annie’s latest book, How to Decide: Simple Tools for Making Better Choices, is available now from Portfolio, a Penguin Random House imprint. Her previous book, Thinking in Bets, is a national bestseller. As a former professional poker player, Annie won more than $4 million in tournament poker before retiring from the game in 2012. Prior to becoming a professional player, Annie was awarded a National Science Foundation Fellowship to study Cognitive Psychology at the University of Pennsylvania.
Annie is the co-founder of The Alliance for Decision Education, a non-profit whose mission is to improve lives by empowering students through decision skills education. She is also a member of the National Board of After-School All-Stars and the Board of Directors of the Franklin Institute. In 2020, she joined the board of the Renew Democracy Initiative.
Main-1
Alex: [00:00:00] Annie Duke is an author corporate speaker and consultants in the decision making space. Annie's latest book, How to Decide: Simple Tools for Making Better Choices is available now from Portfolio, a Penguin Random House imprint. Her previous book, Thinking in Bets is a national bestseller. As a former professional poker player, Annie won more than $4 million in tournament poker before retiring from the game in 2012. Prior to becoming a professional player, Annie was awarded a national science foundation fellowship to study cognitive psychology at the university of Pennsylvania. Annie's the co-founder of the Alliance for Decision Education, a nonprofit whose mission is to improve lives by empowering students through decision skills education.
She's also a member of the national board of after-school all-stars and the board of directors of the Franklin Institute. In 2020, she joined the board of the renew democracy initiative. We had a great conversation spanning quote on quote right decisions when to quit and hedging your bets.
Hope you enjoy.
Okay. So before we get into making bets, what shouldn't you bet on? Cause I heard a story about your brother and brother-in-law making a market for whether or not you and your eventual husband were going to get married, which I think would be a little strange for most people. But it sounds like maybe it runs in the family, right?
Annie: [00:01:21] Yeah. Nobody's brought up that story for a long time.
shold you not bet on? Well, here's the deal betting is a way to reveal preferences. So you're revealing preferences and you're revealing uncertainty. So it's a little bit like, not so much what should you not bet on, but who should you not reveal those things to? So even for me and my husband, like if I were to say, like, what's the probability we get divorced and he made a market where he was like, well, I'll lay you three to two, we're going to get divorced.
Like maybe, you know, I mean, I should know that, but maybe he could express that in a different way by saying like, you know, I'm a little unhappy. Maybe we should go to marriage counseling and not like express that in the most blunt way.
Alex: [00:02:15] Right.
Annie: [00:02:16] So I think it has a little bit to like to do with, like, what is the circumstances?
And what's the value of allowing the other person to know what your preferences are or how certain or uncertain you are. In the case of my brother I think for most people, maybe that would be a little bit of a sacred cow, but, you know, I mean, my response when I heard what the market was, was that my brother, who was the buyer, meaning that he, he thought we were a higher probability than I think that-- I think they came up with 23. So he felt that more than 23% of the time we were going to get married. And I told him he was an idiot. I mean, I think, I don't think we had, yes. So we had not been on a date yet. We had just determined that we were going to go on a date and I was like, what?
How could you possibly make a market that's 23? That's nuts. So that's kind of that thing of like, who are you doing it with? So from my perspective, I wasn't like, I didn't see that as him saying, I'm betting against you getting married. I was like, oh, you're just really trying to think of you guys are having some fun thinking about what the probability of us getting married is.
So I think it has to do with like, who are you doing it with and what is the value to revealing your preferences in that way?
Alex: [00:03:38] Got it. And if we're thinking in bets, we're pretty much going to have to quantify everything around us, right? Is there anything you personally maybe refuse to quantify or think about in that way?
Annie: [00:03:48] No. I mean, no. I mean, I think that it's always helpful to, and, and the thing is that what's interesting is that, so this is a discussion I have with my clients all the time where I say, I will say to them, you need to make a forecast and forecast what you think the probability of this is. And they'll say, well, how could I know?
And I say to them, but when you make a decision, it's implied. Even if you think you're not making a decision, which isn't, there's no such thing because you're still making a decision. It's always implied.
And I try to explain to them, like, you might think that you're not quantifying. How likely this person is if you hire them to embezzle from you. Right? Like, not that I would have them necessarily forecast that in particular, like I would forecast a broader category, but I'm saying, and they would be like, well, how could I possibly know?
And it's like, but you're taking that into account when you hire that person. That you're deciding that the probability of them being some sort of unethical person is low enough that you would be willing to hire them, right? That, that there, whatever your judgment of how ethical they are is above the threshold at which you'd be willing to hire them.
So if you want to tell me, like, you don't know enough about them to determine that, well, of course that's true, but implied in there is that you have there's some range, which is within the range of what would be tolerable. So you ought to just say it. Because, because the thing is you're acting on it anyway, that's the thing, like every decision you make, you're already acting on it regardless.
So if you make it explicit, it's better because then at least you have to think about it and justify it and sort of realize what, what is implied in this decision. It can help you to get to a better place.
Alex: [00:05:27] Right. And, you know, thinking of decisions, something I think about a lot is getting decisions, quote-on-quote right.
I mean, in my mind, being able to make the right decision, every single time is pretty much the definition of utopia . Do you buy that there's such thing as a right decision?
Annie: [00:05:50] So that's hard, most decisions. Aren't mathematical formulas, two plus two is four. I can tell you whether you're right or wrong.
Alex: [00:05:59] Right.
Annie: [00:05:59] It's hard to say what the right decision is because a few things, one is. You know, almost every decision you make has a subjective quality. And even that's, even if you're like trading quantitative strategies, because firstly the, the model that you decide to lay on top of the data. It's going to have some subjective qualities.
It's going to be sort of your opinion on what, what the variables are there. And you can obviously sort of test that and so on and so forth, but there is going to be a subjective quality to that. And then even if you have the exact same model as somebody else, when you look at what that model is, outputting, you might decide to do different things because you have different preferences.
So you have different risk tolerance, you know, whatever, like there's no matter what, like even the most quantitative decision generally is going to have a subjective quality to it. So once you get into that subjective world, it's just difficult to say what a right decision is what-- I can tell you, whether you went through a good process.
But I can't tell you if it's right, because it turned out well, I know that. I can't tell you what's wrong because it turned out poorly, even if, or turned out wrong in the long run. It doesn't mean it was the right decision in the sense that, like, for example, if you're thinking about poker and over the long run, I make a lot of money.
It doesn't mean I'm making the best decision. So if I say that right is compared to the best possible judgment that I could make, I'm rarely ever going to be right. So there was a problem of, I was, I showed preferences. So what is right for me might be different than what's right for you. And then we have this other problem of, we're always trying to forecast what our preferences are, but a lot of times we need the world to unfold and in order to reveal our preferences to us.
So. We have this intuition that we actually know ourselves really well, but, you know, I mean, I think we've all had that experience of being like, for example, like I know that my preference is going to be a small liberal arts college and then you get there and you hate it. I mean, this happened to my son, my son really, really like, he was very sure that his preference was going to be a really big school where the social life was really on campus, strong Greek life and sports teams.
Alex: [00:08:20] Sure.
Annie: [00:08:21] So he chose Syracuse, so he did four years, but it, it became very clear to him pretty quickly that, that actually wasn't the thing that he would like, he never ended up joining a frat. He didn't, he liked pledged he got offered, but then he realized he didn't really want to do it.
And so sometimes like those things, but so then the question becomes, it's sort of. It gets into philosophy or what do you mean by, right? Right. Was it not the right decision for him to go there? Well, based on the information that he had. And the prep and the exploration that he could do of his preferences and try to explore over those next four years, what he would enjoy or like, you know, I mean, he went through a thoughtful process.
It was revealed to him in the end that those weren't his preferences. So does that make the decision wrong? I mean, I think this, this is where it gets really hard. Right? So then you just have to get in, I think you have to roll it back and say, look, I have to think about the process and. Am I going to a reasonable process?
Am I getting enough outside view or outside input? Am I trying to forecast into the future as much as I can and recog-- you know, to try to imagine How things might unfold and actually react to that. You know, there's a whole bunch of different tools you can do in order to get to a best place. But in the end, what we're all shooting for is like, I'd like to get as close as possible to the best possible judgment.
But I recognize that the gap between what I would do, if I were omniscient is always going to be pretty big between what I actually do. And the question is just, am I sort of trying to push myself closer to that?
Alex: [00:09:59] Got it. So when it comes to actually making decisions, I mean, you lay out your thoughts pretty clearly, right?
You have these three groups, you have preferences, payoffs and probabilities. What's like the breakdown that you give people who maybe are totally otherwise uninformed. How do you sum up those three categories to people?
Annie: [00:10:16] Kind of the way I just did, right. What I say is the first thing is you have to know, you have to sort of think about what are the different options that I have available to me. And if I were to think about those options, what are the possibilities for how those might turn out? So if I speed on the highway, there's some possibility I get a ticket. There's some possibility, nothing happens. could get in an accident and so forth. And you're just sort of figuring out, like, what are those different things that could happen that go exactly the speed limit. What are those things that could happen?
The, this is going to be so much similar, less the speeding ticket. Then you figure out. Okay. What are my preferences? Like compared to my goals and then I have to think about what my goals are. And this is something really interesting is then decision-making people often don't think explicitly about what their goals or their values are.
That's kind of implicit and they don't actually surface that in an explicit way, which makes preferences hard. Because what you have to actually ask about the different outcomes is how much is it going to advance me toward my goals or cause me to retreat away from them. And, you know, the options that you have available to you are somewhat going to be determined by what your values are.
Right? So are you Machiavellian or are you not right. Kind of thing. I mean, it's in the simplest terms, but so you have to figure that out and then once you've done that, you have to figure out how likely it is. So if you take the, should I speed or should I go the speed limit, obviously if you, if you take like, oh, I could get in an accident, well, that's going to be true in either case.
So you have to have some forecast of what's the probability that I might get in an accident, so you can understand the difference. And then what you're doing is you're saying for the thing that I'm trying to accomplish, when I look at first, what the downside is, right? What are the things that I really don't want to have happen?
Do I have tolerance for how likely those are to occur combined with how bad they are? So that's what I'm trying to get people to do. And then on the upside, It's you know, is there enough upside to this? In other words, are there enough good things that can come from this, that this is worth my while and that's going to be a balance.
So you can imagine, you know, you could make a decision where you just have a really high likely hood of really good stuff happening to you. But the bad thing in there is death. Okay. So in that particular case, even though the preponderance of the outcomes is really good stuff. If there's enough of a chance that you die, you may not actually be willing to make that decision.
So it's not just like, there's more good stuff than bad stuff, or the good stuff is more probable. Right. You have to think about like how, how bad or good are those outcomes. And then you can flip that for startups. Right? If I do a startup, I've got, you know, mostly there's going to be bad outcomes that occur from that, but the good outcome is like really, really good, even though it's small.
So in that particular case, certain people might be willing to do that. And so that's why you have to be really thinking these things through. And it's not just like, oh, I'm mostly going to have good stuff happen to me. It's like, okay, that's fine. But then you have to compare that to what are the bad things that might happen and how likely are those to occur.
And can you tolerate those things and vice versa.
Alex: [00:13:33] Maybe a more practical question for people now, speaking rationally. When does it make sense to quit?
Annie: [00:13:41] That's my new book. I'm going to give you the technical answer.
When the thing you're doing has negative expected value, you should quit. So that's one. And the other is even when the thing you're doing has positive expected value, if it's costing you the opportunity to be able to do something that would be even more positive for you, you should quit. So to put it into regular terms, when you end up in a losing endeavor, Particularly one that is taking your time and exposing you to risk.
You ought to recognize that you're in a losing endeavor, even though there's some possibility it might work out. If it's not going to work out enough of the time, we need to recognize that and move on to something else. And even if it is going to work enough at the time that the, I think you're doing it's worth the time and effort that you're putting into it and the money that you're putting into it, there may be other things that are more fruitful.
So let me give you a couple of examples. So this would be an example of bad quitting behavior. And then I'll give you a side-by-side example of very good quitting behavior. So, okay, so let's just take Everest, right?
So, what we see on Everest a lot in 1996 would be a good example of this. As people die on the top of the mountain. And very often when they die, particularly in what was documented in into thin air, there's lots and lots and lots of evidence that's coming your way that says, by the way, you should turn around, not the least of which is that they actually set turnaround times.
And those turnaround times are meant to say, if you haven't gotten to where you need to go by this time, then beyond that moment in time, you're now in a failing endeavor. And those turnaround times were ignored by a lot of people on that mountain. And they're ignored by them a lot.
And what happens is that we, you know, we sort of think about this grit problem, right? Like you have to be gritty to succeed, but there's a wheel that, you know, grit and quit are the exact same decision. Right. Cause it's always like, should I stay or should I go at any moment? So what happens though, is that once we're in, once we're in it, we are very unlikely to let go and you can see why that would be right, because we're deciding under uncertainty, right? So we don't know for sure what the outcome is going to be. And obviously if you get to the edge of death, you know, you didn't have a choice, but to turn around. The problem is that then you're already at the edge of death. You're already like 95% dead if you turn around, say at the moment that the turnaround time comes, usually you have like a full tank.
And it's light out and you know, nothing particularly bad is happening to you. It's just that somebody who, you know, people who are in more rational mind and have said, you ought to turn around at this time. And you can see this in all sorts of things, this inability of us to give up on things once we're kind of in it, particularly once we're in the losses that we want to it's like, we want to know that there's no other, there's nothing else we could do.
Because otherwise, like, we feel like we've lost, what's what we've already put into it. So that would be like a sunk cost problem or we're failures and other people are going to judge us. So we're going to judge ourselves or, you know, or we overestimate the chances this is actually going to work.
And what ends up happening is that we end up on the top of Everest dead or very close to it. Right. Even though when, you know, when you're reading that story, You can see like, well, obviously they should have quit, right? Because it's like, there's bad weather and it's too late in the day and people are running out of oxygen and you're like, why are you not turning around?
But it's not really the human tendency, interestingly enough, to turn around in those cities. So you can compare that to you know, and again, I want to say like this counterfactual problem is a really big part of that, right? Like, That woulda, coulda, shoulda, like if I had just kept going, maybe it would've worked out, maybe I could have succeeded, you know?
And the problem is that we sort of lose sight of first of all, the appropriate time horizon. Right. Which is okay. But if I just keep going in this moment, what opportunities is that likely to take away from me later? But then also we also lose sort of the breadth, right. Which is at any given time, like if you're in a particular job, By being in that job, you're giving up the opportunity to be in other jobs.
But like, you want to make it work. Right. And I don't think anybody has ever quit a job and said, oh, I should have stuck it out longer. Like by the time people will actually quit. It's always like, why didn't I do that six months earlier? I'm so much happier. So the best time to quit is the first time it crosses your mind, honestly.
And that's, that's particularly, it's particularly important for me just to add a distinction in which is the best time to quit is when it first crosses your mind under circumstances where things aren't going the best. Now, the reason that I say that is because when things are going really, really well we'll often be too quick to quit.
So there's just an imbalance there. But when things are going poorly, the first time it crosses your mind, it's usually correct to quit. So, and here here's the other thing about quitting that's really important is we have the intuition that when the weather gets bad and it gets late and we're past the turnaround time that we're going to pay attention and we're going to turn around, like, I think that's the intuition that we all have.
Right. Well, I'm going to go in this job. And then if my boss is a jerk, obviously I'm not going to stay there. Right. You know, if I get in this marriage and my spouse starts cheating on me in a situation where we have no arrangement that that's okay, so it's a betrayal, obviously I wouldn't stay, I wouldn't stand for it. You know, but then you do, you stay in the job, you stick it out, you know, you stay, you stay in the marriage and you know, that kind of thing.
So you know, and I'm not talking about like, I think most people would say like, well, if it happened, you know, but you go, you do it past the point at which you would have found intolerable. If you were not in it, if you were not in this situation, if you were looking at a friend, would you, what would you be advising your friend?
Do you act on that for yourself? And the answer is mostly no, right? It's why people will run marathons and on the 24th mile, their leg will shatter. Right? Because they just get very good at like ignoring those things that are saying, Hey, maybe you want to run another marathon later. So it's, it's in those situations where things are going poorly.
That we really ignore the signals that say, you know, maybe you ought to quit. And the problem with that is that, you know, as I say in how to decide like quitting, it's one of our best tools for dealing with uncertainty, because to the point of the start of this conversation is a decision right or wrong--
like you're trying to get close. I mean, that's sort of like, that's the whole point of like good decision making is like there some. You know, if I were omniscient and I had a time machine, here's what the best judgment would be. And I'm trying to get closer. And what you recognize is that because you're deciding where under conditions of uncertainty in a world with luck, that there's going to be new information that reveals itself after that.
And what allows us to still decide, even though we're uncertain, is that we can change our minds. The problem is that when the world gives us those signals that we ought to change our mind we're crap at listening.
Alex: [00:21:22] Definitely.
Annie: [00:21:23] So this is what we're sort of trying to resolve. And that's what I say. Like, whenever you quit, it's going to be earlier than generally it's earlier than you actually do it.
Alex: [00:21:33] And, you know, I guess that brings us to biases, right? Like a big one that comes to mind that I've seen quite a bit throughout college is sort of an inside view versus an outside view thing when it comes to people in bad relationships, you sort of talked about this before, like everyone around them will tell them, the painfully obvious truth, right? That they're sort of in this toxic environment, they should get out of it. But there's this ability to delude yourself otherwise. This is like just one example, maybe an extreme one, but what are, what are like steps you can take to break free of that?
Annie: [00:22:00] First is find someone who will tell you the truth. what that means is that you have to find someone who cares about you enough to tell you the truth. So the issue is like, to your point, when you're all in college and you see the person in the toxic relationship, are you telling them, do you know what you're not telling them?
Cause like, if they stay with them, that's really bad. Cause you're, you're just like, oh, you called the person that I'm with an asshole and now I'm getting married to them. Like nobody wants to take that risk on.
Alex: [00:22:30] [Laughs] Yeah.
Annie: [00:22:31] So you, as an individual, have to seek someone out who will tell you the truth and you'll have to give them permission to do so.
So that's like the first, most important thing is you have to be able to recruit people who will tell you what's what, now you can create some of that by not telling them your own thoughts. So, you know, if I want you to tell me whether you think that the major I've chosen is good for me, I probably shouldn't tell you what I'm thinking.
Right? So, like, for example, if I'm thinking about switching majors and I've already decided what I want to switch to, it'd be better if I said to you, Hey Alex, like. Not I'm trying to decide whether this major is correct for me, or if I should switch to a different major, you know, me really well. What do you think?
Right, period. Right. So that's going to allow you to tell me the truth better because you're not going to be disagreeing with what maybe my heart's desire is, is to go get a PhD in philosophy. And you're like, you know, it's really hard to get a job if you do that. Still enough. If you get a BA in philosophy, it's very easy to get a job from that weird thing.
But anyway so, so you have to find that, and then you have to interact with people in a way that also allow, tell you the truth, which is really important. Then you also have to think about how you can act as that person for yourself as much as possible. So the way that you can do that, like in the relationship situation is like, imagine it's a year from now.
Okay. And this relationship has not improved. Like why do I think that's happened? What do I think the probability of that thing occurring is, is there something that I can do about it? You know, it just allows you to kind of see, let's say, you know, a year from now, if I look back on this decision to stay in this relationship and, and forgive the person, how do I think I'm going to feel about it in a year when I look back on it, and that allows you to just sort of like step out of yourself.
It's like when we were talking before and I said, we tend to get the wrong time horizons. Because when we're in it, where we just don't want to let go, because we feel like we failed that we didn't try hard enough, or we didn't give up, you know, we gave up or whatever. Right. But if you can get yourself like it's a year from now, and I've made this decision, like I'm thinking about breaking up with this person as a year from now, and I'm looking back and I decided to stay, how do I think that decision is going to feel to me?
I've decided to. Break up with them. How do I think that decision is going to feel to me, it allows you to be a little bit more of your outside view. And then the other thing that you can do, that's actually quite helpful is you can do some forward-thinking right? Which is like, when you enter into something, whether it's a relationship or a major school or a job, you can kind of say, what are the circumstances?
Like? What are the things that could be happening in this situation? Ought to cause me to reevaluate and leave. And that's actually, that just helps you to sort of be an advisor to yourself. And that allows you to get to the outside view a little bit better, because then a version of you who isn't in it, who holds, who has your values by the way.
Cause it's, you don't stick your head in that. Now it's like giving advice to you. So, for example, you might say like maybe, maybe it's really important to you to be in a monogamous relationship. If that's true, then you would say, well, if my partner who had agreed to be in a monogamous relationship with me, but you know, then decide, you know, when and cheated on a bunch of, on me with a bunch of people, then I would leave, right?
So maybe write that down or maybe you do the reverse and it's like, well, I want to have an open relationship. So if that's the case, and then it turns out that my partner is really, really jealous. Right. So then it becomes unsustainable. This thing that I would like, then maybe you leave. Right? I mean that, in other words, I'm not saying one value is good over the other.
I'm saying you figure out what it is that you would like to, to occur and then figure out what the things are that could occur. That would cause that to break and do that in advance. And that allow, that allows you to get to the outside view better also.
Alex: [00:26:36] So, what about hedges? Like, to use the marriage example, right? There seems to be this problem where people think hedging, your bets is equivalent to like predicting something will fail. Like prenups are probably a good example of this, right?
Annie: [00:26:49] Yeah. Basically. So when you do this, kind of, let me imagine how this might fail or, and you're trying to figure out what the probability is.
When you see some of those things that have to do with luck, like the things that weren't in your control that can cause you to fail, or some of the things that are, you can kind of look ahead and basically say, well, what can I mitigate the impact of that? And this another, this brings us to another thing about the outside view is that part of the view is what generally happens in the situation that I'm in.
So that's called basic little comfort base rates, right? Like what's the average outcome here. How often does this occur? So that helped us get a sense of how probable it is. And if we go back to sort of probabilities. And possibilities and preferences basically. Then what, what you're doing is saying, okay, something's going to occur at a great enough probability that I feel like I want to get ahead of it.
And the base rates are going to help you understand what the probability is. So in the case of marriage, I think we're at a, I think right now it's about 40% of marriages end in divorce. I think it's gone down recently a little bit, although I don't know, during COVID it might've gone back up. I'm not sure.
But so you, like, you can understand that. And then basically what you do is you say, okay, there's some probability that is higher than what I can tolerate, that something could happen. And, and the effects of that are quite bad. And so therefore I'm going to mitigate that by putting something in place that will mitigate the downside.
So the classic example is having like insurance on your house. So there's some probability your house burns down. So you get fire insurance. Now what's interesting is with something like that, I don't think that people think that you cause the fire by having fire insurance. I think part of the reason for that is that it doesn't get into this like places where emotion, where like magical thinking happens.
But most people have insurance. I mean, working companies make you have it, which probably helps. But they have it. And the, and the thing about insurance, that kind of thing, like a hedge, just that you're, you're kind of crossing your fingers that you never actually have to exercise that option. But so, but you're paying for it.
So but when you get into things like marriage, where it's a very similar hedge, there's just 40% of couples get divorced. Who knows what I'm going to be. When I'm 20, I'm going to get a, try my darndest here. But there's some possibility that we can get divorced. So I ought to just put the natural hedge on, which is a prenup people get into this magical thing, I think.
But then, then I'm saying that I don't actually think that, that, you know, we're going to stay married and it's like, no, you're just acknowledging the reality of the world. In the same way that I'm not saying, like, if I get fired insurance, I'm not saying, oh, I don't think that I'm going to live in my house, fire free.
Like this is gonna make me like set fires to the house. So we, we ought to recognize that those are the exact same thing and it's just good to be prepared. That's all. And you're not, you know, th this happens as well with, like, I've seen this with Some like college coaches who will tell their athletes, like don't have a backup plan.
And it's like, that's the most nutty thing that you could ever tell a human being they're doing something that's a very low probability of ever working out. And while it might be good for you. As the person who just wants every person on the team to dedicate every minute to what they're doing, it's terrible for the individuals.
Ian would argue. It does make them less likely to succeed. That they'd go and get good grades while they're going along the way. But we have this idea that somehow like, oh, if you have a backup plan, you're not going to try as hard. And then what ends up happening is that when, when it doesn't work out and your house is burned down, you're left with a burned down house.
So we've just really need to get out of that mindset because it's, it really is magical thinking and it causes us to be very ill-prepared. And what it does is the worst thing that you can imagine for yourself, which is if you want to have a really successful decision-making you need options. And when people talk about like money makes your life easier, the reason why money makes her life easier is it gives you options.
But you can get options with you can create options without money. So the more options that you have, the better off you are, because as you start to find out that things aren't going well in one direction or another, you have something else that you can go do. So when you get that scholarship to be able to go to college, to be, you know, don't and the coach says, you know, you can't pay attention to anything else.
It's an absurdity because it's closing your options off. So now what happens is that when that low probability thing doesn't work out, you have nothing else to do because you got crap grades. When you don't have a prenup, it's like that limits your options in case you get divorced. If you don't have fire insurance, it limits your options in case your house burns down.
So if we're thinking ahead, we want to make sure that we're always expanding our options, not reducing them.
Alex: [00:31:50] You said you can break down the outcome of your life into two things, luck and the quality of your decisions. So what happens when we stop having to make those decisions ourselves, right? This is the fundamental argument with like AI, but is offloading a net benefit if we get more things right. Or do the lessons of failure matter more?
Annie: [00:32:09] Hmm. Hmm.
So, I mean, algorithms are really helpful for noise reduction. So basically algorithms are really good in two places. One is where. The algorithm is going to improve judgment as an input. In other words, where, what you're saying is that there is the true answer. And by using an algorithm, it's going to reduce.
Your error rate. So now just remember I'm I just want to make this clear algorithm still will have subjective judgments that are inputted into them, and you still have to make a judgment about what the algorithm is outputting. But basically I use algorithms all the time as a way to essentially incorporate the, the outside view and to reduce noise.
So for example, because what we need to remember is like an algorithm would be like a hiring rubric is an algorithm. So it was just basically saying like, here, here are the things at some point that I've decided are the variables that I care about and I'm going to make sure that I'm going to give a rating of those variables each time that's going to output some sort of score.
And then I'm also going to make some judgment based on that score. So so, and obviously algorithms can get. You no deeper than that, but even that is an algorithm. And all that does is it says, at some point I decided what things I cared about. I'm now going to analyze those things in kind of structure, some sort of structured way.
That's going to run the same each time. So that's going to reduce error. It's it may or may not reduce bias. Because algorithms, when they're trained on data that already exists are just going to build in and amplify the bias that might already be in the data. So, as an example, and this is something I think that we sort of think about AI as objective, and we sort of forget this stuff.
So as an example, like Gary Marcus gives a really good example in the book. He has a book called rebooting AI, where he says, like, if you train an algorithm on Google images to identify sisters, It thinks that sisters are only white people and that's just because like, you can imagine when you look at those images, like.
It's, you know, lots of images of white people. And so then that's what it's training on. And so it's going to amplify any of those kinds of biases. So we do need to be cognitive of that. So we have to be judicious and applying it in the right place and understanding what it's really good for and what it's particularly good for is noise reduction.
It just makes your decisions much more consistent. And then we just have to be very careful to be auditing. Those four possibly amplifying bias. So we shouldn't just assume that it's going to it. It's going to reduce bias. And I don't just mean like bias that has to do with, like, in that case, I gave you a racial issue, but it could also just be like a cognitive bias.
Right. So we just want to make sure that we're auditing for that. But where we don't want to really use algorithms though, is like is this art good? Okay.
Right. Like the whole point is like different tastes, you know, different strokes for different folks. Like you should allow that to happen. Now you could imagine, you know, obviously even for matters of taste, you can have recommenders. I mean, that's what that's what Amazon does and that's fine. It's a decision support tool, but you know, is AI ever going to make our decisions for us?
I don't think so. I mean, not anytime soon.
Alex: [00:35:46] Awesome. Well, it was great to meet you, you so much for the time. Really appreciate it.
Bye bye.