Cognitive Bias and Visualization with Evanthia Dimara

The data is the closest we can have to an original information source.

Hi, everyone. Welcome to a new episode of Data stories. My name is Enrico Bertini and I'm a professor at NYU in New York City, where I do research in data visualization.

And my name is Moritz Stefaner and I'm an independent designer of data visualizations.

And together we talk about data visualization, analysis, and generally the role data plays in our lives.

And our podcasts that we do together is listener supported. So there are no ads for mattresses or tools for building websites or something like this. So if you do like the show, please consider supporting us on Patreon. You can find our page on patreon.com Datastories.

Yes, and today we have another special guest. We are very happy to have a person finally able to talk about cognitive biases and their relationship with visualization and data analysis in general. And we have Evanthia Dimara. Hi, Evanthia, how are you?

Hi, Enrico and Moritz. Thank you very much for having me here.

Thanks for joining us.

Yeah, very welcome.

And I'm happy to join the Eurovoice. I will be the greek accent.

Yeah, here is another one, very exotic as well. So, Ivan, Tia, can you briefly introduce yourself, tell us a little bit about your background, current position, your interests and so on.

So I just finished my PhD in visualization and decision making, and I started working in Paris as a postdoc in the HCI team of the Institute of Intelligence Systems and Robotics of Sorbonne. I came to Paris to do human computer interaction. And actually, I would say that the way I see GI is like problem solving. So for me, that was the deal, to find what is the problem that people have and they cannot do so far. So the way I see visualizations is that the data is the closest we can have to an original information source. Let's say the open data that we have about unemployment, about many, many, many different things. But as you know better than these numbers are difficult to handle. We have statistics. Statistics don't tell us everything, so we need to know more things. And this is how visualizations come along in these projects. I wanted to see how by visualizing all this information, we have more chances to make good, informed decisions that are based on reliable data. So for me, having reliable data that we understand, so who prevents us from making good decisions? Most of the times I would say it's actually ourselves and the limitations of human judgment which lead us, I guess, to the topic of today, the cognitive biases.

Yeah, I just wanted to say, this sounds great, but it's a little problem there. And it's exactly this translation of. Well, first of all, is data really objective? That's a good question to ask that we discussed many times already on the show, like, how much is data really representing objective truths? Usually. And the second part is classist.

Yeah.

And the second part is, of course, even if we have really good data, let's say, like, high quality data, the question is, do people still do the right thing with it when they see it? And maybe this brings us to the topic of cognitive biases, among other things. So, what are cognitive biases, and what can we do with them?

So, okay, so to explain what is a cognitive bias, I will start with an example. So, let's take, again the voting thing. Like, we want to vote. We have presidential elections. Okay? So we have two candidates. We have Alice and Bob. Alice, let's say she has a very promising healthcare plan for the country, and we give it to that ten out of ten score, a very good healthcare plan. But her policy for crime control is not well thought and organized. So let's give it a five. Okay? Bob, on the other hand, has a very strong policy against crime. We give it ten out of ten. But when it comes to healthcare, his plan is rather weak. So we give it a five. So, if you assume now that health and safety are both important to you, this is not an easy answer. Right? We have a trade off. There is no optimal answer. So if I want now to manipulate your vote and make you vote Bob over a list, this is what I'm gonna do. I will do a marketing trick. I will add a third candidate in the race. Let's call it decoy. So, the decoy candidate is very similar to Bob. He also gives, let's say, more emphasis to safety and health. But he's slightly worse than Bob. So let's give him eight out of ten for the safety and five in health. Now, for some reason, the addition of the decoy option, the probability for you to vote for both instead of a list will mysteriously increase because of this decoy option. And this is an example of a cognitive bias that is called the attraction effect or the decoy effect. So, a cognitive bias is practically, this is a deviation of human judgment from a norm. So, the norm in our case, is a particular rationality principle that says that your choice between Boba than Liz should not depend on any relevant third option. But in principle, you should never want to choose that. It's an irrelevant item, inferior from Bob. But this item can make you choose Bob over a list. So this is considered as an irrational to some extent. I mean, to some norm, any rational decision.

So you could say these are like systematic thinking mistakes we keep making all the time, or like certain traps we keep falling into. Right.

There are controversial on this. What should be considered as norm, what is truly rationality, what's not. But most of the time it has to do with the deviation from a norm. The norm can be from decision making, can be a memory. A social norm can be many, many things. Another example, for example, is like we have the availability bias. Again, the same example of voting. If I ask you what is more likely to happen to you, a terrorist attack or a happy topic. So the probability of a terrorist attack, in fact, is, as you may know, is extremely small, almost 7 million times more likely to die from a heart disease or cancer rather than a terrorist attack. However, if you're exposed to media that they often report incidents of terrorist attacks, and I guess not much report hair diseases because it's not a hot topic. I guess this makes you feel that a terrorist attack is more likely to affect your life. So in this case, a candidate that is focusing on terrorist counteraction, for example, may be more appealing to you.

Isn't there something called like the vividness bias or something like that?

Exactly.

When something is more vivid than in drugs, a lot more.

There are many bias that have to do with familiarity. That's the mere exposure effect. So the more you're exposed to something. Exactly, exactly. That's another case of the bias.

Yeah, there are so many.

Yeah, there's this huge list at Wikipedia, we will link to it. List of cognitive bias. It's like thousands, thousands or hundreds. But as you scan them, you realize, wow, our brains really don't work that well. So we always think like, yeah, we are fairly rational, like plus, minus, give or take. But then you realize, wow, there's so many subtle ways we can be misled very easily.

Yeah, yeah. The confirmation bias that you already search information in the way that confirms what you already believe, or like the clustering illusion that we see patterns and conspiracy theories and connections among random events that don't exist if we want to describe them. In general, we can say that people usually fail to apply probability rules. They're afraid to apply logical rules. They appear overconfident, they rely too much on the first piece of information they see and stuff like that.

Yeah, I think a lot also has to do with some heuristics. Like, we try to follow a certain path that is cheap in terms of thinking, and often it's okay. Ish. But sometimes it's also totally wrong.

Exactly. Sometimes it's okay to do that. And sometimes this is a very effective strategy we have, so it's not always to blame for.

Yeah, because engaging with the full information is much, much. It takes much more effort. Right. So I think that's the thing.

Can I mention two more favorites? So, yeah, go ahead. And it's a great list. You have to read the whole list. Or maybe read a couple each day. It's enough. You can make a calendar with it. But a really good one is rhyme as reason. Effectively, rhyming statements are perceived as more truthful. A famous example being used in the OJ Simpson trial with the defenses use of the phrase, if the gloves don't fit, then you must acquit. And this actually works. You know, you listen to it and you're like, yeah, that makes sense. That works. That works.

It rhymes.

Right. And so immediately you're sort of biased to believe it. It's crazy. And the other really nice one is Ikea effect, the tendency for people to place a disproportionately high value on objects that they partially assembled themselves, such as furniture from Ikea, regardless of the quality of the end result. Coming back to visualizations, it's actually a problem in design. Like, if you spend a lot of work, it was so hard to get all these D3 scripts to work, and finally it works. And then you say, wow, that was a lot of work. Yeah, it's good work.

And then you just have paper prototypes. The less effort, the more.

Yeah, yeah. So the more you invest into something, the more valuable you perceive it, even if the absolute value is not there. And all these. Yeah, and there's so many of these mind traps and thinking fallacies we can fall into. It's crazy.

Yeah. Moritz, you just gave me an idea, because when I talk with my students and I tell them that they have to trash their prototypes, they're always so sad. And I'm gonna tell them, you're just a victim of the AQF.

Exactly.

Or the loss of action.

Also, it was just cardboard and.

Yeah, cheap stuff.

Yeah, exactly. Yeah. So, yeah. And it's hard. And I think it's. Once you learn about confirmation bias, you see it everywhere.

It's not a bias.

Yeah. It's something, if you're not aware of it, you don't see it. But generally with biases, I think it's like if once. Once you have a sense of them and understand a few of them, you start to see how often they occur. And you also learn about, like how arguments, certain arguments are structured and then you see where the trick is, basically, often with politicians, for instance. Right. Like how they set up a certain framing or an anchoring just to put you in an emotional mindset that will actually have you misjudged something. And, yeah, it's kind of crazy.

Even the order on the debate, if you have a debate and you see many politicians, you remember the first and the last, and you forget the ones in the middle. So they fight who is going to go first and who's going to go last in the shows.

Yeah. So a lot of rhetorics and you also said marketing or advertisement, of course, use these tricks. Right. But also the classical, like, skill of delivering a good speech or delivering a good persuading argument is of course, has a lot to do with triggering these, these mind patterns.

Yeah, exactly.

Can you train your brain to not be so bad? Like, can you sort of like, do the biases go away when you know about them? Is it a bit like monsters under.

The telling people about the bias, what the status. So is that simply telling about people about the bias doesn't really work? Sometimes even, doesn't work even immediately, but even after some time, it fades away. Even when you put a warning message that says, don't be biased, biased, it doesn't work. And amazingly, even when you have people who have domain knowledge on those things, like you have intelligent analysts, that they actually take decision about national security things in the United States particularly, watch the study, they compare the intelligent analysts with college students based on how much their decisions were biased. And the college students outperformed.

Oh, my God.

Were more rational than the domain experts in finance. So it's a very complex thing.

Yeah. I think one thing I remember from the classic book from Daniel Kahneman, what is called thinking fast and slow, one thing that stayed with me is the idea that maybe one way to deal with some biases is just to have processes. Right. So don't count on yourself. And there is this story of, I believe he set up a process to evaluate soldiers in the Israeii army. Right. Something like that. I hope I remember it right. I should. And he knows that himself and his team is biased in many different ways. Right. So I think what they did was to just create a set of questions with scores and also guidelines on how to score things before seeing any candidates. Right? Yeah. I found this kind of thing pretty inspiring because how to distance yourself from the decisions that you're going to make in the future by using some kind of processes, I think it's an interesting idea.

Even a blind review we do. I mean it's.

Oh yeah, yeah.

But still, even a blind reviewer can have biases, right? It's like, oh yeah, that's tricky. Yeah. So how does this all relate to data visualization mind or also in your work?

But still, even a blind reviewer can have biases, right? It's like, oh yeah, that's tricky. Yeah. So how does this all relate to data visualization mind or also in your work?

Yeah, so I mean, okay, we understand that all these biases illustrate how human judgment can be distorted, right? So, okay. At the same time, visualizations are designed to rely on human judgment. So in the first phase, it seems reasonable that we should care about these limitations. However, I mean, I guess many people that hear us now and they work in visualizations or data analysis, they will say, okay, all these distortions of jasmine may appear just because people are not very well educated, are not very intelligent, or more importantly, they haven't seen the data yet. And I mean, as we discuss, okay, we expect visualizations are closer to objective knowledge because they access the original data. I'm afraid that I will disappoint you here because we have to address many challenges before we be able to say that visualization can help on alleviating bias or before we even know which biases can occur in which visualization activities. So the first challenge that we already somehow said is that more knowledge does not guarantee unbiased judgment. So we say telling people about the bias doesn't really work. Even domain expertise that we saw doesn't always work. And they also did studies on people like if they are intelligent or open minded, and this didn't even seem to affect the cognitive biases. So since we say that simply knowledge cannot solve the biases, I guess it's not, there is no reason to be super confident that simply visualized data will also do. The second challenge is that the cognitive biases we discuss so far are not really tested with visualized data. So in order to verify the existence of all these biases, you will see many experiments, like people. This must be replicated in a lot of experiments. But if you read those studies, you will see that the people in these experiments are mostly, see texts, oral instructions, everyday objects. Some of them are with data, some of them without data, and we don't really know what is going to happen if we expose people to visualize information. So cure interlacing discusses those things, but we don't really have a lot of.

Empirical and we don't really know which biases apply when.

Yeah, exactly. There are many, as you said, 300.

Yeah, it's literally hundreds. Yeah. So can you test like, or are there like rules or like, do you have to test individually, like, which bias applies in which, for which chart type or, you know, I'm not even sure how to, how to approach this at all.

I can tell you, for example, one example of my project, that we addressed a very particular bias in visualizations. So you remember the first example of a bias I gave you, and bias too. I started with my bias. So I. The one with the elections and Boba Dalis, the attraction effect. So, for example, let's see, let's take only this bias. Now, this particular bias is only. People have, in previous studies, have only seen numerical tables or commercial products like towels and pens. They haven't seen visualized data yet. Okay, so what we did in this study was to see if this particular bias can affect the way people make decisions using a visualization diagram. So we ran a study asking people to choose chimps that are presented in a 2d scatter plot. It's the typical mathematical diagram with the two coordinates. So we asked people to choose a gem according to two attributes, the cleanliness of the gin and the variety of the machines. So one gym was better in cleanliness, and the other one was better in variety, as we saw before. So again, there is no optimal. Here we have, again, a trade off between those two attributes. So let's imagine now how this plot will look like. Okay, the vertical axis was the cleanliness, and the horizontal axis, we have the variety. So in the very upper left corner, we have the clean gym, the one that is better in clean limits. And down on the very right, we have the dot of the gym with a big variety. So what we did is that we also displayed some decoy gyms. So some particular people saw a decoy dot on cleanliness, which means that in the upper part, next to the clean jean, we put another gym slightly inferior, so slightly more down. Some other people saw the decoy on the variety. So they saw again the two dots, I told you. And they saw a third decoy gene close to the variety, so down and a little bit on the left. So what we observe is that we could indeed influence people's choices with these decoys in the scatterplot task. And people were choosing the cleanzime more likely when they see the korium cleanliness, and again, the variety gene when they were seeing the decorum variety. But I mean, okay, I mean, in this first study, we have only three data points, and usually for visualizations, we mostly care about larger datasets, right? So, and actually, this particular bias and many other biases are like that, is only defined and studied with three with three choices. And what I found actually very interesting is that when I read in psychology literature to see what's going on, I wanted to replicate something with more data. Where are the other options? So what psychologists suggested is that the attraction effect may not generalize to more items because people will have to do a lot of pairwise comparisons. They will not do it immediately. So likely that the effect will eliminate, will go away. So I said, okay, but visualizations are really famous for doing these tasks really effectively, like pairwise comparisons, and see many data at the same time and in immediate way. So I said, okay, maybe the attraction effect eliminates for other things. What if accelerates for us? So we did that. This is what we, what we actually observed that when we extended the task for more data points, people were affected by those dense clusters of decoys. So the attractiveness of the optimal poise were affected according to those, let's say, inferior clusters.

So people were relating the extremes to the closest cluster and said like, oh, yeah, this is like much bigger.

Maybe they were seeing many inferiors.

Ten gyms in this area.

Right?

And so they were saying that if.

This point here dominates so many decoys, so many inferior things must be something, it must be better, something like that. And even though they had no reason to, I mean, because we asked them, people after we finished the study, what was your reasoning behind that? They didn't have a particular reasoning. So they didn't say, I did this because I felt they're gonna be a trend. We expect some people may think of something happening there.

No.

They say, no, I'm okay.

It has a lot to do. Like, when I teach, I often make this point that we don't really visualize numbers or individual data points, but we always visualize relationships between things. And this is what also people read. And so they read the relationship of a point to the axis or the frame or the other point, but never directly the pure information, but always this relative judgment. Yeah, yeah, super interesting. And so you were able to prove it exists. It's like a measurable effect, right. In scatter plots.

Yeah. But for me, you know, this study, I mean, okay, yeah, we observed an fraction effect when we use scatter plots. But for me, the important message from this work was not only the attraction effect, that, okay, it applies to scatter plots. For me was this, that the people who did this study, the participants, understood the data, understood the scatterproject, they didn't do any mistake. They were choosing one of the two superior options. Right. So in the way we see these right now, in the way we evaluate visualizations, we treat data somehow like a holy grail. So we believe that once people understand the data and these people understood the data immediately draw a right conclusion. So we focus on the limitations of human vision. And the reason?

Just readability, basically. But, yeah, that doesn't get.

Just readability, basically. But, yeah, that doesn't get.

I remember your talk actually, with Jessica Hullman about. About this topic, about. Yeah, okay. Perception is vision. Perception, of course, is super critical. But what happened after.

But, yeah, I mean, if I turn this around, if I'm a bit like, playing devil's advocate, maybe. So doesn't this mean if we turn it around that nothing matters and that.

We can, like, that's a common theme here, right? So we dig this deep hole and then we ask ourselves, should we trash everything?

That's a challenge for the.

What I'm trying to say is, doesn't it mean we can be much more loose in how we display our data if the basic message is right?

Well, the thing is that it's a matter of framing. So since it seems that framing can affect us so much, like the way we frame, a problem can affect us so much, then that can also turn, as in our sleep, because then we can reframe the problem and manipulate the monkeys.

Yeah.

And then precision is good again. Once you have established the right frame and the right way of thinking about something, then precision is good again. Right.

I mean, you as a designer, you can change the framing. So if a problem is framed in a way that confuses people and make them certain in a certain way, if you change this design and you change the framing, you know, maybe you can make.

But I think it's a good, like, case to think about. Like, okay, maybe precision is not always everything and maybe even you can leave out data, or you should leave out data if, you know, if you include it, people will sort of read that part of the graph way too much or, like, over interpret it or establish these wrong rankings or relationship effects. So I think that's a very interesting takeaway here, that the whole point is what people take away from reading the graph, not if you have depicted every single data point. Right.

Exactly. That's a very good point, actually. And it particularly applies to the attraction effect because the truth is that if you remove the decoys, you don't have attraction effect. Okay.

I don't suggest data. Better judgment.

Yeah.

Yeah. It's kind of interesting.

Yeah.

Crazy.

It's kind of a case.

Yeah.

A very interesting find, actually.

Yeah. And so this attraction effect could be fixed with a different. Yeah. Displaying a more, I don't know, working on the distribution of the data points you show, probably, or are there other strategies how you could fix it?

I think we can explore, because I'm working a bit on a project like that. I don't have yet enough evidence to say for sure, but it seems that, I mean, the way you interact with the data, as you say, can affect the way you see those decoys. So for example, if you treat them in a way, for example, different design variations. So even if you display those decoys, but you somehow hide them or de highlight them or make people interact with them in another pattern, maybe likely people will see them as less important. Because the problem is here that for some reason you consider the decoys as important and you shouldn't. So I think this is particularly for the attraction effect. I think this is a good, could be an approach, but you know, we have so many biases that I mean, yeah, but I think that because, you know, they say that one good reason why biases occur is information overload. This is what makes us need to have a shortcut, need to rely on the most pronounced part of the information. So if we do what you say, like we try to find a way to manage this information. So instead of displaying everything to find a way to help people to manage the overload and clean the decision space and organize it better, this, I think it can be a very promising thing to do.

Would you also say user testing could be a good way in general, like to find out if your visualization method is, is like very prone to specific biases?

Thank you very much for this question. Actually, it was not prepared. That's why I thank you so generously, because actually this was my thought. I mean, okay, somehow you remember this list that you saw in Wikipedia, right? It was so big. And I'm saying to myself, okay, I would like somehow to make people test those things in visualization, right? So this is my second project. So I wanted to encourage people do that, like to see which of those biases can affect and then try to think of design variation. That was my sneaky plan. So what we did in this project is that I wanted to take those biases and do a taxonomy based on tasks and then based on this task to encourage somehow a designer to try to test them. Visualization. So let's say imagine for example, you are a visualization designer and you work in a company that makes tools for choice support. Okay. Okay. You want to present your data very well. You are aware that people make decisions, can be subject to biases and once you go to the Wikipedia page and you see all these hundreds of biases here somehow. Okay, what do I do now? I mean, how do I handle all this? So the taxonomies we see right now on biases rely a lot on explanatory theories. So we have, let's say we divide biases according to what we think that people think when they do the mistake. So let's say we say that there are false memory associations. It's a formation overload. But all these explanations, I mean, what should a designer do about that? It's not easy to grasp and test those concepts. So the idea behind this project was to base them on task. Let's say one of the tasks in this taxonomy is the decision task. So some biases involve experimental tasks that people have to make a decision. And then I hope that this can be a way of thinking of them. Instead of trying to understand all this complexity around psychology theories that it's hard for us to grasp, maybe we can think of them. Just as this is a task, this is an arm, this is a deviation, we test it, we put some imagination to think it in a visualization context, and we move on. I mean, we do a test.

Yeah. So basically you are creating a sort of translation of the existing taxonomies for people working in visualization to better understand how existing cognitive biases may translate in situations where people are using visualization to make decisions.

Right. That could be super.

Try to avoid the psychology. I love it. Personally, I have this thing that I love to read about psychology stuff, but I guess that everybody wants to dive into this.

Yeah. And you need some pragmatic way of approaching it because it's such an overwhelming, like. Yeah. Multitude of potential biases out there. So some checklist or some like, decision tree approach could be super helpful. Yeah, that sounds good. Yeah. Cool.

Yeah, I was just saying that I think another aspect that I find really interesting in your work is that you seem to focus a lot on decision making. And this made me think that, surprisingly, there is not a lot of work on how visualization works when it's used for the specific purpose of making decisions, but we know that it can be really powerful.

Sounds like a very fundamental use of data.

Right, right. Yes. That always amazes me. Right. It's like, why don't we actually, because we always talk about the use of visualization for presentation, the use of visualization in data science, the use of visualization in blah, blah, blah, using visualizations for.

Getting famous on Twitter. So that sounds exactly right. I mean, you need to start somewhere.

You know what you say now? The painful story of my life. Because, you know, my PhD started with a title that involved group decision making.

Yeah.

And combination of visualization and automated approach. That was my first title.

Yeah, that's the whole thing.

So my thought was this, that in visualization, of course, the decision making is super addressed so far. So I need to go one step up.

Right.

That was my initial thought in the. Before I start.

No, but it's not at all.

Then I started, okay. It's always in the introduction of every good book and visualization. So you always say we do decision making. Yes, yes, we assist. This is okay. But then when I dive into the experimental empirical approach is what we really test. I didn't find any, almost any decision making task.

Yeah. Nobody.

Of course, I saw many, I mean, hypothesis testing and many important things, but didn't see any people choosing stuff. That was one thing. So then I narrowed down the PhD, decision making information visualization, not. Yeah. So that was it.

Perfect. Yeah. I think we have to wrap up soon. I think just one last bias song.

We need to listen to.

Yeah, we should listen to the bias song before we conclude.

Oh, yeah.

I just wanted to ask you. I think so, for people who are listening, what are your suggestions in terms of. I would say a couple of things. One is if they want to know more about cognitive biases and how they may influence the work of people who are working in visualization or with data in general. And. Yeah. And what to do in practice. Right. So say I am a visualization designer and I listened to this episode. So what should I do next? Right.

So actually I would say that, first of all, what I believe should be the view of view when it comes to cognitive biases, in the way I thinking of it. Exactly. So sometimes, you know, the research is not a linear process. Always. Right. So sometimes what we do is that we have an idea about a cool design that we really like, and once we have everything done, we want to find a problem. So we say, okay, now what do I do with it? What people cannot do or what tasks should people do? So one way of seeing those biases and those huge lists that you see in Wikipedia is to see them as a list of open research, small problems, that they have a way to measure them, they have a way to. There are well identified problems. So if you're stuck and you don't know, what would be your next problem that you can address? You can take a cheat sheet, which is the cognitive bias case, and say, okay, these are the things that people cannot do. They cannot escape from their initial hypothesis. They cannot. They are affected by. For example, I give you an example of this, the simplest I can think of. We know from biases that order matters. Right? What comes first, what comes next? There are some particular contexts that the order matters and affects the way you decide. So if I was a visualization designer, I would say, okay, what if I put a circular layout? Let's say that the order is not particularly perceived in a certain way. Could this de-bias those people or I cause other effects or something? So for me, this is a way to see the whole. These are well identified problems.

Just be aware that some things, very superficial things, might lead people in a certain direction. And just assume that all the time. Yeah, that's true. Yeah. Cool. I think we should wrap it up. Fantastic topic. It's one of the most fascinating psychological topics, I think. I hope we were able to excite a few people for it. And if not, there's a song coming up that summarizes everything, so you can learn anything about in three minutes. Yeah. And it's really good. And so at least this one should entertain you. Thanks so much for joining us.

Thank you very much for inviting me. It was great. I had a very good time with you.

Thanks so much.

Thanks so much. Bye bye.

Thank you. Bye bye.

Hindsight bias. I knew it all along. I'm biased because I put you in a category which you may or may not belong. Representativeness biased. Don't stereotype this song. I'm biased because of a small detail that throws off the big picture of a thing. Anchoring bias in the forest for the trees. I'm biased toward the first example that comes to mind. Availability bias. The first thing that comes to mind. Oh, bias. Don't let bias in your life bias. Don't try this. It'll influence your thinking and memories. Don't mess with these. But you're guilty of distorted drinking. Caught enough to buy it, your mind becomes blinded. Decisions and problems, you've been forced to solve them wrong. I'm biased because I'll only listen to what I agree with. Confirmation bias. You never mind if you are this. I'm biased because I take credit for success but no blame for failure. Self deserving bias. My success and your failure. I'm biased when I remember things the way I would have expected them to be. Expectancy biased. False memories are shaped by ease. I'm biased because I think my opinion. Now what's my opinion then? Self consistency bias. But you felt different way back when. O bias, don't let fly in. Bias in your mind. Bias, don't try this. It'll influence your thinking and memories. Don't mess with these. You're guilty of distorted thinking. Cognitive bias. Man becomes blinded decisions and problems you've been forced to solve in my heart.

Hey folks, thanks for listening to data stories again. Before you leave a few last notes, this show is now completely crowdfunded, so you can support us by going on Patreon. That's patreon.com Datastories. And if you can spend a couple of minutes reading us on iTunes, that would be extremely helpful for the show.

And here's also some information on the many ways you can get news directly from us. We are, of course, on twitter@twitter.com. Datastories. We have a Facebook page@Facebook.com. datastoriespodcast all in one word. And we also have a slack channel where you can chat with us directly. And to sign up you can go to our homepage datastory EA s and there is a button at the bottom of the page.

And we also have an email newsletter. So if you want to get news directly into your inbox and be notified whenever we publish an episode, you can go to our home page Datastories es and look for the link you find at the bottom in the footer.

So one last thing we want to tell you is that we love to get in touch with our listeners, especially if you want to suggest a way to improve the show or amazing people you want us to invite or even projects you want us to talk about.

Yeah, absolutely. And don't hesitate to get in touch with us. It's always a great thing to hear from you. So see you next time, and thanks for listening to data stories.