How do you evaluate visualization?

Here we go.

Hi, everyone. This is Errico from data stories together with Moritz again. Hi, Moritz.

Hello.

How are you?

Good, good. Bit jet lagged. I just came back from Texas.

Ah, from Austin. I'm jealous. How was it?

It was good. I was at south by southwest. It's a really huge conference. Lots of speakers, lots of tracks, lots of things. Interactive music, movies. Al Gore was there. Stephen Wolfram gave a keynote. So big thing.

Please tell me you had some German weather.

Yeah, very German spring weather. So I could wear my German t shirts? No, it was quite nice to have some sun. Finally.

Fantastic. Okay, so for today, we decided to have sort of composite program. We have a first section where we will try to answer some of the questions that we received from the listeners. Not all of them, of course, but we have a selection. And then the main topic of the episode will be what makes a good, good visualization and how can we measure it? And by the way, can we measure it? And I think we can directly start with the questions, maybe. Moritz, you want to start?

Fantastic. Okay, so for today, we decided to have sort of composite program. We have a first section where we will try to answer some of the questions that we received from the listeners. Not all of them, of course, but we have a selection. And then the main topic of the episode will be what makes a good, good visualization and how can we measure it? And by the way, can we measure it? And I think we can directly start with the questions, maybe. Moritz, you want to start?

Sure. So for the first question, I mean, we had really a lot of questions. It was really great. But we can see how many we can answer. One was quite interesting from SEC, talking about the, the different competitions. And he said, or he asks if we spend much time looking at the terms and conditions. And so he looked at that Nielsen contest that was recently launched. And there they say, by accepting a prize, each winner agrees that his or her entry will be deemed a work made for hire under the copyright law of the United States. The winner hereby waives in favor of sponsor all rights of droid moral. Is that or is that droit morale or moral rights of authors or any similar rights or principles of law under his entry. Have you found this to be common practice? It makes me want to run and hide from this contest. How does it make you feel? How does it make us feel?

Sure. So for the first question, I mean, we had really a lot of questions. It was really great. But we can see how many we can answer. One was quite interesting from SEC, talking about the, the different competitions. And he said, or he asks if we spend much time looking at the terms and conditions. And so he looked at that Nielsen contest that was recently launched. And there they say, by accepting a prize, each winner agrees that his or her entry will be deemed a work made for hire under the copyright law of the United States. The winner hereby waives in favor of sponsor all rights of droid moral. Is that or is that droit morale or moral rights of authors or any similar rights or principles of law under his entry. Have you found this to be common practice? It makes me want to run and hide from this contest. How does it make you feel? How does it make us feel?

Bad, I think. Right?

Yeah. Yeah. It's awful. That's really awful. Terms and conditions.

I mean, is it common practice?

I mean, in some way, yes. Because whenever you run a platform or a contest, you need some rights on the work because otherwise you couldn't show it on your website. Right. So you will, as a platform owner or somebody who runs a contest, you will always want to have that right to redistribute the work, you know, or copy the work. But what they're asking for here is to waive all copyrights in favor of them. And that's really the worst case for, you know, if you have, you're a creator, and you don't receive money for your work, but still, you have to waive all your rights. I mean, that's really not a very friendly practice. No, really. And it's something to watch out for. He's absolutely right. I didn't look there. Exactly. And if you read these clauses, these extreme clauses in the terms, I would not take part. Really? Yeah.

Okay, what else we have?

Moritz second one is you mentioned that it would be great also by SEC to have more interaction and collaboration between practitioners and academics in this field. Any thoughts on how these types of relationships can be initiated? Short of a match.com for datawiz, that's.

A good one for me. Can I answer to this?

Yes, please.

I like this one because. So, one reason behind having my blog fell in love with data was. Exactly because I thought there was a big divide between practitioners and academics. And I think that both parties have to gain if we find ways to make these people, let these people meet or discuss. Okay. From the one end, I think that practitioners might not know what happens or what happened in the past in academia. And there is the risk, there is always the risk that they will reinvent the wheel. And I have seen that many times. And at the same time, I also know that my fellow academics, some of them don't have any idea about what is happening out of the lab. And my feeling is that some of them don't know how many talented people in industry or just in the design world are creating visualizations that are really stunning. I mean, some things I see are really, really, really stunning, and some of them have a quality that I've never seen in the labs.

Okay, exactly.

How can we initiate that? I think the best way is to let these people meet. They have to know each other. And probably the best way is to try to invite, if we have conferences that are more where the participants are, more from the area of industry or design, let some academics participate. And the other way around. Yeah, absolutely. So, for instance, we have the this week conference that is an academic conference. And I'm always pushing for having more people from industry or designers coming to this conference. Of course, it's not easy, but this could be a first step. And we also have other conferences like the C conference, which is another occasion to let people to invite academics into an environment where normally designers and practitioners belong to.

Right, right, exactly. Yeah, yeah, yeah. The C is interesting. It will be end of April in Wiesbaden in Germany. So if you are based in Europe, that should really be interesting. So there's on Saturday there's the conference organized by Scholten Volkma, and this year I know there will be Manuel Lima and Stephanie Posavec and a few other great speakers. And on Sunday there's always an event where we have a few behind the scenes talks. And last year that was organized by Andrew Vande Moere Benjamin in Vidacare and me, and this year we will have even more people around. And so if you're based in Europe, that should surely be interesting and hopefully also a good contribution to bringing the worlds together. And in my experience, you're right. It's always the smaller meetups where you can talk informally, where you exchange, like your way of working, what you find important in a smaller group. This is where the actual demystifying of these two worlds happens, you know, then you suddenly realize it's not that different. Or at least it's interesting what the other one is doing.

Or maybe it's very different.

Yeah, it's too different. That's fine, too. But at least you understand a bit better.

Yeah, from difference we learn, right?

Yeah, exactly.

Okay, can we move on to the next one? So we have a question from Rio, and he or she asks. I don't know.

Okay, can we move on to the next one? So we have a question from Rio, and he or she asks. I don't know.

That's a good question.

Good question. Let us know. Rio, he asks, how do we look at our work objectively and critically? Probably because of looking at it so long or being attached to their own work. What do you do? What do we do to look at our work objectively?

Yeah, it's a good question. Because if you work on something too long, of course you might be biased or you might have problems having a fresh look at something that is so familiar to you and that you created yourself. And my trick here is to not invest too much work on one track and one work, and put all of my effort into that, but work on a few alternatives in parallel, because then you have a much easier decision. Instead of having to say, is this one thing good, very good, or bad? Then you just need to decide, do I like alternative a better or b better? And if you do that all the time, hopefully the outcome will be quite good. You know, that's the idea behind it. And so this is my tip is not to invest too much work into one single design, but rather invest the same amount of work into three and in the beginning, and then decide what you roll with. Because this is the. This is probably the best way to not fall in love too much with one thing, but be open. Also, design is a lot about leaving stuff out or also deciding against something you cannot include everything in your final design. And so practicing this throwing away stuff or saying no to stuff, it's an important skill, and the more you practice it, the easier it becomes.

Yeah, sure. This reminds me, I think you wrote a chapter in the book, beautiful visualization, where you describe what's the process behind creating a visualization. And in the chapter there is a clear understanding of how you start with an initial design and then you discard it. And I think this is pretty common in visualization design. I experience it all the time. But my feeling is that you cannot see it if you just surf the web and see what visualizations people create, because they just don't explain what's the process behind. But there is normally a very long process and it's sort of, I don't know, you go through lots of trials, right?

Yeah, sure. This reminds me, I think you wrote a chapter in the book, beautiful visualization, where you describe what's the process behind creating a visualization. And in the chapter there is a clear understanding of how you start with an initial design and then you discard it. And I think this is pretty common in visualization design. I experience it all the time. But my feeling is that you cannot see it if you just surf the web and see what visualizations people create, because they just don't explain what's the process behind. But there is normally a very long process and it's sort of, I don't know, you go through lots of trials, right?

Yeah, that's a good point. We can link to the chapter. I think maybe it's a good read. Yeah, sure. Second, he asks, or she, it would be interesting to hear if you guys criticize each other's work and go a bit outside the comfort zone.

Oh, that's easy. Do we have to mention all the eye candy that you produce?

Yeah, and your boring academic explorations.

Is that enough? I think we should do a whole episode like that, right?

Oh, yeah. Interesting idea.

That would be fun.

But the truth is we don't criticize each other's work. But probably we should. Probably we should. No, but we haven't really, like, collaborated in detail on a project where we would say, now show me your design and I criticize it. So we haven't done it then that way. So. Yeah, could be interesting. Yeah.

Yeah. So we have another one from Chuck the nerd. Yeah, I love it.

Pearl tricks, probably.

So, any words of wisdom on how to slowly, cheaply introduce business people to the benefits costs of looking beyond excel for data visualization? Oh, that's for you, Moritz.

Oh, I'm not so sure, actually. I mean, it's a good question, but actually, I'm not that great at convincing business people. For me, it's more like I work more with people that already, let's say, have seen the light and are interested in the type of thing. So I know probably it can be tough in a bigger organization or in a more conservative, let's say, surrounding to convince people. But I mean, ideally you have a really striking example that relates to their practice. So it's not just something about, you know, soccer or wine or something, whatever, but really something where they can immediately see how that would make a difference in their daily practice, right?

Yeah, yeah, yeah.

Or ideally you have a success story. I mean, often that's the best case where you say, listen here, we had that design and I, the effect of that was like 30% more revenues. If you can construct such a story, or if you have proof of such a story, that's of course the best thing. But usually you don't have that yet because you're just getting started or so, as I said, the most important thing is really to have something where they immediately see the value personally, not just some theoretical value, but you need probably then an example of where it really.

Yeah. And maybe, Chuck, you can, if you are listening to us, you can give a look to the blog called Excel charts, written by George Cumus. And I think he wrote extensively about how to convince these people. And I think he's a little bit, I don't know, at some time, at some point he gave up a little bit, as far as I understand, is not to, I don't know, think. I think he says that at some point you have to be ready to bend the rules and do for them what they want, which is a huge topic. How much you are ready to bend the rules and whether it's a good practice. That deserves another whole episode.

That's another whole podcast, I think. So shall we move on to the news section? There have been quite some nice developments this week. So first one is visualizing sprint. I found that really exciting. That's a whole new way of collaborating online on visualizations. Have you seen it, Enrico?

That's another whole podcast, I think. So shall we move on to the news section? There have been quite some nice developments this week. So first one is visualizing sprint. I found that really exciting. That's a whole new way of collaborating online on visualizations. Have you seen it, Enrico?

Yeah, yeah, I really liked it and I'm really glad we can say something positive now about this new idea coming from visualizing.org since we criticized a little bit them during our last podcast. That's true, and I think it's a very nice concept. So basically, if I understand well, the idea is that there is a person who comes up with a first design and publishes it on visualizing.org, and then it is possible for everyone to create new branches of this design and design on top of what is there, right?

Extended, modify it, anything you would do with source code in a shared repository. And the nice thing is you can always also see the state of the design in a preview window. So it's not just the code and you have to download it and run it, but you see immediately what has changed since my last visit. Or you can click through the tree of the different variations and variants that were produced. There is voting on versions so people can say, I like this version or I don't like that one so you can also be involved with comments and votes, even if you don't program.

Yeah, yeah.

And it's really, it's a neat concept in a way. It's one of these concepts where you wonder why it hasn't been around all the time, you know, so. And that usually means it's a good idea. So. And now they start with some statistics on global water quality.

Yep.

And. But in principle, you can, you can do that for any topic. I think it's, it's really amazing.

Yeah, yeah, I like that. Then we have what, we have this new nice library that is called cartograph from.

Yeah, yeah, I like that. Then we have what, we have this new nice library that is called cartograph from.

That's amazing. It's amazing.

I didn't have time to look into details, did you? I mean, my first impression is that it's amazing. It's amazing and it covers a clear gap. Do you want to describe what it is, Moritz, since you saw it more in details?

Sure. So it was written by Gregor Aisch, who we both know quite well, and so he's driven by data on Twitter, and he was frustrated with, if you want to do maps online, what you're stuck with, it's always the, let's say the Google Maps approach. You have a lot of tiles and a Mercator projection. Right. And he, he rightly said there's so much you can do in mapping, and especially for world maps, this projection is really awful, you know, and also, if you have more zoomed in views, there's many things. Oftentimes you wish you were much more flexible in how your map looks and how it is projected and how you work with the different map elements as well, like the country borders, you know, or county borders, the regions and the streets and whatever. And so what he did is provide this framework cartograph. It's very in its early stages, but you can demonstrate already how it works and that it works really well. And what you do there is first, in Python, you render a vector based map based on existing files, of course, and you export that sort of to the web. And then on the web you can view it in SVG or also it works in older Internet Explorer versions because he uses Raphael J's for rendering. And then you can add interactive features so you can overlay on the map maybe some dots indicating stuff or some bars or whatever. And so it has these two components, like the Webviewer and the Python based map generator. And I think that makes it really interesting, for example, for news outlets, because, for instance, if you have an article about South Europe, you can render a map really focused on South Europe and add some data points there and it will be this really high end and customized version of a map you need exactly for your purpose. And as it's vector based, you can also print it nicely. So I think he's really filling a gap there. And it's exciting to see frameworks like this develop.

Yeah. It always amazes me how some people can create these open frameworks and basically donate their time.

Yeah, it's amazing. And it's so much work to get something to run. In general, to have one demo up, it's easy, but to have a framework running, it's a lot of work. And so I also really appreciate that.

Yeah. And the website itself is so well crafted already. I mean, it's really amazing.

He does a great job. Definitely.

Yeah. Yeah.

So check it out. Get involved maybe. Or if you have, let's say, a project where it might fit. I think his ideas of is also to finance further developments by having it been used in projects and use that to develop it further. So if you have a project where at Medfit, I think he's very open to collaborations.

I think now we should ask a fee to Gregor. Right? Exactly.

5% commission. 5% commission.

Okay.

Everybody has to get along somehow.

Yeah, sure.

Yeah. I have kids to feed.

Yeah. Moritz, do you want to briefly tell us something about your panel at south by Southwest?

Yeah. Moritz, do you want to briefly tell us something about your panel at south by Southwest?

Sure, sure. It was called intent and impact. So Benjamin Wiederker from data visualization, ch, or interactive things, that's his agency, he thought that out. And basically it was about, okay, we all do visualization, but ultimately, why do we do it and what's the effect of it? And can we measure that effect, you know, and can we maybe be more precise about our intentions with the whole thing about the purpose of visualizations and have reached that goal in individual projects or not? And I think that's a very worthwhile thing to do, to think about that, I have to admit. I mean, I do some evaluations, but probably I could improve quite a lot. And there was, for instance, there's one project working on the OECD builder life index. And there we now actually analyze a lot what people are doing on the site and which buttons they are clicking. And, you know, we have all these different dimensions there with like health and life satisfaction and safety and what have you. And we are looking into like, which user groups find which things most important also from the content point of view.

Okay.

And so I reported a bit on this practice of having, on the one hand, both the original data set change your mind you know, Hans Rosling says that have your dataset change your mindset, but how, after launch, after you have produced it, you should then also, let's say, watch how people use it and have that change your mindset as well. Be it for a new iteration of the visualization, or be it for simply the next one where you do something similar, or just to keep learning. So there's a lot to be learned once things are out in the wild. And so I reported a bit on these things. So this is a lot also related to the main topic of our podcast today, what makes a good info biz? Can we measure it? How can we measure it? If we can? I mean, that also ties quite a lot to the contest question we asked last time. Right. So if you're in the jury of a contest, yeah.

And so I reported a bit on this practice of having, on the one hand, both the original data set change your mind you know, Hans Rosling says that have your dataset change your mindset, but how, after launch, after you have produced it, you should then also, let's say, watch how people use it and have that change your mindset as well. Be it for a new iteration of the visualization, or be it for simply the next one where you do something similar, or just to keep learning. So there's a lot to be learned once things are out in the wild. And so I reported a bit on these things. So this is a lot also related to the main topic of our podcast today, what makes a good info biz? Can we measure it? How can we measure it? If we can? I mean, that also ties quite a lot to the contest question we asked last time. Right. So if you're in the jury of a contest, yeah.

How do you evaluate who should be the winner?

Exactly. I mean, that's exactly the central question. What is a good visualization? What isn't? Yeah. How is it in academia, Enrico, who gets to present at conferences with, how do you know what's good?

Of course, in academia, we are somewhat obliged to evaluate whatever we do. So every time we write a new research article, we have to have a strategy to evaluate whatever we propose and convince the reviewers that what we propose is valid and true, actually. So there are some traditional ways of evaluating visualization, and I think one of the most common is by argumentation. So basically, you try to convince people that what you propose just works by argumentation and giving use cases with interesting datasets and showing that it works, and it addresses the problem that you raised at the beginning of your paper. Okay, this was quite common at the beginning. So during, during the nineties, for instance, then some people started saying, yeah, but this should be done more formally. And there are many people in the community that actually come more from the HCI community. So they started evaluating visualization using the means that come from HCI. So the most traditional mean is controlled experiments, where basically you have different versions of a given visualization with some variations, and you let a number of participants run some predefined tasks, and then you measure some metrics, like, for instance, how much time it takes to perform this task, or how many errors are made using these visualizations and so on. And then you compare them formally using some statistics and showing that, I don't know, visualization technique a is better than b in this case, because it's faster and produces less error.

Okay, but I mean, that only works obviously, when you have a concrete task or like a concrete thing, you can measure. I mean, it's more like if visualization is a hammer, you hammer, like how many nails you get into at a time, you know, it's like.

Exactly. This is why many people in the field during the last few years voiced their, they were not satisfied with this, with this methodology especially. I think the main reason is the one that you mentioned, because basically in visualization you cannot really come up with predefined tasks and not necessarily something that is faster or with less error is necessarily leading to better outcomes in visualization. So what is really important is to reason about what's the purpose of visualization first. And visualization doesn't have one single purpose. You can have many purposes. Okay? So it's pretty much complicated. So what happened in academia is that some people started investigating visualization in terms of their ability to generate insights. And this is one of the biggest achievements during the last years. So the idea is visualization, the purpose of visualization is to generate new insights, which I think you can also call discoveries. And let's measure whether. So if you want to compare visualization a with visualization b, why don't we measure the number and quality of these insights? So there is a very, there is a pioneering paper from Chris north from the University of. I think it's from Virginia Tech. Yeah, I think he is. And he came up with this beautiful paper about comparing different visualizations with the same purpose. I think it was in the biological domain, if I remember well. And he measured how many insights some biologists could draw by using different visualizations. And what.

But how do you do that? Does he have an inside counter sitting next to the person that goes click, click?

It's not clicky. So actually, yeah, that's the problem, of course, because the more qualitatively you try to measure, let's say, a visualization, the harder it gets because you go from far from the standards of controlled experiments. So there are a very large number of issues here that I'm not going to mention. But what he did was really impressive. So basically he took three or four different visualizations, and then he had these people writing down the things that they could find by using them. And then they had some external raters giving grades to these insights, how useful they are, how important they are, what's their complexity, whether they are correct or not, and so on. And, and then they drew some statistics on top of them and they could show that, for instance, one kind of visualization seems to be better in generating some kind of insight. Okay.

Yeah, I liked it a lot yeah, yeah.

And I think this is, this actually opens new directions for evaluating, evaluating visualization. And I think the most important thing is introducing the idea that visualization should be evaluated in terms of the outcome. Okay. Because the traditional way, it's effect, basically. Yeah. The traditional way is very indirect. If I tell, if I start measuring how faster one visualization is with respect to another, I can come up with. I can come up with a result that says that one is faster than another, but then I don't know if being faster actually leads to better outcomes. So there is a gap there, right.

I think it fits for a couple of use cases of visualization. Let's say you're Homer Simpson and you're sitting in that nuclear plant and you have that huge dashboard, and then you can really make that. You can construct a situation where it's really good that if the person just looks from, let's say, just looks and immediately sees what's up with the power plant, if there's an unusual situation and there, it could be really about seconds. But then you have to acknowledge also that some effects of visualization are long term. So you might want to look at people using the thing very long times. This is often ignored in lab experiments. And the much deeper point, in my view, is that visualization is not just a hammer, especially today or over the last few years, there have been so many different purposes, you know, or context also, in which visualization has been used. And all of them have to be evaluated, I think, a bit differently. So also on the panel we had, for instance, the people from Jess three and Jess three, they do a lot of fun, explanatory, entertaining infographics. And their main goal, basically, is just to drive conversation, to entertain people, to have a positive image for the brand they're working for. And maybe on the way, transport some little factoids about a phenomenon, you know? And I mean, that's so far away from the nuclear power plant dashboard, you know, as is. You know, as is maybe Homer away from a manga comic, you know, it's worlds apart, you know? And in my view, I mean, that maybe it's. Or I think it's a bit hard to understand what that actually means. But I'm fairly convinced that we should look at visualizations as cultural artifacts today, as are books, as are movies, as is our podcast, you know, so we always have to understand it in that context of existing culture and existing memes, existing signs, and also from that storytelling point of view or what the not so immediate effect of a visualization is.

But I think it's really important to stress this aspect of purpose. I mean, as long as it is clear what's the main purpose, because you can have of course, several purposes at the same time. But if you can clarify what's the main purpose of a visualization, then it's much, much clearer how to design it first and how to evaluate it then. Right. So I can, for instance, I have a short list of purposes that came into my mind. You can have a visualization whose purpose is more to learn or understand a phenomenon. You can have visualization that is more targeted toward discovery, maybe with an exploratory phase. Right. Then you can have visualizations that are more targeted to monitoring something. And in that case you really want to be sure that if something happens, you can see it, right?

Exactly.

So detection there is really, really important and you don't want to miss anything that is important. Right. And then we have a huge set of visualizations that you can see mainly on the web or on magazines that are more targeted toward explaining or communicating things and in some. Sorry. And sometimes even to provoke some reaction to the reader. Right. Which is really important. How do you provoke a reaction with a visualization? And actually do you, I mean, that's really important if you are designing a visualization to provoke a reaction, are you sure that you're really provoking it or not?

Exactly, yeah. So in fact that even applies for all these cases where you'd say, yeah, but it was just for entertainment or it was just to raise awareness or it was just for, you know, giving shape to that complex phenomenon. So even if you say, I have this more, let's say softer use case, even then you can still evaluate different solutions. Maybe you can measure like hard facts that well, but you can still, for instance, ask ten people about it and, you know, work on their reactions to your work and so on.

So, yeah, I think here the only problem is that you have to be very careful not to cross the fine line between telling the truth and deceiving people. Right? So every time you create a visualization to provoke a reaction, of course you are proposing your own view, right?

Sure. Yeah.

So it's a little bit. You have to be careful there, right?

No, what I want to say is basically even if you're doing propaganda, you can test if you're doing good propaganda.

Absolutely, absolutely. I agree, I agree.

So independent of your motivation or your methodology, it sort of always applies. You just have to understand how you evaluate that specific thing.

I must admit that this is actually what I like of the work of David McCandless. So even if I find, I always found many ways to criticize his work in terms of the specific visualization he makes and how he makes them, I must confess that he's really a master in selecting wonderful topics. And almost, I would say, 80% of the times I look at an image that he produces, and he makes me think about an interesting topic. Right.

Yeah. See, and that's something too. But this is not something that would fit into traditional evaluation frameworks in the academic world. I think we can say that. I mean, there was one paper by Andrew van der Mure and Helen purchase recently, and I think they wrote really nicely on the role of design in information visualization. And they sort of tried to develop or basically just reapply or transform traditional design frameworks to evaluating information visualization or framing it. And so I think that would cover a lot of what we discussed, because they try to look both at the utility of a visualization as well as the soundness. Let's say, is it technically well done and correct? Does it display the right thing in the right manner, but also the attractiveness and the lust factor, let's say. And these are old design principles, or from architecture, you have this old motto of utilitas, vermitas and venustas. So I found that a really good framework, maybe, to encompass also these more, let's say, subjective measures and the fun factor, the last factor.

Yeah, yeah. And I must say that from my perspective, even if I live more into the academic world, it's true that I think there is some kind of correlation between the beauty of a technique or of a system and its technical advancement and solution.

I think truth and beauty and echo. Truth and beauty, I tell you. Yeah, that's it.

Truth and beauty. Operator. Yeah, I forgot.

Exactly. No, but there's absolutely something to it, so. But you have that in mathematics or in physics too, right? Which are hard sciences and, you know, not concerned with colors and forms in principle, but still, people talk about elegant solutions there and the sense of when you have an elegant solution, you know it when you see it. Right. So it's, it's hard to come up with them, but once you have one in your hands, you know, immediately. That's it. Yeah, it's great, you know?

Yeah. And I.

Good information visualization has the same thing. So that spotlight of profitability last week, we both just looked at the thumbnail and we knew it's a good visualization. We knew it's good. It could be that you look closer and then you discover, oh, they messed up the scales or they messed up the data transformations or so. But just from the look of the image, you knew it hit exactly that sweet spot between order and chaos. So it's a very clearly structured picture, but that's still visually exciting and that draws you in and provides you enough detail to study it, really. So it's structured enough to understand immediately the patterns and the big picture, but it still allows you to investigate all the little details that make up that big picture at one time. And for me, that's my measure of what is a good info with is this, does it immediately capture me visually and does it exhibit exactly this great mixture of order and chaos?

I think this is somewhat related to the idea of having elegant solutions.

Right.

I think this is true everywhere, even in algorithms. There are elegant algorithms. Right. You see a solution to a problem and it's just beautiful. It's so simple. And normally it's very much correlated with simplicity. Right. An elegant solution. Normally it's simple. Right.

It's simple, but still surprisingly sophisticated at the same time. Yeah, exactly, exactly. Simple and powerful is also like tree maps. You know, it's, it's a simple mechanism, but it's so powerful. So that also has a really big appeal and. Yeah, or we talked about edge bundling. Right. So developed by Danny Holton, these edge bundling approaches, they're so beautiful and still useful at the same time. You immediately see how they help in understanding information better, but they're also very attractive.

Yeah. And that's another case. Yeah. I remember the first time I saw the paper from edge bundling, and I think there was a picture right in front of the paper. And I mean, even before reading the paper, I knew it. It's perfect. It's. Yes.

Yeah.

Right. I mean, of course, then you want to read the details, but if you see the image of.

Yeah, you know, it's good, you know.

It's good, it works. I'm convinced I don't need to read the paper.

Right, exactly.

Of course, it's very hard to achieve this kind of things, but I would.

Like to rise another point. So we talk about a lot of, a lot about the end product. Right.

And.

But before, we also said the process is so important. So I would postulate at this point that if we evaluate quality, we should also look at how the end product came about. And I would like to compare that a bit to, let's say, food. If you have really nice food and really high end food, it is also a lot related to where the ingredients came from. If they were produced in a way that it's also morally acceptable. Does that apply to data as well? So I think we also have a politics of data we should take care of. So who generated that data set? Is it okay to display this type of data in general? And how did it come about? Was it stolen? Was it tracked without the user knowing? And then you have a whole chain of things happening until you come to the end product. And I think it's really important to look at the whole process to say this is a good visualization or not, and not just look at the utility aspect in the end. But yeah, as with a meal, you want to know exactly what happened in the kitchen, you know? You know what I mean? And that's a bit the same for me for data visualization.

Yeah. And this reminds me, I mean, it's really important to understand if you, that visualization is just one component of a very, of a larger process. Right. That that takes into account many other, many other elements, especially data. Right. So you cannot really distinguish between visualization and data. And what I found many times is that it doesn't matter how good your visualization is, if there is nothing really interesting in the data, your visualization is crap.

Right, exactly. So it's really even if you technically maybe did the right thing.

Yeah, exactly, exactly. So data visualization is not only about visualization, it's about data first. And there is, yeah, but I have.

To add, I have to add, you could do the right thing data wise, and you could do the right thing visualization wise, but still, you might have solved the wrong problem for the type of guy who uses it, right, absolutely. And that's, that reminds me of Tamara Munzner's taxonomy.

Oh.

Because she has this really nicely nested model of, let's say, on which levels can you fail. Like she describes these million ways of how a visualization can and fail.

That's the best model I've seen so far.

Yeah, it's a really good one. And on the outer shell, or the most general level, she talks about domain problem characterization, which answers basically the question, are we building the right type of thing? Right. So it's the basic idea of what we display here and what use cases we support. Is that the right thing for the people, you know, we want to help? And then if you get more concrete, you can talk about data operation, abstraction design. So if you're computer scientists, you can do that. Basically it just means to be the right mappings from the domain knowledge we have and the things we know about the world to a computable representation, like what goes inside a database and what can be expressed in a computer form. And then you have the level of visual encoding and interaction design. And if you maybe just start thinking about it, this is the level you would talk about. Like, do we use the right visual variables? Should we use circles or maybe bar charts? What can the user do? Is it the right type of interaction to support the use cases and so on? And once you have all that fixed, then you can think about the algorithm design, or you can think about how the algorithm design might fail. For instance, you have a great idea. You're solving the right problem with the right data, but your application keeps crashing.

Or it just takes ages.

Exactly. And then it's no fun to use. So you did everything right, but you failed on that fundamental execution level. I mean, that happens too. I mean, I can report. So I think she did a great job of splitting up these different areas of where visualization can be excellent or can totally fail, you know, and you have to sort of be good at all these areas at the same time in order to succeed in the end.

Yeah. And I think this is also a very nice sketch for a life cycle visualization design.

Right.

You start, it's not necessarily linear, of course, but you have these components in the, in the process.

Yeah.

And, yeah, I mean, but I would.

Still wrap her thing, these four layers. She has this as a nested box, sort of. She has that like a mamushka type of thing. But I would even put that into the bigger box of what's the cultural role of the thing you're building.

Yeah.

What does it mean for the people that they use it or that they can use this type of thing? And, you know, what does that change in the world? I think that's. So she's still very task focused or workplace focused in her view, I think. But that's just a personal thing. Maybe I'm just too casually motivated, who knows?

What does it mean for the people that they use it or that they can use this type of thing? And, you know, what does that change in the world? I think that's. So she's still very task focused or workplace focused in her view, I think. But that's just a personal thing. Maybe I'm just too casually motivated, who knows?

Yeah, but I think as far as I know, I think you have to understand that the model has been built with having some specific kind of use of visualization in mind.

Exactly. That's where we are at the beginning. Again, it's more this serious stuff, right?

Yeah. I think Tamara's target is more what she calls design studies, which is basically having a problem that comes from some kind of domain expert and trying to come up with a fully interactive visualization that solves this problem.

Solves this problem, exactly. Yeah.

So I think.

And for that, it applies really well.

Yeah.

And even for 90% of the other stuff. So I think it's a great model. One thing is also interesting, there's a code of ethics for data visualization professionals. I liked it a lot because I've been discussing it now, really, I've been discussing it over the years with a few people that we should have this sort of a dogma, you know, where we can say this visualization follows the following quality standards, how the data has been treated, how the visualization has been done. And apparently Jason Moore suggested a hippocratic oath for visualization. So, you know, as the doctors, you know, they promised to do like only good things and not so many bad things. We can now do the same. And it was nicely presented on the visually blog. So we can link there and you can read through it and see if it applies to your work as well or where it should be extended. So I found it a really a good idea to just say we have to, first of all, we have a certain standard of ethics and also to verify if certain works, you know, fall under that or not. I think it's a great idea.

I think it would be nice if having these code of ethics on every webpage or blog post of visualization designers would be really nice. I mean.

Yeah. Or on, let's say, let's have like this quality seal and I have a little sticker visualization code approved.

It would be nice to design a logo, actually.

That's a good idea.

Nice, great.

I think we can wrap it up. Basically. It's already 45 minutes again. We are talking and talking.

I think we can wrap it up. Basically. It's already 45 minutes again. We are talking and talking.

It runs fast as usual.

Wow. Yeah. So much stuff. But there's so much things to talk about. What can you do? What can you do? Anyways, do we have a suggestion for next time?

I think we were thinking about either discussing applications and tools, details and platforms, maybe.

There have been a few interesting launches this week, so we could discuss that.

Or we wanted to discuss how to learn and teach visualization. What are the options out there? Which is also a very interesting and.

Hot topic that has been raised a lot in the comments. I think, you know, like how the teaching works and how to best learn it. What are the best books for beginners? Where should you get started? I think that's a recurring thing.

So I think you guys can vote and let us know what you prefer.

I think we will do both episodes at one point, but do let us know what you want to hear first. And we should bring in some guests soon. I think we are ready. Are we ready? I think we are ready.

I am ready.

Yeah. I'm ready to.

Did you talk with Al Gore?

Not yet, no. Somehow he was busy. I don't know. Yeah, I wanted to do some interviews at south by southwest, but it was just too much people running around all the time. But I might use the chance. Next week I go to Malofiej. It's the World Infographics summit. So I might use the chance to snatch some people there and record a few sound bites with smart and entertaining guests. Hopefully.

Did you remember to call Edward Tufte?

Eddie? He doesn't pick up the phone. I don't know. No, I haven't. I haven't made any efforts. In the meantime. I was busy having barbecue, drinking beer and t shirts, you know? Yeah. Tough times. Okay. Shall we wrap it up?

Yep. Sure.

Yeah. I'm good, too. I have talked enough. I'm happy.

I think it's enough. Okay. To the next episode.

Have a great two weeks, everyone.

You too. Bye bye.

Bye bye.

Bye bye.

You close.