IEEE VIS'13 Highlights w/ Robert Kosara

Hi, everyone. This is Enrico here. This is a data stories special edition directly from Atlanta. I'm attending the BIs conference, formerly known as Visweek conference, and I am here with Robert Kosara. Hi, Robert.

Hi, Enrico.

It's great to have you here on the show again. Thank you. That's fantastic. Actually, I have to say that I'm planning with Moritz to have you on the show once again after this one. We didn't finish our talk last night last time. We still have to talk about a lot of storytelling kind of things and what you are up to at Tableau right now. I'm sure you've been doing a lot more stuff in the meantime, but let's talk about this. Of course, this is totally improvised, and, of course, the audio quality might not be the best, but I think. I'm sure. I'm sure it's gonna be good enough for listening to what we have to say here. So, Robert, what do you think about the conference? Did you enjoy it as usual?

It's great to have you here on the show again. Thank you. That's fantastic. Actually, I have to say that I'm planning with Moritz to have you on the show once again after this one. We didn't finish our talk last night last time. We still have to talk about a lot of storytelling kind of things and what you are up to at Tableau right now. I'm sure you've been doing a lot more stuff in the meantime, but let's talk about this. Of course, this is totally improvised, and, of course, the audio quality might not be the best, but I think. I'm sure. I'm sure it's gonna be good enough for listening to what we have to say here. So, Robert, what do you think about the conference? Did you enjoy it as usual?

Yes.

Is it any different from the past?

I think I actually enjoyed it more than usual. I mean, right now I'm a bit fatigued from sitting in all these sessions, and I'm kind of. Everything just blurs together. But I think it was actually really good. I'm a bit careful not to say that it necessarily was better than last year's, but I think it was pretty good. I liked a lot of the paper, and, yeah, I think I really enjoyed it. And I really like the venue, too. The hotel is actually much better than most of the ones we've had in.

The last few years. I agree. And Wi Fi finally worked. There were a few glitches here and there, but the Wi Fi most of the time worked really, really well. I'm surprised, actually. Yeah. I didn't have time myself to go around a little bit or visiting, like, the Coca Cola museum or anything like that. Did you? The aquarium?

No. Yeah, no, I just. I just went out on a run on Sunday, a bit into the area outside, and it's actually kind of nice. It has, like, the fall weather now, so that there are these residential areas around here that are quite nice. But, yeah, I didn't get to see any of the actual sites around here.

Yeah. Yeah. So let's start. What? Where do you want to start?

Yeah. Yeah. So let's start. What? Where do you want to start?

We could go through, like, the program a bit.

Yeah, let's go through the program. Yeah. I actually have to say that I came here late on Tuesday, so I skipped quite some sessions. But, yeah, I'll try to follow you. Okay. Just start wherever you are.

Well, I was just going to start with the keynote, so I didn't actually go to a lot of sessions before that, but. So the keynote was interesting because the. Or the opening session, I should say, because they talked a bit about the number of attendees and the fact that there are about 100 people who aren't actually here because of the government shutdown here in the US. And so that actually means that we had quite a few fewer people here than last year. I think we were just under 900, and last year was just over a thousand, so we lost a number of people there, but I think it was still good. And then there was the keynote. Aries Lieberman Aiden did a really good job, actually. I liked his keynote. He talked about a number of different topics that had to do with his work on modeling genome data, I guess. And he also did some work that somewhat related to that, that had to do with text and n grams. And he did a really good job, I think, making some interesting connections between social network analysis and gene analysis and stuff like that. It was a pretty good keynote, I thought.

So is he originally, what? I think he's assistant professor. Is he a biologist or what?

Well, he's some sort of mix, I guess, between a computational scientist or something like that. I don't actually know, an ebologist, but yeah, I think he's somewhere in between the different fields from what I know.

Any special take home messages from his talk?

Well, so one thing I really appreciated after that talk is how little you actually get from the sequencing data, because you get these kind of keyhole pictures of the data and how much you have to infer from that. And he did a good job of comparing that to what you get when you look at small numbers or small parts of a data set. Of large data set. But he had some interesting things there. So, yeah, it was pretty interesting. I don't know if there's anything in particular that I really took away from that.

Yeah. Okay, let's move on to the program then. That is Tuesday. Right. So in which session were you?

Yeah. Okay, let's move on to the program then. That is Tuesday. Right. So in which session were you?

So I was in the vast session because I was chairing that one.

Okay.

That was the first session, had a bit of an intro to vast, and the interesting thing there was that vast is now at 26%.

Maybe we should explain to our listeners what is vast. They might actually not know it.

So there are three conferences at vizenental and viz now actually stands for. Is now an acronym for those three conferences. So v stands for vast, I is info vis, and S is for SCIVIS and so vast is also an acronym and it stands for visual analysis. No, sorry, visual analytics, science and technology. That's what vast means. And that's the latest addition to the conferences. It used to be a symposium, it's now been a conference for three years or so, three or four years. And this is the first year that the vast proceedings are also in TVC. That's the transactions for visualization and computer graphics journal, and that's also Infobase and cybis papers are also in that. And this year they had an acceptance rate at vast of 26%, which is essentially what you expect from a good conference.

It's comparable to the others, right?

Yes, it's very close, I think, to the other two and to Euroviz and others. And so I don't know exactly how this really compares to last year, but I think was closer to 30%. And it's been. So the quality has been increasing over time, I think. So we're doing well.

The million dollar question, just answering a few seconds. What's the difference between past Infobase and Cybis? Very, very briefly. I know that's a tough one. I mean, maybe you should. Yeah, I shouldn't have asked.

No, I can try. This is all about what used to be original data visualization, which is volume data and flow data. That's what this is called. Scientific data. Scientific data visualization.

It's more on the graphics side.

Yes. And that's very heavy on the rendering and on the algorithms. And then infovis information visualization is more about data. That's nothing coming from those areas, essentially. So everything that has to do with networks, with financial data, all these things that aren't tied to physical objects necessarily. That is infovis and vast. And there's been this discussion going on for a while between Infovis and vast. What's the actual difference? And I think maybe the difference is that Infovis is a tad more on the theory side and vast, at least in the past, had a bit more on process, on how you build systems and how you go through data acquisition and provenance of data and so on, which is much less in Infovis. But it's a very blurry line. It's very hard to actually distinguish the two.

Okay, thank you. Sorry for the question. It was a tough one. Yeah. Let's move on to the sessions. So what did you see there? Anything interesting worth mentioning? So you have a best paper there? Well, I mean, there was a best paper in your session.

Okay, thank you. Sorry for the question. It was a tough one. Yeah. Let's move on to the sessions. So what did you see there? Anything interesting worth mentioning? So you have a best paper there? Well, I mean, there was a best paper in your session.

Yes. That one was about the modeling of decision. Yeah, basically data to support decision making. Essentially, and kind of high level modeling and how to show the model versus the data and so on. That was a pretty interesting paper. So, yeah, so that was a good paper. Some of the more memorable papers or the more, the ones that were more interesting for me in particular were more on the infovis side, I guess. In particular, Michel Borkins paper on memorability.

Yes. That one was about the modeling of decision. Yeah, basically data to support decision making. Essentially, and kind of high level modeling and how to show the model versus the data and so on. That was a pretty interesting paper. So, yeah, so that was a good paper. Some of the more memorable papers or the more, the ones that were more interesting for me in particular were more on the infovis side, I guess. In particular, Michel Borkins paper on memorability.

Yeah, let's talk about that.

Very interesting.

That's an interesting one.

So I know. Do you want to go? Let me talk about it.

Yeah, I think if I remember well, the paper on memorability is an experiment trying to understand when a chart is memorable. Right. And why. And if I remember correctly, they have an interesting experimental setup where they experimented with thousands of existing charts and infographics taken from big database that I think they have built on their own. Right?

Yeah, I think so. Yeah.

Yeah. Kind of like 6000 charts, something like that. And then they set up an experiment on Mechanical Turk. And how was the setup? So they have been shown? Oh, yeah. I think it was kind of like they showed a chart for a few seconds and then the next chart, next chart, next chart. And at a certain point, chart starts repeating and the users can click whenever he or she sees a chart that he has viewed before. Right. He has already seen. And then they try to figure out what are the factors that actually lead to memorability. Right. They have a way to. So they manually created a few descriptors for the charts and try to see how the most memorable one distribute over these factors. Right. That was a really interesting one. I mean, what I personally really, really liked about the paper and the presentation itself is that the study seems to be carried out in a very rigorous manner. And it's really, really interesting. I am a little more skeptical on what's the impact of memorability itself because there is a little caveat. So memorability, the way they define it, is not necessarily, it's actually not whether people remember the content of the chart, but it's more the chart itself. Right. And I think it's really, really interesting, but I'm not sure how applicable this information is. So what's your take on that?

Well, I don't know. I think in order. So, yes, that's about it. Point certainly. But I wonder if that's a first step at least, because if you remember the chart, then even if you don't remember the actual content, that makes it easier for you to go back and figure out what was maybe. So it's a bit like it's kind of a slightly different piece of work than what was done last year in the year before at CHI's about people remembering the actual content, the Johnson. And there was a follow up paper last year, and so that was more about the content than the actual chart. And some people have asked on Twitter whether it's basically about, not so much about memorability, but about whether the chart stands out or not, which may be part of the issue. But that could also be a good thing because if you want to draw attention and people just kind of scroll through things or they flip through pages and certain charts stand out, they're going to stop and look at them. So if you want to draw attention, I think this was also useful guidance to figure out what is useful.

I think another tricky issue there is that if I remember, well, I don't remember the details, but basically one of the outcome of the study is giving a little bit of guidelines on how to make the chart memorable. Right. And this seems to go, at least to some extent, against some of the principles that we have in visualization from the past. Right. So I don't know. What do you think? I mean, I remember, I think one of the outcome is that if you put images in your charts or objects that are recognizable, figures or colors, lots of colors, then it becomes more memorable. Right. But then maybe also even more cluttered. I don't know.

Yeah, well, it's true, but so one thing they found, which I thought was really interesting, is that they had these four classes of different charts, and I forget what all of them are, but one of them is. Well, so there's infographics. Oh, yeah. I think I actually remember them. So there's infographics, there's news graphics, there's visualization examples, I think, from the info conference. And then there are government charts. And they found that the least memorable were the government charts.

Oh, yeah, yeah, yeah.

They all look the same and they all have these very, very simple structure and very kind of very similar types of colors and so on. And so I think that the interesting thing there is also that if you want to make a point, like when you're speaking, you want to also kind of, you know, you punch the words, you make, you make, you try to be a bit louder and a bit more deliberate about those things. And it's similar with a chart. If you're trying to make a point about something using, perhaps you want to stand out and you want to make it look a particular way. It doesn't mean it has to be lots of chunk in there or there has to be a lot of graphics necessarily, but maybe you want to have a bit of a unique color scheme or.

Absolutely.

Something like that to just make it stand out a bit so people actually remember it. Otherwise, if you've seen, you know, if you see 15 similar bar charts, they all blend together. You don't remember what they actually were.

Yeah. And I think the charts coming from scientific papers were second place there. Right, right, yeah, scientific papers. And this looked pretty clean. Right. I mean, the first one, infographics were lots of clutter, but the second one was quite, quite clean and still quite memorable as well. So probably reconciling these two kind of things, it's an interesting further kind of work and research. Right? Yeah. And I think we also have to mention the dinosaur kind of meme that was fun.

Yeah. And I think the charts coming from scientific papers were second place there. Right, right, yeah, scientific papers. And this looked pretty clean. Right. I mean, the first one, infographics were lots of clutter, but the second one was quite, quite clean and still quite memorable as well. So probably reconciling these two kind of things, it's an interesting further kind of work and research. Right? Yeah. And I think we also have to mention the dinosaur kind of meme that was fun.

That took on a bit of a life on its own. Of its own, yeah, yeah.

You want to explain what.

Oh, so there was one of the examples that Michelle showed, I think, had a dinosaur in it, and that was kind of her example for putting memorable or the recognizable images into a chart. And then people started doing that in their talks to say, this is an important point. So here's a dinosaur. Yeah, actually. Well, so I stole that idea from Danyel Fisher. He did that in his talk. And so I googled for a dinosaur picture and said, put this in there as well just to make it fun and more memorable.

Yeah, yeah. That was a lot of fun. Yeah. But let's move on. In the same sessions, there were. So I have to say that this was the session on storytelling and presentation. So arguably a very interesting session for our listeners here. What else we had there. So we had sketchy story from, I think, mostly people from Microsoft research here. Right?

Yeah, yeah. That was a lot of fun. Yeah. But let's move on. In the same sessions, there were. So I have to say that this was the session on storytelling and presentation. So arguably a very interesting session for our listeners here. What else we had there. So we had sketchy story from, I think, mostly people from Microsoft research here. Right?

Yeah. So that paper was about basically creating infographics, style visualizations very quickly by sketching and then having the system use those sketches to show data in different ways, using bar chart type things or circular charts. They're like donut charts and a few other types and just quickly sketching those and laying them out on a large display, I guess.

Yeah, yeah. I think it's hard to explain.

Yeah, you can see the images.

We should actually. Maybe I'll try to post something on the blog post after recording the.

Maybe some links also, the papers would be used.

Yeah, yeah, yeah, yeah. Absolutely.

Well, actually, the first one. So this is a different session than what we talked about earlier, but the. The first paper in that session was Jessica Hammond's paper on understanding sequence in narrative visualization, which I thought was really interesting because she. News graphics use sequence to get points across and how you can look at different kinds of transitions. So that either walk you through. One of the terms I liked from that paper is something that she calls the metric walk, where you go from one way of looking at the data to a different type of looking at the same data, but using a different measure. And then walk people through, essentially an argument of what you're trying to tell them. And then also there's a thing about how there are different transitions that basically try to minimize the change so you can see the same thing from kind of different sides. Walk through the story that way. So there's lots of interesting stuff in that paper.

They have something. Did they define something like change cost, if I remember, or something. Some kind of cost, yeah, that was really interesting. I think what is interesting there, again, I think, is it's how they integrated different pieces of research there. So they've been coding a very large number of examples to try to tease out what are the main factors in coming up with. With useful sequences. Right. And then they also build some software on top of that. Right? Am I right? Or a library. I think they have a library or something that actually tries to suggest what's the best sequence given an existing sequence.

Okay, yeah, I remember that.

Yeah, yeah, yeah, yeah. I think there is also some software built on top of that. Really interesting work. I really like that one. And useful. I think it's really useful.

Oh, yeah. Absolutely. Yeah.

What else? Oh, then we had this one on, using concrete scales, a practical framework for effective visual depiction of complex measures. I know that you had some little concerns with this one. Yeah. You want to explain what that was?

Yeah, sure. So the idea there is. It's actually a really good idea.

I find it really interesting, I have to admit. The idea was interesting, right?

Yeah. Okay. So the idea was to basically use real world objects to compare something to. So one of the examples was how much sugar is in a can of coke, for example. So they would show the actual sugar amount in terms of sugar cubes. They would actually depict it like that. Or they would have, like, when it's about money, they would stack up money and then compare that to, like, the statue of liberty or something like that. So that would be a concrete thing that you can compare to. And that's a good idea. I don't actually argue against that. The thing that I had a problem with is that when you get. There's a certain limit to the size that you can depict, because it works well for small things and for the sugar cubes, because you can relate to those. But when they compare things to the Statue of Liberty or another example that they used is a graphic that has an image of the globe of the earth and it has all the water in the ocean as a sphere sitting on top of Europe, basically. And I just have no idea what that means because I don't have a sense of how big Europe is and how big a. In terms of something I can relate to. That just doesn't make any sense to me. And because it's just way too big. And then you have a sphere on top of that that is roughly. Diameter is roughly that of western Europe. And that just doesn't really. That as a sphere is just not a concept that I can grok. So I just didn't. So I feel that there is a certain limit to what you can depict using concrete scales because it just goes way outside your actual experience. And then you get to the point where it doesn't actually make any sense anymore. But. So that's my concern. But otherwise it's a really good paper. And the presentation was actually very funny because they did some really clever things there with actually using some physical objects to. To show stuff. And that was pretty well done.

Yeah, it was kind of a little show there. It reminded me, Nigel Holmes, when he.

Did this at Tapestry.

Yeah, tapestry, when he actually came. How was it? So he wanted to show the length of the longest jump at the Olympics. Right?

Yeah. So 29ft and a bit. And he had like a piece of string of lengths to show how long that actually is. Because just that number isn't something that you really can, even if it's a very easy number, but it's actually not something you can really easily depict or picture in your head. And so you actually showed it as an actual length. That was pretty cool.

Yeah. And I have to say that this reminded me that depiction in the real world, when you see these things in your own physical space, then it really makes a big, big difference.

Right?

Yeah. Cool paper. Anyway, then we add what visual sedimentation was cool as well. Right. So if I remember well, the idea of visual sedimentation is to use the metaphor of snow accumulating on top of some objects. So this metaphor is used to deal with dynamic data or streaming data. So how do you deal with streaming data and visualization? And they come up with this interesting metaphors that basically the objects, the incoming objects, form some kind of. Oh, that's mine. Yeah. For instance, in one of the examples they have objects that are falling on top of some bar charts and the length of the bar charts changes in time. But you can also see these little bubbles accumulating one on top of the other and slowly being absorbed by the bars. Right. And then I think they did a good job in showing what are the parameters of this kind of system and how the visualization changes when these parameters change. And they also try to apply the same idea to many different kind of visualization techniques. I find it really interesting, and I think they also have a library out there that people can use. I think it's written on D3. So I think they also did a quite good job on having a nice webpage out there, a GitHub repository with the library. So if any of you guys is interested in doing, in using this library again, I will post the link on the blog post, and you can go there and just give it a look and use it. Do you have any comments on that? On that one?

Yeah. Cool paper. Anyway, then we add what visual sedimentation was cool as well. Right. So if I remember well, the idea of visual sedimentation is to use the metaphor of snow accumulating on top of some objects. So this metaphor is used to deal with dynamic data or streaming data. So how do you deal with streaming data and visualization? And they come up with this interesting metaphors that basically the objects, the incoming objects, form some kind of. Oh, that's mine. Yeah. For instance, in one of the examples they have objects that are falling on top of some bar charts and the length of the bar charts changes in time. But you can also see these little bubbles accumulating one on top of the other and slowly being absorbed by the bars. Right. And then I think they did a good job in showing what are the parameters of this kind of system and how the visualization changes when these parameters change. And they also try to apply the same idea to many different kind of visualization techniques. I find it really interesting, and I think they also have a library out there that people can use. I think it's written on D3. So I think they also did a quite good job on having a nice webpage out there, a GitHub repository with the library. So if any of you guys is interested in doing, in using this library again, I will post the link on the blog post, and you can go there and just give it a look and use it. Do you have any comments on that? On that one?

Well, yeah, I liked it. It's actually, it's a very effective way of showing change. They had a neat collaboration with a tv station where they were using as part of a tv show, and you could see these bubbles kind of dropping onto the bars in the background and for like, tweets that I remember what it actually was about, but it's nice. And it's an actual real use of that visualization in the media, which is kind of neat. So that was pretty cool.

That was cool. Okay, let's move on to something else that might be interesting for our listeners. What else? I'm just flipping through the program. Is there anything special you want to mention, Robert? I have to confess, I skipped so many sessions.

That was cool. Okay, let's move on to something else that might be interesting for our listeners. What else? I'm just flipping through the program. Is there anything special you want to mention, Robert? I have to confess, I skipped so many sessions.

Yeah, I didn't go to other sessions either, but there was a paper by Carlos Scheidegger and his colleagues at at and t research on nanocubes, which is a way of showing a lot of data on the map. I'm not sure if it's actually tied to map necessarily, but that's what they're doing in their examples, in their current data users. And it's very nice because it actually shows a lot of data. Like, for example, it shows tweets on the map that are happening during a certain time, and you can zoom in and out, and it adapts the display to where you are so that the density or the resolution actually changes depending on how far away, how far you zoomed in or out. And it's very, very fast, and it can deal with a very large amount of data with a very nice system. And it's also all open source. So there's a website nanocubes.net, where you can actually play with it if your browser does webgl. So it uses Webgl, which basically means chrome, I guess. Maybe Firefox, I'm not sure. I don't use Firefox, but Chrome definitely works, so you can play with it. And also there are links there to, I guess, the paper and to the GitHub repository, so you can download the code and play with that. So it's pretty nice. It's a really cool idea. It's kind of straightforward because you can really see very easily what it's about. But the engineering behind it is pretty interesting as well.

Yeah, I think that was a cool one. I think especially. I think it's the first technology I see on the web where for visualization that really scales to a lot of data points. Right. That's new. What else? Well, you participated to a quite nice panel that was a very controversial one, maybe a little artificially controversial, I don't know. No, I think it was quite controversial that the panel was on big data. I don't remember exactly the title was it means to an end, or an end to a means, something like that. Organized by Aritra Dasgupta, who is actually also working in my same department, and he's a former student of yours. Right. So there were how many participants on the panel? You. Carlos Scheidegger from at and Taideh. Danyel Fish, Danyel Fisher from Microsoft Research, Heidi Lam from Google, and then Danyel Keim from University of Konstanz, who basically played the part of the academics against people from industry. Right. And there were quite some good points out there. So what was your main take there? I mean, I think you try to give the Tableau perspective kind of thing.

Yeah, I think that was a cool one. I think especially. I think it's the first technology I see on the web where for visualization that really scales to a lot of data points. Right. That's new. What else? Well, you participated to a quite nice panel that was a very controversial one, maybe a little artificially controversial, I don't know. No, I think it was quite controversial that the panel was on big data. I don't remember exactly the title was it means to an end, or an end to a means, something like that. Organized by Aritra Dasgupta, who is actually also working in my same department, and he's a former student of yours. Right. So there were how many participants on the panel? You. Carlos Scheidegger from at and Taideh. Danyel Fish, Danyel Fisher from Microsoft Research, Heidi Lam from Google, and then Danyel Keim from University of Konstanz, who basically played the part of the academics against people from industry. Right. And there were quite some good points out there. So what was your main take there? I mean, I think you try to give the Tableau perspective kind of thing.

Yeah, I wouldn't necessarily call it that, but I was trying. So my part was, I was trying to be the nasty guy, basically saying that, you know, academics need to. Need to do things differently if they want to be relevant in the big data era. And my main two points are basically that infovis prototypes, the kind of software that people built for research, needs to talk to databases, because that's where the actual real big data lives. People don't use CSV files, they have the data in databases. And databases are also useful because they can do a lot of work for you. So you don't aggregation, which I'll get to in a second, but aggregation can be done through database. A lot of information about data can be gotten from the database, and of course you can't if it's really big data, you can actually have it all in memory, like a lot of infovis tools do, load in a data file, and they expect to be able to do that in memory, which you just can't do if you have terabytes of data. So that was my first point. My second point was to say, well, if you have millions or hundreds of millions of data points or records in your database, you can't just show them all as individual points, like in a scatterplot or parallel coordinates. You have to actually go and think about aggregation and think about different ways of showing that data than just doing a one to one mapping of data. Point to something on the screen. And that's something I think is really, really important because we still see a lot of work that goes essentially one to one from data point to point on the screen, or whatever the point actually is. And so that's something that I really feel people need to change their thinking a bit and think more about how to aggregate the data up, how to break it up from the top down rather than from the bottom up.

Yeah, absolutely. And if you start thinking about how to interact with that, then it's even more complex. I think there were also some interesting discussions on whether we really. I mean, is it really a problem for visualization? So should we try to tackle this problem at all, or. I think one of the points of Danyel Keim was, well, we still have to solve so many problems with small data sets that it doesn't really make sense to work with super big data sets, which I think it's an interesting point, but at the same time, I think you also answered well, but there are people who really have problems with a lot of data.

Right.

And what do we do with them? It's definitely a big problem there. Not easy to solve. Yeah. What else was there? I think at some point, the discussion was quite heated.

Yeah, we started the little fight on stage, actually. But it was good. There were good comments, and we had a good discussion about all kinds of issues, like where does the actual, where does the name come from and things like that. But I think that turned out into a pretty good discussion overall.

Yeah. I think another interesting point is that, I think, again, it was Danyel mentioning that it was kind of like, well, right now we are all dealing with this big data hype, but in a few years, it's probably gonna end, and we still are. We will still have the same problems in our hands. Right. That was interesting to me. What else, Robert?

Yeah. I think another interesting point is that, I think, again, it was Danyel mentioning that it was kind of like, well, right now we are all dealing with this big data hype, but in a few years, it's probably gonna end, and we still are. We will still have the same problems in our hands. Right. That was interesting to me. What else, Robert?

There was actually another panel. So there was another panel on evaluation. That was the day before. I think that target, where I remember who was actually on that panel, but there was Min Chen, Tamara Munzner, who else? Brian Fisher, and one more. Oh, DVD Ebert. And it was organized by Bob Laramie. And the idea was to basically talk about the role of evaluation in visualization, which kind of turned into the role of evaluation in getting papers accepted.

Yeah, yeah, yeah, yeah.

That's an important discussion for people. And it actually was kind of interesting to see people basically argue against the need for so much evaluation. So evaluation is obviously a good thing. And that was a good discussion. Min Chen has some good points about what he called user studies versus empirical studies. So one of them is basically testing your new technique. It's a user study, and empirical studies is basically about learning more about how visualization works and how humans work. Right, exactly. And that was really good. But then much of the other discussion was about how much the reviewers should insist on user studies for new techniques and new ideas. And I think that there is a bit of a perception now that a lot of papers get shut down, shut down by reviewers who just insist on a study, even if it isn't all that important in the beginning, perhaps if you can kind of see that it's a good idea, but you always insist on getting the study for it as well. And it might help to just get out there and then see, you know, have somebody else do the study, for example. So it's not even the same authors and so on. So there are some interesting points there.

Yeah, yeah, I think so. What else, Robert? I let you.

I'm talking too much.

No, go ahead. Go ahead. Sure.

There are some. Let me see. I'm also going through my notes here. There was some papers that we just saw, actually. Oh, yeah. So there is a paper that was kind of interesting in somewhat unusual way that talked about how to pick the right aspect ratio for a scatterplot. And that is an interesting problem. So basically, the question is, if you have a certain distribution of points that you want to depict in a scatterplot, should it be a square or a rectangle? How long versus how wide? Maybe it should be higher, like taller than wide or the other way around. And that paper had some interesting. Well, the presentation was interesting because that's obviously an important question. And they had some data technique that's based on the Delaunay triangulation of the points. And that has kind of a bit of a recursive kind of nature because they have to basically scale it, triangulate, see if certain criteria met, and then change it and then see if the triangulation changes or nothing. What was interesting is that they mentioned that there's a study apparently that says that when you give people points that are distributed on a piece of paper and ask them to draw lines between them, that 98% of the lines that people draw are actually Delaunay edges.

Oh yeah. Yeah. That was a cool one.

Yeah. I did not know that. That was really interesting. I didn't realize that Delaunay triangulation was so close to kind of the natural way we think about edges.

Did you try to search for the original?

No, but I think there's probably a reference in the paper, I would assume. I haven't actually looked at it, but that's another I took and I figured that would be really interesting to follow up on. But that paper looks interesting. The problem is that the technique is just really slow. So it works for a couple hundred data points in a quarter of a second or so. But when you increase the number of data points to a couple thousand, it's probably going to take way, way longer. So it's a good idea, that's for sure. But it has its limitations the way it's done right now. But this idea of picking, there were papers in the past on the how to pick the aspect ratio for line charts, and they're probably going to be more on different kinds of visualizations and picking more of those parameters for those. So I think that's related to the.

Cleveland's banking to 45 degrees.

Right.

I think the original idea was only.

For time series or technically it was only for comparison of two lines. Okay, so the original study wasn't actually about the whole chart, it was more about what if you have two line segments and you want to compare their slopes? And there was actually a paper last year or the year before that talked about some limitations in that study and actually showed that the 45 degrees is not actually the actual correct interpretation of that study. But that's what stuck. But there are some interesting ideas there on how to pick that the right way.

Yeah, I agree. That was an interesting one. There is another one I want to talk about from later infovis sessions on time trees and graphs. Have you seen this one? Any about animating or comparing graphs? Have you seen that one?

Yeah, I agree. That was an interesting one. There is another one I want to talk about from later infovis sessions on time trees and graphs. Have you seen this one? Any about animating or comparing graphs? Have you seen that one?

No, I didn't see that one.

That was a really interesting one because this is basically a user study kind of work. They tried to. So the problem that they try to address is if I have something as complex as a graph with nodes and edges connecting points, and this graph changes in time, what's the best way to represent that. And some of the obvious solutions are having things like small multiples, or turn these charts into difference charts, where basically every single chart represents the difference between one data .1 point in time and the previous one, or having some sort of animation. So what these guys did was, first of all, defining all these different solutions and also trying to combine these solutions. So if you want to see our graph changes between ten different data points in time, there might be some kind of changes that are better perceived with showing the difference, or showing an animation, or showing a small multiple and so on. And they ran a few studies on top of that, and I thought it was really, really interesting. Not necessarily the most solid thing I've seen around, but I think that was interesting, at least in the way they try to lay out the design space there. And I think this could also be extended to many other kind of cases, at least to some extent, because every time you have something complex on the screen that is changing over time, you don't have too many choices. You either animate it or trying to figure out how to depict the difference, or you have some sort of small multiples. And the idea of integrating these things together, I think it's an interesting one. A very interesting one. What else, Robert?

That was a really interesting one because this is basically a user study kind of work. They tried to. So the problem that they try to address is if I have something as complex as a graph with nodes and edges connecting points, and this graph changes in time, what's the best way to represent that. And some of the obvious solutions are having things like small multiples, or turn these charts into difference charts, where basically every single chart represents the difference between one data .1 point in time and the previous one, or having some sort of animation. So what these guys did was, first of all, defining all these different solutions and also trying to combine these solutions. So if you want to see our graph changes between ten different data points in time, there might be some kind of changes that are better perceived with showing the difference, or showing an animation, or showing a small multiple and so on. And they ran a few studies on top of that, and I thought it was really, really interesting. Not necessarily the most solid thing I've seen around, but I think that was interesting, at least in the way they try to lay out the design space there. And I think this could also be extended to many other kind of cases, at least to some extent, because every time you have something complex on the screen that is changing over time, you don't have too many choices. You either animate it or trying to figure out how to depict the difference, or you have some sort of small multiples. And the idea of integrating these things together, I think it's an interesting one. A very interesting one. What else, Robert?

Well, those were the most ones that I found really interesting. Oh, there was one that was actually kind of unusual today that talked about how you lay out hierarchical data tables. Well, not tables as such, but the actual. Well, in the end it was tables, but he talked about these forms that, these very kind of old fashioned forms where you fill in certain fields, but the actual technique is more for showing the data, I believe. Maybe I'm wrong, but the idea was that if you have tables and those and this, or if you have essentially more complex structures, so they look at XML files that can have they have hierarchical structure, but they can, they're much more complex than simple tables, and they do this. And so they have this very interesting nested system where you can put a table inside a table and then also have those tables kind of next to each other, or become multi column tables. And this sounds kind of odd, perhaps, but it actually makes a lot of sense when you see it, because there is a more compact and more interesting layout that way that gives you a better sense of what the data looks like. And this is used for very textual or like mixed data sets. So where you have, like, they were looking at course descriptions, where you have a list of, you have a title and maybe a description of the course and then you have readings and you have exams and you have assignments and you have like for each assignment you have a number of points you get and a bunch of other things. And so all of that to combine it into a single display, even though it's textual in a way that's automatically laid out. And the presenter was talking about how they. How this is actually done fairly well when it's done manually, but not when it's done automatically. And they had some really good ways of doing things that look like it was done manually, even though it's automatic. That was actually a surprisingly interesting paper, even though it wasn't actually visualization as such. It was more like layout of a data table.

Okay. And I just noticed that we didn't mention at all the best paper. And I have to confess that I didn't give a look to it. I don't know, have you attended?

But I was in the other session. In the vast session.

Okay. Yeah. So we don't know anything about that. Okay. Too bad. Yeah, that's fine. I think we pretty much covered all the things that at least were interesting for us. The only thing that I want to mention is that of course there are many, many other things going on at this. It's just that that's our very personal view according to what we have been able to attend. But in the conference there are many other events taking place, like tutorials.

Oh, yes.

Yeah, yeah. Tutorial. I think it's. It's one of those things that might be interesting in the future for some of the people who are listening to the podcast. I just want to mention that there are many, many other things going on here. I think there was also an art exhibition or something like that. Did you have a chance to go there?

Yeah. So this was actually pretty good this year. So they had. The art show has been a part of this week for a while, but it kind of died down over the last few years. And now this year somebody else is running it and they actually got some really good pieces. And they actually had a paper session where they had a number of presentations from people who were doing more artistic work for individualization, which I didn't, of course, see because it was at the same time as our big data panel. So thereabouts five parallel sessions here or tracks at the same time. And so that includes tutorials, it includes workshops. There are a number of workshops here. There's also. We entirely ignored cybus here.

That's our personal bias, of course. Right.

Yeah. So each of us just get to see a small number of things. And then, you know, you run into people as you're heading to a session, and then you, before you know it, you've talked for half an hour and missed a couple of papers. So that's actually, it's kind of good because you tend to get to talk to lots of people, but you also miss a lot of the actual presentations.

Yeah. And there was also an industry session kind of thing with posters and some. And also some panels or workshops.

Well, there was this kind of track. So in the program, there are these little marks that kind of tell people which of the sessions are considered more irrelevant for industry folks. And those are actually, that's not a new thing that started last year, but it's a good idea to give people a bit of. A bit of guidance to do the more applied or to show the more applied work and things that might be more relevant for industry folks.

Yeah. And I think there was also a session on the Bas challenge, so we had actually an episode on the show, on the podcast on the bass challenge. So I wanted to mention that again. Unfortunately, I couldn't attend that, but I've heard that normally this session is pretty cool because the participants have a chance to compare the results. And I think there are also some awards that they give to the people and have heard that there have been quite some good, interesting, some interesting submissions. And that was cool. That's definitely cool.

Oh, another session that I also missed was viz life.

Oh, this, like, we should mention that. Yeah, unfortunately, I missed that, too. Yeah.

So this is a session that they organize now every year, I think for the last couple of years, that where the idea is that people can just bring things that. That are funny and that are usually bad visualizations, and they're, they're not. It's not in the program, unfortunately, but it's kind of, it's organized as a meetup, and that was on Tuesday night or so, and that they had, I think they had about 100 people there this time and apparently was really funny. You basically just go up and show stuff that you found somewhere that are really bad examples and that are funny. And so that can be really good, especially when you get enough people there to actually show things that they found. So that's really. And it's nice because it gives people a way to kind of relax. Yeah. And to make fun of a bit of things, which you also need to do sometimes.

Yeah. Do people bring things they have done on their own?

Well, I don't know. We could.

We should. Yeah. Okay. And so, unfortunately, we cannot comment on the, on the capstone because that's gonna happen tomorrow. But I have to say I'm really excited because the Capstone will be given by Jarke van Weiche that maybe some of you might not know, but is actually the person behind a lot of cool stuff in Infobase. He's been in this field forever. Right. I think it's.

We should. Yeah. Okay. And so, unfortunately, we cannot comment on the, on the capstone because that's gonna happen tomorrow. But I have to say I'm really excited because the Capstone will be given by Jarke van Weiche that maybe some of you might not know, but is actually the person behind a lot of cool stuff in Infobase. He's been in this field forever. Right. I think it's.

And both scientific and information vision.

Yeah, both scientific and information visualization. So just to make it clear, he's the guy behind things like bundling. Edge. What's the name of this technique? Edge bundling. Yeah, edge bundling. What else he has done? I think he has done things like verified square.

No, not square fight, but the cushion.

Tree, the cushion three map.

I think.

I think it's qualified as well.

Oh, you're right. Yes. I thought it was Martin Wattenberg, but he was doing, he just used them for the, for his members of the market.

Yeah, yeah.

And he also, so Jarke also had like papers on flow visualization. He's done a lot of work there. There's a technique, the line integral convolution was his paper or his work and a whole number of things in SCIVIS as well. So he's got a very broad range of things. Something really cool that he showed at Darksthul a couple years ago is something that's called the, it's a map projection. I'm trying to remember the name of it. The multi or not Malta Hick. No, it's got a very clever name that I can't remember. But basically the idea is that rather than breaking up the globe in the usual way, which is very, very regular, he basically goes along lines that are defined by some error metric and that produces some very cool ways of flattening out the globe and produces some really cool maps. I think it's called myriahedral.

Yeah, projection. Yeah, yeah. Which actually look quite artistic as well.

Yeah, they're awesome. And there's a video that shows how he kind of unfolds the globe in different ways, which is just amazing. So it's really, really cool stuff that he's doing.

Yeah. This is what I also like of his work, that he has this kind of artistic or designy kind of touch. I think he's originally an industrial designer by training. So it's really, I'm really, really looking forward to see what he's gonna.

Oh, he had at some point he had this tree visualization paper that was called botanical.

Oh yeah, yeah, botanical tree, which was.

Like trees, you know, that looked like actual real trees. It was pretty cool. Yeah, yeah, yeah. There's lots of stuff that he's done, so it's gonna be fun. And he's a really good speaker. He will be really fun.

Yeah, yeah, yeah, yeah, yeah. Okay, I think we can basically conclude here. I just want to mention that, yeah, this has been really cool, as usual. And the next one is going to be in Paris.

Yeah, yeah, yeah, yeah, yeah. Okay, I think we can basically conclude here. I just want to mention that, yeah, this has been really cool, as usual. And the next one is going to be in Paris.

Oh, yeah.

So I think that that could be an option for some of our European listeners. Might actually want to go there. And Paris is always a nice place to go. So I have to mention that I hope that Moritz will manage to go there. I'm always pushing him to go to come to this. I think he's never been to any vis conference before. So, Moritz, when you will be listening to that, you cannot skip that one. And, yeah, you no longer have excuses next year. Well, Robert, thanks a lot. That's a great help. I'm really happy to have you on the show again, and as I told you at the beginning, we want to have another session with you sometime soon.

Sure. Absolutely. I'd be happy to.

Okay. Thank you.

Thank you. Bye.