Researching the Boundaries of InfoVis with Sheelagh Carpendale

I am learning all the time and I can learn from everybody.

Hey, everyone, welcome to a new episode of Data stories. My name is Enrico Bertini and I am a professor at New York University where I they do research in data visualization.

And my name is Moritz Stefaner and I'm an independent designer of data visualizations.

And on this podcast we talk about data visualization, analysis, and generally the role that data plays in our lives. And usually we do that together with a guest we invite on the show.

That's right. But before we start, just a quick note. Our podcast is listener supported, so you might notice there are no ads interrupting us and you get the best enjoyment of the show. If you do enjoy the show, you could consider supporting us. You can do that with either recurring payments on patreon.com Datastories. So basically you can pledge a dollar or two or three or as much as you can afford for each episode that we put out. Or if this is too much of a commitment for you, you can also send us just a one time donation on Paypal me Datastories and just send a little bit of money away. Please consider doing that in case you, well, you have a good time listening to us or maybe even learn something for your professional life, especially. Then, I think a little payback would be in order, but that's totally optional. But we are, as we said, just listener support, so every dollar counts for us.

Morris, you are dispensing such good advice to our listeners. So let's get started. So today I'm really happy to have a person on the show who we wanted to have for a long time. She's definitely one of my all time favorite researchers, professors, and a constant source of inspiration. We have Sheelagh Carpentel on the show. Hi, Sheelagh. Welcome on the show.

Morris, you are dispensing such good advice to our listeners. So let's get started. So today I'm really happy to have a person on the show who we wanted to have for a long time. She's definitely one of my all time favorite researchers, professors, and a constant source of inspiration. We have Sheelagh Carpentel on the show. Hi, Sheelagh. Welcome on the show.

Hi, how are you? I'm good, thank you.

So, Sheelagh, we always ask our guests to introduce themselves. So can you give us a little bit of a short bio?

Okay, well, I'm a professor in computer science here at the University of Calgary and I have been here for 19 years. I think one of the. Yeah, it's a long time. One of the most important things for as far as the short bio goes is that I was initially in the arts world. I went to art school and design school before I switched and did a couple of degrees in computer science. And I have actually really found that both backgrounds are super useful in doing information visualization research. And just kind of an added note, I have, after all these years, decided that I will switch universities and come fall, I will be at, at Simon Fraser University in Vancouver.

Wow. Wow. That's a big change.

It's a very big change. It's a very big change.

Nice.

That's exciting.

So, Sheelagh. Yeah. What I really like of your work, among many other things, is exactly this kind of mixed background that you have between art and computer science and many other things. And I can tell you how many things I learned from you over the years, and it's been always a constant source of inspiration. As I said at the beginning and before starting, I also want to mention that I realized that preparing for this show, I realized that we had lots of people on the show previously who actually had experiences in your lab. Right. So we had Chris Collins back then, Marion Dork, Petra Eisenberg. I'm not even sure I'm mentioning all of them. Alice Thudt, Dominic Osbower. So many people. So it's fantastic. Yeah. And a few weeks ago, I personally visited your lab, and I loved it. You gave us a great tour of your lab. We spent some time there, and I finally thought we should definitely have Sheelagh on the show. That's been a long time coming. So I would like to start with, I think the other day when I was visiting your lab, I kept thinking one of the most, one of the things that characterizes Sheelagh's work is pushing the boundaries. Right. I see your work as it's like, for me, it's like, it's not the traditional things that we see around. It's really like you're pushing the boundaries in so many directions. It looks to me you have a mix of art, psychology, philosophy, computer science. So I'm wondering if you can briefly talk about what is your approach to visualization and maybe your philosophy.

So, Sheelagh. Yeah. What I really like of your work, among many other things, is exactly this kind of mixed background that you have between art and computer science and many other things. And I can tell you how many things I learned from you over the years, and it's been always a constant source of inspiration. As I said at the beginning and before starting, I also want to mention that I realized that preparing for this show, I realized that we had lots of people on the show previously who actually had experiences in your lab. Right. So we had Chris Collins back then, Marion Dork, Petra Eisenberg. I'm not even sure I'm mentioning all of them. Alice Thudt, Dominic Osbower. So many people. So it's fantastic. Yeah. And a few weeks ago, I personally visited your lab, and I loved it. You gave us a great tour of your lab. We spent some time there, and I finally thought we should definitely have Sheelagh on the show. That's been a long time coming. So I would like to start with, I think the other day when I was visiting your lab, I kept thinking one of the most, one of the things that characterizes Sheelagh's work is pushing the boundaries. Right. I see your work as it's like, for me, it's like, it's not the traditional things that we see around. It's really like you're pushing the boundaries in so many directions. It looks to me you have a mix of art, psychology, philosophy, computer science. So I'm wondering if you can briefly talk about what is your approach to visualization and maybe your philosophy.

Right. Well, you see, I actually think, you know, it's well known in science that to be successful, you have to be innovative, inventive, etcetera. But in science, we tend to think that, I don't know how people are going to pick this up, but by osmosis or something. To be good, you must be inventive. But, you know, so where does it come from? Yeah, in design school and art school, but particularly in design school, it's something that you practice. And so that has been super useful. And so when I started being a professor, I did start right away actually introducing all kinds of different practices and also, like, adjusting practices to incorporate design thinking. I mean, I think that one of the things that I do a lot is what I think of as observation for design. So I kind of have, it's a mashup of ethnographic approaches, which, with all the rigor of qualitative studies, but also always keeping your eyes and mind open for those moments where you see that something could help, where you see somebody struggling with their own data or struggling explaining it, like you think, okay, maybe there we can do something. So. Yes. So the combination of, you know, watching and actively using design thinking in our whole ideation process, also actually really encouraging students to give themselves a bit of time in that. In that period of the research.

Yeah, yeah, I really like that. I think we have this notion that visualization can be learned by following a whole bunch of rules, and this never seems to work. And. Yeah, right. I mean, it's frustrating, but that's not the way it works. I'm sorry.

I know that actually, just a week or two ago, I was so. I know that, like, in the nineties, when I was looking up just, like, casual design textbooks, not like ones that were, like, fabulous or anything, they all had in their front piece or in the first paragraph of the first chapter or somewhere right at the beginning, they had kind of two statements, and one that if you wanted to be a designer, you would be very foolish not to know and understand all of the guidelines and all of the rules that everybody before you had made, but that you would be even more foolish if you followed them. I was looking for somewhere so I could have a quote for this. I couldn't. These days. I couldn't find it. They all are spouting their guidelines, like, everybody should follow them. And I thought, really, you should know. You shouldn't follow them. You should know them. You should know them because they inform how you don't follow them. And if you don't follow them, you should know why not?

Yeah, that's perfect advice and similar to some of the things that I tell my own students. So maybe we can start by. I would like to dive right in into some of your projects, so maybe we can start from the work you and your students have been doing on sketching. I think that's a very interesting, very interesting direction. And I think you've been studying how sketching works and how it relates to data visualization. So can you talk a little bit about it?

Yeah, that's perfect advice and similar to some of the things that I tell my own students. So maybe we can start by. I would like to dive right in into some of your projects, so maybe we can start from the work you and your students have been doing on sketching. I think that's a very interesting, very interesting direction. And I think you've been studying how sketching works and how it relates to data visualization. So can you talk a little bit about it?

Yeah. Okay. So, well, one of the things that I did very, very actively do when I first became a professor, like, really encourage everybody to sketch, and we didn't kind of look at it formally. We sometimes. I know. You know how in art school you'll have, you know, a life drawing night where everybody gets together and just practices drawing? Well, we've sometimes had, like, data sketching nights where we have a small bit of data and we all get together and sketch it. Right, just to kind of, whatever, limber up, keep your mind going, thinking of new ways of doing things. And then at one point, we thought, you know, we actually should, like, run a study about this. We should actually look at how people sketch and how people sketch who aren't necessarily infovis people. Right. So we ran a more formal study, and we gave them one of my favorite little data sets. I think it still exists at the back of SPSF. So if you have that software, you also have the data set. It's about manners. It's about whether it's polite to, like, burp in a classroom or in a church or. So. Yes, exactly. So it's kind of nice because simple manners like that, you can think of everybody as an expert, so we don't have to worry about finding data experts. And it's also, it's amusing. So people are working with this data set. You hear them chuckling to themselves. Right? So people drew us all kinds of fabulous sketches. And. Yeah. So one of the things in analyzing what we got, one of the things that we did is look at how numeric some of these sketches were. And, you know, how some of them got increasingly abstract. So all the way to, some of them were actual comic strips. And I know that when I first picked up one of the comic strips, my thought was, I guess this person was not particularly into the data. You know, I mean, it's very nice. They drew us a comic strip, okay? But then we did another. We had also asked people if they found anything interesting about the data to just tell us about it. And people wrote us quite a lot. Everybody wrote about half a page and some more. And we characterized the kinds of things they told us. And, you know, they ranged all from the very particular, like, one person saying, I can't believe that anybody rated it as non zero to fall asleep in a job interview all the way to people kind of making sort of suggestions of things like, you know, things that could lead to future research, some sort of little loosey goosey conjectures. Right. And the interesting thing, and now actually, so this I have to be careful about on a podcast, is there was nothing. This is not, this is a qualitative study, so this is not a correlation. This is just something that would be, again, like, worth exploring further. The people who drew us comics, they also had really interesting things to say about the data. Quite often, some of the more profound things. So in drawing us comics, they hadn't just blown off the data, they had actually thought about it pretty deeply. So that was really interesting and has actually fed a whole new stream of things that I'm doing with Benjamin Bach and Nathalie Henry Riche on data comics.

So this was not just a user study, but actually for yours, an idea, like an idea generation tool, basically.

Oh, I think actually all the best studies are generative.

They make an end point.

They make you think about something and then you know what you want to do next.

Right? Yes, they make you think.

But usually people confront others with a finished product or two of them or three, and say, like, do you prefer this or that? Or watch them interact with, like, something made already, but you turn that on its head basically by saying, now you make something and then we talk about what you made.

That's right. And I find that so much more interesting because so to me, you know, one of the things that's fascinating about this type of work is I am learning all the time and I can learn from everybody. All of our participants are, you know, sometimes they're, you know, it's mind blowing. So, yeah, I think it's. I do actually think it's a really important distinction to call them participants because I think that they are, they're giving some of their time and, and some of their effort, and it can be totally inspiring.

Yeah, yeah, yeah.

I think that seems to go through all of your work is like this experience aspect of, okay, there is actually somebody who is creating this graphic, but there's also somebody experiencing the visualization. So I think in all of your projects, there's always this, this component quite present. If you just scroll through like your huge project list on the website, I think there's always this, this aspect there and really thinking about, ok, what is the user experience of data? Maybe that also ties into another topic you've been working on quite a bit, is this idea of active readings, of visualizations that also the consumption can be an active process. Right?

I think that seems to go through all of your work is like this experience aspect of, okay, there is actually somebody who is creating this graphic, but there's also somebody experiencing the visualization. So I think in all of your projects, there's always this, this component quite present. If you just scroll through like your huge project list on the website, I think there's always this, this aspect there and really thinking about, ok, what is the user experience of data? Maybe that also ties into another topic you've been working on quite a bit, is this idea of active readings, of visualizations that also the consumption can be an active process. Right?

Yeah, I think that we are really familiar. We have now started to become cognizant of the fact that visualization literacy is not automatic, not for everybody, and that maybe we have to actually think about this. And so one of the projects we did, again, it was the same thing. We gave them a very simple social network about who was friends with who and asked them to do things like, you know, arrange a seating for a dinner party so there wouldn't be conflicts. Right. And just let them do as they will. And they naturally used all kinds of simple aids of, you know, writing on top of the visualization, making notes on some other piece of paper, various kinds of ways of helping themselves figure out what they wanted to do. And this is very akin to what is taught as active reading of text. So an active reading of text, they will sort of help people. Like, okay, first look for, you know, the nouns of the sentences, or first, you know, like, they will give people ideas of how to actually get a deeper reading of the text. So, yes, so we were. We were interested in what people naturally do and then thinking about whether we can actually support that technologically. So we just took one of the kind of more encompassing ideas of what the people did in our qualitative study and provided a digital tool with free form annotation, which is a simple thing for us to provide digitally. And people certainly used it, and they actually were slower because I think they got a much more in depth understanding of the graph. So that, again, is conjecture. They were much more accurate. So I think it really did help support their thinking process, and I think that it's something that we should think about in trying to help people read visualizations.

So, Sheelagh, so it's really hard to cover everything here. So I would like to briefly cover one of your projects that is more about developing some visualizations. There are so many that I've seen in your lab, it's hard to pick and choose. So maybe we can talk about the one about visualizing northern lights with your scientific laborator. I think I was really impressed by that one. Can you describe it for our listeners?

Okay, so we worked with Eric Donovan, who studies. He's an astrophysicist, and he studies the northern lights or the aurora borealis. And he has actually a really active website called Aurora Max. So we wanted to work with his data in a kind of outreach way. So, I mean, sometimes when you're focusing on data, you're focusing on trying to expose new factors in the data for scientific insight. But this time, we were focusing on making the data kind of accessible for people in a public space, like in a science center or in a gallery. So we made what are called kyograms. And that's a word that actually Eric was already using with his group. It comes from an inuit word, but essentially what it is, is like, for the whole, he's got cameras all over the north straight pointing into the night sky and taking pictures of the night sky and the aurora happening in all its glory. And it takes a strip through that and just one, like, one pixel for each frame. Of the video. So you kind of have, like. Yes, a quick access version of the. Excuse me, of the video. So we could actually take these Kia grams and line them up so that essentially you have a night sky histogram. And you can see all kinds of things in the night sky histogram. Like, you can see when the full moon was, because the sky is actually quite a bit lighter. You can see where the aurora was active. You can see how the nights are really long in the middle of the winter and get shorter as you come towards spring. You can see all kinds of things just from the histogram. And because, of course, the data itself is so intrinsically beautiful, so is the histogram. But you can actually. You can touch one or click on one and it will open up. And you can see that particular video. And you can use the Kia gram to control which part of the video. So you can go and select the most active parts. One of the things that we did with this that was also pretty cool is we used proxemic interaction. So that is that as a person was walking by, the sensors would sense their motion and respond to them. So we have, like, videos of people walking by and out of the corner of their eye, they see the display responding to them. And you can always tell when it's related to your own movement. And it would actually, like, invite them. And so that, you know, when they moved up to it, it could show them that they could open it up and actually control the access themselves if they want.

Yeah, it's nice because for these natural phenomena, you also often need to be in exactly the right spot and do the right thing to even be able to experience them. So it's nice that you translate that in the interaction with the actual piece.

Yes, because the chances. I mean, I know that people. I mean, here one does see the northern lights quite a lot of. Yeah, it's quite amazing and wonderful. But sometimes somebody will come, and they will come during the times of year that normally you would get to see it, and it just doesn't happen.

These are beautiful. Like, these rearranged images are really beautiful. Is it related to slit scan techniques?

Yes, it is.

In fact, it sounds quite similar, right?

Yes, it is quite similar.

Yeah, it's beautiful. And I love this, taking original photo material, but then sort of warping it in that way that suddenly new patterns, like, in this case, time is being mapped onto space, and suddenly you see the same thing, but in a totally different way. It's like flipping it 90 degrees, basically. It's like whoop. What happened there? But you still see the original material in the plots. It's very nice.

Yeah. I would strongly encourage our listeners, if they can, to stop and take a look at the images of what we're talking about in our show notes because it's very hard to describe with words. Right. But it's stunning. It's one of the things that, like, when I saw it in your life, it made me pause and it's just, just beautiful. And I think you also briefly recounted the story of how your collaborator reacted the first time you saw it.

Yeah, he was kind of, I mean, it was quite, that was quite wonderful because, I mean, yeah, I knew I wasn't like, normally when you're collaborating with a scientist, you are actually trying to help them further science. And this time we had actually decided to take a different take on it and that was actually really satisfying that he found it. That he found it very moving.

That's great. Best possible outcome.

Yeah, exactly.

Maybe we can talk a bit wider. Again, fantastic projects. There's like 20 on the website. So you're very prolific person, but you also have great collaborators, I think, and you seem to have a good way of running your lab. It seems like there's so much great output there. And as Enrico said already, it's often very like out of the box lateral thinking, especially compared to the rest of information visualization research. Maybe we can talk a bit about like, in your view, what are some of the, maybe the myths or some of the beliefs that everybody has about data visualization that seem to hold us up in being actually creative and innovative with it? And how can we sort of break these chains?

That's a very interesting, difficult question because it's hard to know. I do think that what we were talking about at the beginning is trying to actually set yourself up to. To always be open to noticing the distinctions, the little catches where you might actually be able to do something different and be willing to investigate those, to giving yourself space for ideation, I think that's actually really important. I think, you know, I do find like the integration of design thinking with infovis as being like super rich. What else can be specific about that? I think, you know, we have a tendency in the community to, well, maybe actually step back a bit. And one of the things I think that we have to recognize is that this business that we are trying to do, which is to take some data which has a kind of probably fairly, you know, simplistic, perhaps just numbers, or perhaps it's also numbers and text. It may be more complex, but it has a representation. And what we are actually trying to do is come up with a new representation. And this is non trivial thinking, not an easy thing. The easier thing is to actually look at existing representations and trying to figure out, like, how to work with them. But actually being able to think fresh is something that I think we should value more as a community. And I don't think. I think we've got a little bit stuck. There were some, you know, great innovators around the development of the printing press, and they came up with all of our standard visualizations and we still mostly use them. They've been around for some 400 years.

The 19th century was maybe even more innovative than we are now.

I would say in many ways, yes, but I think that we have to recognize that we actually also, they had the printing press and that was pretty cool and they really made good use of it and they did new things. Right. We have a computer, this is also pretty cool, and we can do new things, but a lot of those new things are in spaces that people don't expect. Well, like interaction. So we were just talking just before about, you know, incorporating proxemic interaction into visualization. I think one of the things is to value more how bye changing how something responds to you, that you're actually changing how you can understand it. I think to some extent, you know, all of the talk we did about active reading that was showing that that give people a bit of space, that they will be active with it and that. So I think valuing changes in interaction more is something important to do as a community. I also think we have got really chicken about 3D. There's all kinds of stuff about never do 3D.

I would say in many ways, yes, but I think that we have to recognize that we actually also, they had the printing press and that was pretty cool and they really made good use of it and they did new things. Right. We have a computer, this is also pretty cool, and we can do new things, but a lot of those new things are in spaces that people don't expect. Well, like interaction. So we were just talking just before about, you know, incorporating proxemic interaction into visualization. I think one of the things is to value more how bye changing how something responds to you, that you're actually changing how you can understand it. I think to some extent, you know, all of the talk we did about active reading that was showing that that give people a bit of space, that they will be active with it and that. So I think valuing changes in interaction more is something important to do as a community. I also think we have got really chicken about 3D. There's all kinds of stuff about never do 3D.

It's like a burnt field. Nobody wants to go there.

This is where the opportunities are, right?

Well, some. Not all, but some. But what I actually think is that, you know, we've been doing 2d for hundreds of years, right? So we've got moderately good at it. And actually, I have to admit we kind of suck at 3D. But all of the stuff that we're doing in physicalization and all the stuff we've done on constructivism shows that actually there are lots of things about 3D that are really useful for people. So I think it's a question that we haven't got it right. So I do think that we make lots of mistakes. It does seem that we more naturally get it right when we do it in physicalizations, which is cool and super exciting, but that we shouldn't be so scared of trying to actually do it right in three D. I think this.

All still plays together. Also with that point you briefly mentioned before is this idea that, well, reading a visualization can be a very, like, can take a while and can be a very active process and sort of nonlinear. And often there's this idea that every single data visualization needs to be understood immediately, like within split seconds, and otherwise it's a bad chart, you know, and then there's this whole literature around, like how to make good charts, not the bad ones. And I think this sort of forgets that maybe depending on the context or the purpose, any time you need to understand something can be totally fine, or it can be even rewarding to spend some time with something and not figure it out in the first split second.

All still plays together. Also with that point you briefly mentioned before is this idea that, well, reading a visualization can be a very, like, can take a while and can be a very active process and sort of nonlinear. And often there's this idea that every single data visualization needs to be understood immediately, like within split seconds, and otherwise it's a bad chart, you know, and then there's this whole literature around, like how to make good charts, not the bad ones. And I think this sort of forgets that maybe depending on the context or the purpose, any time you need to understand something can be totally fine, or it can be even rewarding to spend some time with something and not figure it out in the first split second.

In fact, I actually often talk to my students about how one of the best and most brilliant visualizations that has ever been invented is the Alphabet. It visualizes our spoken word and my goodness, it is complex and it's weird.

Takes years to learn.

It takes years to learn. And we were so lucky that one of our members of our community came up with something as brilliant as this. Nobody would understand it in 20 minutes, right? I mean, I think that there is a difference between ease of understanding and power of understanding in that we need to think about the power side as well and that, you know, everybody makes their child learn how to read and learn the Alphabet because they know how it's going to empower them through their whole life. I mean, yeah, I doubt that we are going to come up with something like that in our community, but we should not preclude the possibility because we want everything to be understandable in 20 minutes. So I think that there is, I guess, I mean, yeah, I guess my thing is like, I think we need to value interaction more. I think we need to think about the power representation and looking at what we can do to empower people. And I think that we need to have a better understanding of how personally empowering and how expanding it can be for a whole research area to incorporate really good qualitative research.

Yeah, yeah, I think that's very important. And I'm looking forward to seeing what else is coming up in the, in future years. I personally myself, learning a lot about how to do more qualitative research, how to introduce more interaction, and also, yeah, look beyond these boundaries that we set for ourselves. I think sometimes they are a little too strict.

Yeah, I mean, I think that really we're at the beginning of, it's only been whatever, 20, 2025 years that we've been doing visualization on computers, and computers are still changing themselves. The potential and the power that we have available is amazing. And we need to kind of, I don't know, revel in it. Sure.

Okay. Sure.

Yeah.

So much potential. But it's also to get on the ground. You know, it's like, I think there's often this also this innovator's dilemma that you might have something better, but is it so good or so much better that people switch over to it? Right. And I think that often this crucial point is like, does it reward you so much that it's worth the effort of breaking out of your comfort zone?

Well, yeah, and I don't think it necessarily needs to for us. So I think that we need to be willing to think about things that are different just because they're different and we're trying to actually understand. Well, maybe, you know, the first time I do some new thing and it needs some more exploration and it needs to get out there because maybe one of you is going to actually say, oh, yeah, well, that's kind of cool, but really, you should have done it like this. Right? And then we think, oh, yeah, now. But we need to be able to share with each other our beginning attempts that are maybe not as polished as something that's been around for many, many decades. Right? Yeah.

Oh, absolutely. Yeah. In that sense, let's keep the dialogue going and flowing. I really like what you're doing there for bringing these often so unconnected fields together, and not just by talking about it, but by actually doing it. I think that's a huge value and a huge achievement. And we're really excited to see what you will be doing in Vancouver. Maybe we to get you back on in a few years to see how that switch has played out. It's very exciting. I think that you're moving there and, yeah. Thanks so much for joining us.

Okay. Thank you.

Thank you, Sheelagh. Bye bye.

Bye bye.

Hey, folks, thanks for listening to data stories again. Before you leave, a few last notes, this show is now completely crowdfunded, so you can support us by going on Patreon. That's patreon.com Datastories. And if you can spend a couple of minutes reading us on iTunes, that would be extremely helpful for the show.

And here's also some information on the many ways you can get news directly from us. We are, of course, on twitter@twitter.com. Datastories. We have a Facebook page@Facebook.com. datastoriespodcast. All in one word. And we also have a Slack channel where you can chat with us directly. And to sign up, you can go to our homepage datastory eas, and there is a button at the bottom of the page.

And we also have an email newsletter. So if you want to get news directly into your inbox and be notified whenever we publish an episode, you can go to our home page Datastore es and look for the link you find at the bottom in the footer.

So one last thing we want to tell you is that we love to get in touch with our listeners, especially if you want to suggest a way to improve the show or amazing people you want us to invite or even projects you want us to talk about.

Yeah, absolutely. And don't hesitate to get in touch with us. It's always a great thing to hear from you. So see you next time, and thanks for listening to data stories.