Visualization on Mobile & Touch Devices w/ Dominikus Baur

Hi, everyone. Data stories number 25. Hi, Moritz. How are you?

How are you doing? Good.

Summer is here. Summer is here. Maybe it's not in Germany. It is. It is still a bit cool.

Sometimes raining, but mostly. Yeah, mostly nice. Yeah.

What's going on?

I'm wrapping up things. I'm off to leave for vacations. Yeah, the day after tomorrow.

Oh, I'm jealous.

So I'm in this sort of limbo state now of. I have to finish everything.

Yeah, wrap up.

I couldn't finish everything.

I already gave up having this kind of feeling. I mean, yeah.

I have actually two to do lists and I don't tell people on which to do list. Their stuff is. But.

Yeah.

And the one is looking good and the other one. Well, what can you do? Right?

Yeah. I am myself in a very risky situation because for holidays I'm gonna stay here in New York. So it's, you know, I mean, if you don't leave and go somewhere else. Yeah. It can be very nice. But there is always the temptation to go back to work for any given reason. I hope that's not gonna happen.

No, just be disappointed.

I'll do my best. I just trash my laptop somewhere.

Just get drunk.

Do they sell locks for laptops? Yeah, you could like.

Yeah. Lock it away for a week. Like secret code. Yeah.

Give the key to my wife, to your lawyer, the best person. No, I think my wife would be much, much harder to convince than my lawyer.

Yeah, it's a good idea. I actually go to a hut in the mountains. Like, I don't think they have wifi.

Yeah, I don't think. Well, you never know. Have you ever seen those map where you. Basically. I think it's a map of the few places that are completely unreachable. There are very few areas around the.

World, like no cell phone.

No cell phone, no signal at all. I mean, there are very few ones. Very, very few, unfortunately.

Probably some bunkers somewhere in Russia.

Some bunkers. Yeah. In Switzerland, there are some bunkers as well. When I lived there. There are many.

We could go there.

Yeah, there are many, many around. Anyway, let's start these episodes. We have another fantastic guest today. We have Dominicos Bau and we're going to talk about mobile and touch interfaces for visualization. Hi, Dominicos, how are you?

Yeah, there are many, many around. Anyway, let's start these episodes. We have another fantastic guest today. We have Dominicos Bau and we're going to talk about mobile and touch interfaces for visualization. Hi, Dominicos, how are you?

Hi, Enrico. I'm good, thanks.

So Dominicos is an expert in this area. If you didn't hear about him yet, I suggest you to go to his webpage. That is dot minic, dot as. And it's a very nice URL. But you will find it in the blog post. And he gave a very nice presentation at the Openviz conference on data visualization on mobile and touch interfaces, and we decided to invite him. Plus he's a very old friend of us, so it's a great pleasure to have him here in hosted in data stories. So, Dominicos, you want to tell something about yourself, introduce yourself a little bit longer or more details than I did.

So Dominicos is an expert in this area. If you didn't hear about him yet, I suggest you to go to his webpage. That is dot minic, dot as. And it's a very nice URL. But you will find it in the blog post. And he gave a very nice presentation at the Openviz conference on data visualization on mobile and touch interfaces, and we decided to invite him. Plus he's a very old friend of us, so it's a great pleasure to have him here in hosted in data stories. So, Dominicos, you want to tell something about yourself, introduce yourself a little bit longer or more details than I did.

Sure. Yeah, I can do that. So I'm Dominikus Baur. I work as a visualization freelancer in Munich at the moment. And I also do a lot of interaction design, mostly on mobile. And so the thing that I'm most thrilled about at the moment is touch and how to combine touch input with information visualization and data visualization. So actually, I haven't been doing this freelance thing for too long. I was in academia for a while, so I did my PhD in information visualization and I started working on this whole touch topic there. But I also did more exotic things about live vlogging. So tracking details of your life, like music listening histories, for example, and trying to make sense of that by doing visualization, that was kind of my thing.

I remember your stuff on music visualization. That was, that was really cool. It was quite some time ago, right? When did you do that?

Oh, so I started my PhD in 2007. Five years ago.

Yeah.

Oh, that's. Time flies.

Yeah. Never look back.

Yeah. No, that was a lot of fun, this whole music listening stuff, especially as people could relate. So most of them were thrilled when they heard, oh, it's possible to track all the music that I listen to. That's awesome. I want to do that.

Yeah. That's actually the fantastic thing of visualization, of personal data that people can relate to it in an instant. And that's really, really cool.

Yeah, definitely, like a lot.

So, Moritz, we have an imbalance of German accent today. We didn't invite any Italian yet, so we have to. That's true.

Yeah, yeah. But at least it's euro, you know, at least.

Yeah, it's gonna be quite exotic today.

We're all a big family and if you're from Europe.

So. Dominicos. So why don't we start discussing about why doing visualization on mobile devices at all? I mean, I think that's the main question. I will start from that. So what is the need of moving visualization on a device that is mobile, actually?

So. Dominicos. So why don't we start discussing about why doing visualization on mobile devices at all? I mean, I think that's the main question. I will start from that. So what is the need of moving visualization on a device that is mobile, actually?

So I guess the reason is that mobile is becoming more and more popular. So I think this year we actually reached a point where people spend more time using their mobile devices than using their desktops and laptops. So. Well, I guess it's just a sign of the times you can't really avoid it. And of course, the thing is, if you have something online, for example, and you want people to see that, then you need to be able to support them even if they use a mobile device.

Oh, sure, sure.

Yeah.

Makes a lot of sense.

Sure, yeah. So, because, I mean, I guess all of us are, I don't know, checking out Twitter feeds on our mobile devices, and if there's a link that doesn't work on our smartphone, then we don't really go to our laptops and check it there, I guess. At least I don't.

You don't type in the URL you're just reading on the iPhone?

Nope. Usually you don't. So there's already one person that you've lost, so.

Sure.

So what do you say? What's the main difference? Or is it the same thing in small?

Not really. I mean, the other thing that's very different for mobile is, as the name implies, that the whole context is changing. So the usage context is. Can be completely different. I mean, people can still sit on the couch and use their smartphone while watching tv or something, but mostly they, I don't know, they're waiting for their bus or whatever or walking around somewhere waiting for friends. So they're in a completely different space than people who are sitting in front of their laptops and. Yeah. That makes it difficult to address both scenarios, I guess.

Right, yeah. And it's very small devices. I mean, that's the other thing. Like, we're very used to making like, very dense representations, lots of numbers, little.

Sure, sure.

And, yeah, then you just have a few inches to work with.

I guess. That was actually one of the reasons why a lot of researchers avoided mobile devices for a long time. Just because it seems like, okay, for visualization, we want to have as much screen real estate as possible and fill everything with data. Why would we restrict ourselves to something that is only four inches?

Yeah, it seems like a step back if you just look at the display. Right. But I think you're absolutely right that it opens all these very interesting contexts, like different locations where people are new life situations. Mobile is also a very personal device. It's much more personal than personal air quotes, computer. And so that's kind of interesting. Yeah, yeah. But I also find it hard. It's. Yeah, if you're used to building, like, desktop visualizations, it's hard to wrap your head around, like, how to do it right. On mobile, what's your experience? What are the biggest, like, what are the things you have to learn when you switch from desktop to designing for mobile?

Yeah, it seems like a step back if you just look at the display. Right. But I think you're absolutely right that it opens all these very interesting contexts, like different locations where people are new life situations. Mobile is also a very personal device. It's much more personal than personal air quotes, computer. And so that's kind of interesting. Yeah, yeah. But I also find it hard. It's. Yeah, if you're used to building, like, desktop visualizations, it's hard to wrap your head around, like, how to do it right. On mobile, what's your experience? What are the biggest, like, what are the things you have to learn when you switch from desktop to designing for mobile?

I guess the major difference is the input modality that we have on mobile, because for a desktop, well, we have mostly a touchpad or something like that, or a mouse. So some pointing device and of course a usable keyboard compared to mobile. I'm always frustrated by those virtual things. And I mean, that's kind of to add insult to injury, not only do we have a smaller screen, but we also have huge fingers that occlude content on there. So I guess this whole touch input is the major difference, really.

And of course, the screen size at the same time. My feeling is that being able to touch a visualization virtually is also a sort of better direct manipulation. Right. So I'm sure that there are lots of limitations, but there might be also some advantages. Did you notice that? Do you have any experience with that?

So direct manipulation is really key in this regard.

I mean, in this case, it's really direct. Right. You are really kind of like touching the thing.

It actually is direct. So when the term was originally coined, the whole direct manipulation happened with a mouse, which is, well, kind of ironic, but yeah, now we're actually directly manipulating stuff on screen. And I would say it feels different than using a mouse, especially if you're able to use more than one finger, which is really rare still. But I guess we're getting.

Everybody remembers the first time they used Google maps on a tablet, right? Not really. I mean, for me at least, it was like, oh, man, I can totally move this map sounds so trivial.

I mean, think about zooming. Zooming on an iPad is so much easier. I mean, it's amazing.

It's fun, right?

And it's fun. And it's fun. Or even rotating. Think about rotating with a map.

You move two fingers and, you know, they stay where they are. And the photo or the map, like rotates according to what you do. I mean, yeah, before it was always just like much more discreet actions like Zoom in, move left.

Yeah, exactly.

Yeah, pretty much. And you had some labeled buttons that did something. And now you actually do. You directly do the thing that you want to do and it just happens. You don't have to look for the button that says zoom in, but you just use your fingers.

It's like really like acting in a space as opposed to executing command. Yeah, to some degree, yeah.

Of course, the downside of that is that more abstract commands are more difficult to pull off. You often find that people have some pretty much arbitrary gestures to do something. So swipe up with three fingers to do x and then you have to memorize that. Yeah, it's a fine line.

Yeah. And I guess the screen real estate is reduced by the fact that you need large areas to let people interact with the user interface, right?

Yeah, usually you do. I mean, the advantage of having gestures is that you don't have to have really discreet areas that you can interact with. So when we think back to the, to the maps example, you can use your fingers on any part of the map to do something with it. So you don't have specific interaction areas, but. Yeah. Well, sometimes you do. And of course, it always depends on the type of application that you want to do.

Sure.

The other thing are, of course, sensors. So you can also work with how the device is positioned or where you are in the world and where you pointed away. So you had this really interesting project where you would use a smartphone as a sort of a lens on a projected display.

You mean this virtual projection thing? Yeah, that was interesting. So basically, we recreated kind of this notion of optical projection by tracking the, the smartphone in front of a large display, and then you could use your virtual projection cone to interact with the stuff on the large tv.

So you would point your smartphone somewhere and it would be as if the smartphone would emit light and project something.

Right, exactly. Yeah, yeah.

But it didn't.

No, of course it didn't. That was actually the first thing that people always asked. How do you do that?

How did you get the projector in there?

Yeah, but no, it was all just simulated.

What happened to augmented reality and layer and all these types of applications? Is that still a thing or is that like in the valley of. How is it called? The value of despair?

I actually think it's coming because, of course, I'm also saying that about touchable visualization.

For years now.

Yeah, I mean, now we have reached a point in the hardware development where it actually makes sense to do augmented reality because you have all these sensors and powerful mobile computers. I don't know if you guys seen the first augmented reality prototypes. That was always a huge 30 kilo backpack and, yeah, it was pretty ridiculous. But now we have all this power just in a smartphone.

Yeah, but it's interesting because two years ago it made big headlines and now we actually have the power. But now it doesn't seem to be very present, but it might just come in a half a year and then be really integrated in product.

Maybe. Yeah. I mean, there is stuff like Google Glass coming up, for example, and that is.

It's pretty organized.

It was very promising and scary, of course.

Yeah. Yeah. So I was actually thinking about your work on your. I think it's called touchwave. Right. I mean, so where you basically, if I. If I remember correctly, you. You've been implementing some specific kind of new interaction techniques for the tablet that actually solves or try to solve some of the limitations of stack graphs, right?

Yeah. Yeah. So I was actually thinking about your work on your. I think it's called touchwave. Right. I mean, so where you basically, if I. If I remember correctly, you. You've been implementing some specific kind of new interaction techniques for the tablet that actually solves or try to solve some of the limitations of stack graphs, right?

I. Yeah.

Is that correct? Do I remember correctly?

That is correct, yeah.

So can you tell us a little more about what you've been doing there and what's the outcome of this work?

Sure.

That's really interesting because I think it's a nice example of how using touch interfaces can actually be. Can provide some benefits that we might not actually get by using standard interfaces. Standard devices, actually.

Yeah. Yeah. Thank you. That was kind of what we were going for. So Touchwave was a research project that I did last year at the University of Calgary with Sheelagh Carpentale and Wang Shen Li. And the basic idea was we wanted to use or to show how interacting with touch is not necessarily a downside or something that you have to design around, but can be something that actually enriches interaction and makes certain visualization techniques usable. Because, I mean, you often see that you have some visualization technique that doesn't really work that well in its static form, and then you tack on interaction to make it better.

Yeah, I think in general, this is a huge topic which goes very well beyond the problem of touch interfaces. It's really interesting. Yeah. Sorry for interrupting.

No, no, that's fine. Yeah, so they kind of went into the same direction. So we wanted to have something that's pretty. So we took stacked graphs because they are. But they're just. Well, they. I mean, there's this huge discussion going on about how useful they actually are as a visualization technique. And I don't want to go into that, don't want to make any enemies, but we said that, okay, we want to have stack graphs on an iPad. Now, what can we do using touch? And the basic idea behind touch wave was to have the visualization as a virtual object. So going back to this direct manipulation idea, to have this visualization as something that is behind this little layer of glass there and that you can manipulate using your fingers. And based on this metaphor, we came up with certain interaction techniques and stuff that became possible by using your fingers so you could take single layers from the stacked graphs and drag them out and use, of course, multiple fingers to zoom in on certain areas and kind of squish and distort the whole thing. So, yeah, that was, of course it was just a research project, so can't really buy it. But it was an interesting experiment in trying to harness the power of touch for visualization and I think that could definitely be something that we might do more often in the future.

Sure.

Because right now, I mean, there's not a lot of visualization happening on tablets, unfortunately.

So do you think that there are some potential interaction mechanisms on top of some visualizations that are too costly when performed on a desktop device and become. If you move the visualization to a touch device, they become much less costly and feasible and then they become much more useful. You see what I mean?

No, not really. Sure. No.

I try to reformulate the question. It's probably my fault. So if you think about new interaction techniques in visualization, you can potentially develop a new visualization. Sorry, a new interaction technique for manipulating visualizations that doesn't necessarily depend on which device you use. Right. Kind of like I think now you've been mentioning taking one layer and moving this layer off to another, to another area of the screen. Right. So in principle you can do that with the mouse as well, but it's probably too costly. Right. So I think nobody did it before because just the. It doesn't make sense if you do it with a mouse, but if you do it with your hand, all in a sudden it becomes feasible and it makes sense.

See what you mean? Yeah, yeah, that's definitely the case. So, I mean, just the whole pinch to zoom thing that became so iconic. Of course you could enlarge and shrink photos before that, but it just wasn't as much fun because you had some plus button that you had to click on. So, yeah, that was definitely also a part of the project that we could just go crazy with all of these things that suddenly became much more feasible, as you say, and also much more fun. So fun was definitely an aspect of the whole project.

Yeah, I mean, the other thing is really multi touch and it's hard. I think we're all very much trained on this one pointer paradigm, but there's really many things you can do with multiple pointers, like what we've seen also with the data sculptures that you can mark, for instance, with your left hand, one part of the graphic or the sculpture, and then interact with the other, but still keep your index, like your index finger to mark just one position, stuff like that, which you wouldn't do on screen because you leave ugly traces and it doesn't fit to your mouse interaction. But suddenly on a tablet it can make total sense to say, I'll use the left hand just to fix something and keep it in place more or less interacts with the right hand, stuff like that.

Yeah, yeah. Especially if you know that the visualization that you see on the tablet will somewhat behave like a physical object. So that also makes stuff like discoverability and learnability and so on much more, much easier. It's always nice if people know or if nothing unexpected happens in an app to put it like that.

Nothing breaks.

Yeah.

How do you feel? So I feel like on mobile it's a bit harder to do very exploratory visualizations or very complex ones, but you have lots of filter settings and different options.

You just told my question mark, how.

Do you work with that? I mean, is it just because we are not so used to do complex things on a mobile? Orlando, should we just like keep things a bit simpler and a bit more, let's say, put things, let's rather produce a series of five simple charts and put them in a nice sequence instead of producing that big data visualization. What's your take on that? Is it like visible difference or just.

So. I think that when we think back to the whole context thing, so if I'm waiting for a bus somewhere and I have two minutes to kill, then maybe I don't want to go into the full scale data analysis that I had planned for weeks. So I guess it's definitely better to do complex analysis on a laptop, on a computer, especially as the screen is bigger and so on. So mobile definitely has its place, of course. But I think it's more relevant for more of these small scale visualizations that are heavily focused on presentation, for example. So the stuff you see a lot in journalism, for example, where the actual message is clear and you can explore a little bit, but not too much.

But at the same time I'm wondering if there is a space for some sort of analytical kind of tasks performed with this kind of devices. I don't know. I imagine people working in the field or have you ever heard anything like that? I'm just curious because I'm not familiar with anything similar to that. But I guess in principle there might be people around who actually need to do some sort of analytical task on the field. And having mobile devices with some sort of visualization there might help. I don't know. Have you ever heard of anything like that?

Yeah, definitely. There are these things, of course, you don't hear too much about them because they're mostly specifically developed for a certain use case. So you have, I don't know, engineers for example, that are working on something like on a ship or so. But I guess even for regular people it might be interesting if we think back to the whole live vlogging stuff. So if I want to look up something while I'm on the go. So imagine you have this great song that's stuck in your head and you can't remember the name and you want to look it up to tell it to your friend. I mean that's somewhat an analytical task. It's very small, but just how to.

Book a flight or something, you know.

Or something like that.

It's a complex task too. I mean it's really hard to book a good flight. Right. And I'm always frustrated then if the mobile application just, you know, offers a list of three flights and a big buy now, but, and then I have to go back to the desktop actually and you know, use kayak or hipmunkers or something like that. And so. Yeah, and I think there could be really nice applications for also these, let's say, halfway complex tasks.

Yeah. And I also think, as you mentioned, the whole area of life logging and also this trend, what is it called, quantified self. The idea of monitoring yourself is probably very interesting there. Right. You might want. So as soon as you have reached sources that are monitoring aspects of your own life, you might want to be able to check that with some sort of visualization on a regular basis. Right. And then a mobile, mobile visualization would work really well there. Right?

Yeah. It's a lot of data that you can collect about yourself, especially if you do it automatically. So visualization is usually the way to go, I would say.

Yeah.

But again, wouldn't you use the mobile to collect it? But then the desktop probably too.

Yeah, but you might want to have access to your data on the go as well.

Yeah.

So I think that's the use case. Of course, if you want to do something more complex like, I don't know, comparing your nutritional behavior to your health or whatever, then you will probably use your desktop for that.

Yeah, but I think that's all things. The whole idea of monitoring yourself, I think in order to be effective you have to have some sort of steady kind of feedback. So I think that there will be a need for visualizations on a mobile device that give you a regular access, continuous access to what's going on there. Right.

And I mean, and the other thing is, of course you and your surroundings like I'm in the city, you know, what's around me, where are my friends, all these types of things. That's huge. And I think none of the, let's say traditional visualization people have that on the radar even because it's not a use case for your desktop. Really strong. The use case is not as strong as on the mobile.

Yeah. That's one really interesting thing that I haven't really seen so far, location based visualization. So filtering your data based on where you are at the moment, for example.

That would be pretty cool and stuff like that.

Yeah. So the map applications are far beyond that. Yeah, of course maps are visualizations, but in some way. No, they've been around for so long, little niche area, nobody uses them anyway.

No, no.

But now I'm thinking that mobile devices could imprint. I don't know if anyone is already using that for this purpose, but in principle they can also have some sensing devices on, attached to them. Right. And if there is anything that you need to sense from the environment, then you might need to have some sort of visualization to. To display whatever it's sensed. Right. I mean, I'm reasoning on a very abstract level, but I think the smell of gram. Yeah, the smell of. I was good to say something really bad. Yeah, whatever. I don't know. I mean, why not? Why not? At least from an artistic, for artistic purposes, that would be nice.

Sure.

Do you remember those glasses that they used to, I don't know if you had that in Germany, we used to have that in Italy that you could buy these glasses that allegedly they allowed you to see. To see through clothes. I don't know.

Yours work? I don't know.

I never tried, but I remember when I was a kid, this, this kind of glasses, they were sold in magazines.

All your pocket money on the.

Yeah, yeah, yeah.

Now it's an app on the app store.

Buy it.

It works.

Sure. I should actually totally works. You tried? Of course. Yeah.

I don't know. Yeah. Interesting.

Yeah, very interesting.

But it's. Right, I mean, I haven't seen that much really great visualization on mobile device. Like occasionally a nice app or a nice little trick, but it's probably very early, right.

I mean, visualization itself is very young visualization on mobile touch devices. It's really, really young. Right?

Yeah. And there's also a technical barrier.

Yeah. So Dominicos, what's the coolest thing you have seen on the coolest visualization you've seen on mobile devices apart from yours except your.

Yeah. So Dominicos, what's the coolest thing you have seen on the coolest visualization you've seen on mobile devices apart from yours except your.

Oh man. I wanted to say that. So actually, one of my personal favorites is the Datum app. So the companion app to Nicholas Felton's live logging service. So I think that is really well done. And he did that, I don't know, two or three years ago. Even. So, it's really old as well for app store lifespans. So, yeah, that is really neat. It uses visualization to present your data and you can even explore it a little bit. Of course, it works better using a browser and a laptop, but that is really nicely done.

He has a new app now, but I don't think it's public. But he has a reporter app that nags him every 15 minutes.

Yeah, I heard that. So he's still teasing that it's supposed to be come out in fall or so.

I don't know if it's a mass product.

What is that?

Maybe he's the only one who wants that.

It nags you every 15 minutes to enter some data. Like, what are you doing? Who are you with? Are you having fun? And so on.

So if you want to do annual reports, then you need something like that, I guess.

No, but I agree, the data map is really kind of neat. It's very simple and it more or less automatically creates all these graphs based on your data. It's kind of nice.

Yeah. And, I mean, so you can find a lot of visualization things on the app store, but they mostly fall into two categories. So on the one hand, you have these presentation apps that allow you to create some type of presentation, like haiku deck or Apple's keynote, of course. So you have your pie charts handy and you can present them immediately. And then the other category is business intelligence. So it's things like, even things like Tableau, for example, that have an iPad version. So you can find that. But all of these apps aren't terribly exciting, I think. So there. Well, there are bar charts and pie charts and line charts, and it works and it's well done usually, but it doesn't really use touch, to be honest.

But what kind of need do you think they cover? I'm curious about that. So why do people use Tableau on an iPhone? I mean, it's because they.

Kind of mislead if you're sitting on your couch and trying to do some data analysis. I don't know.

I think at the moment, a lot of tablet apps are being like, are for many companies the chance to try something new? So they have a website already, but you cannot change that too much. You know, it's like, yeah, we can't lose like confuse the people, you know, and so on. But if you work on the iPad app, you can basically do what you want. And so I think many people tryouts like, funny stuff in the apps just to see what happens. Yeah. What happens and what makes sense and what doesn't. So it's actually kind of.

That could be. Yeah. And also you shouldn't underestimate the business market for things like that, I guess. So that people not only want to have some application for the PCs of their workforce, but also for the tablets of the managers.

Yeah, sure.

I've seen a few nice sports apps, like visualizations. Like there was like a real time visualization of soccer matches, I think, two or three years ago already. Can't remember the name though.

Yeah, that was during some World Cup.

World cup, yes, exactly. Yeah. You see, we're all experts here, but that's a good use case. Of course, this whole second screen, like, you support like a sports event or like some shared event with an extra layer of information. Everybody's after that, basically.

So in a way, now that I think about it, even your Moritz, even your Emoto project could have been something developed on a. Yeah, yeah, we had.

That use case in mind. I mean, we didn't like really aim it exactly at that, but we had that use case in mind. And we also had a few mock ups with like an iPad, our site, running on it. It basically also worked, but we also learned you have to sort of. Yeah, you have to boil things down a bit more than for the desktop in order to make it really work nicely on a mobile.

Yeah, sure.

Yeah. You know what would be nice on a tablet? A really nice atlas. Like something you can browse through forever, like super detailed maps and statistics around the world. Like, where's National Geographic these days? Yeah, I mean, they could totally do something like that.

Yeah, they have an iPad app, but it's just for the articles.

Right. And I think they had some sort of a Globe app, like world Factbook type thing, but it didn't go very deep.

But I'm sure we are going to see much more stuff coming.

No, I think it's going away. Mobile is not staying. I think ghetto blasters are coming back.

And we're gonna go back to this bulky, huge, exactly like strap.

You know, you have to carry a computer with a strap. I think it's kind of cool.

Yeah, yeah, it's gonna be fun when we will tell our kids that these things existed. I still remember my, the first portable computer. My, my dad bought some quite many years ago. That was really weird.

The size of a washing machine.

Yeah, yeah.

It was really, really bulky, portable. Well, if you have ten friends around.

That was really weird. Anyway, so another thing I wanted to ask to Dominicos, I think another interesting kind of application is interaction between different devices, right? So in a way, you could. So as soon as you have this kind of mobile touch devices, they can also interact with other screens or devices. Right. So for instance, here we have been, I have seen people in my lab interested in the idea of interacting with large displays using touch devices in the rans. Right. And I guess there is a whole ecology of devices that might want to interact one to another, and it's going to be much easier to interact with one and use this one to pilot something else in other devices, right?

That was really weird. Anyway, so another thing I wanted to ask to Dominicos, I think another interesting kind of application is interaction between different devices, right? So in a way, you could. So as soon as you have this kind of mobile touch devices, they can also interact with other screens or devices. Right. So for instance, here we have been, I have seen people in my lab interested in the idea of interacting with large displays using touch devices in the rans. Right. And I guess there is a whole ecology of devices that might want to interact one to another, and it's going to be much easier to interact with one and use this one to pilot something else in other devices, right?

Yeah, yeah. I think that makes a lot of sense, especially as we have, most of us have these really small portable devices like smartphones, and then we are surrounded by larger displays in our environment that are not necessarily touch enabled and not necessarily reachable. So I think it makes sense to use your own smartphone to remotely control something, and you don't really see that too often in the consumer market, of course. So I guess it's still a research thing, but people are using their mobile devices while they're watching tv, for example. So having some kind of connection between those might be interesting. I think the next generation of game consoles, like the new Xbox and PlayStation are doing something like that, but I guess it won't go too deep. So you can, I don't know, maybe change the channel using your smartphone or something.

Yeah, I think for visualization, that's an interesting opportunity because you can migrate some components of the user interface on the mobile device and free up some space on the visualization side. And also, I have one device that works really well in terms of interaction and one device that works really, really well in terms of visualization.

True. Yeah. You can also use your smartphone as a pointing device to select a certain region of the visualization, for example, and do something with that. I don't know, filter it or so.

Yeah, it's kind of exciting perspective. You think about all these screens being around and you could just walk up to a screen and you have your mobile on you. That's your data and your identification token, more or less. You can just grab one of these display spaces and interact with it with your mobile and then move on and stuff like that. So that's very good perspective to have there.

If it's fast enough and you don't have any cables, then that might actually be fun. Yeah, of course, the main downside of portable devices is that the screens are so small, so doing something about that might actually be nice.

Battery life is another issue.

Oh yeah, that, yeah, I think by now we've already reached the point where a laptop battery actually lasts longer than a smartphone battery.

It's true, it's true. My iPhone is terrible.

Oh, yours as well? Yeah, at some point the batteries just give in. I guess that's just the way it works, so.

But that's the thing from the technological point of view, how much harder it is to implement visualization on, let's say, an iPad or any other mobile device. I think the technological component or challenges there is also a very interesting angle. So what's your experience with that?

Okay, so I think there are two ways to develop visualizations at the moment, I think for any platform. So one is do it natively, so create an actual application that is specifically written for this type of device or do it on the web. And of course the web is newer and better and we have D3 there, so everything is great. But the problem with using a web based visualization on a mobile device is that mobile devices are really slow compared to an actual computer. So with the Touchwave project, we started out with a web based visualization and then we went to a native implementation just because it didn't work at all. So it was just basically it looked like this terrible slideshow of a visualization. So yeah, I can, I don't know, it always depends on what you want to do. So if you have a more complex visualization and animations and things like that, then you probably can't avoid going native. And of course that has all the downsides of if you develop for iOS, then it won't run on Android and vice versa. And you have this whole zoo of devices and operating systems and yeah, you always restrict your, your market.

So you basically code everything if you want to cover more than one native device.

Pretty much, yeah. So, and iOS is objective C, of course, and Android is Java, so it's completely different. You can't really reuse any code. Okay. Yeah.

And how are the graphical libraries there? Are they usable, reasonably usable or it's just.

Yeah, my main experience is with iOS and core animation, I think it's called on iOS, there's also called core graphics, which is on OS X then, and that is a really nice library. So once you get the whole objective c stuff, once you understand that, which takes you around, I don't know, a year maybe, that's a really great library.

Fantastic. Only one year.

Yeah, it's a bargain.

I mean, one option might be to have good performance on both platforms, but I'm not 100% sure. But I know that Cinder runs really well on iOS, so it's a C library and you can compile like to have it run on iOS and it like uses hardware acceleration. It's really fast and I think you can also target Android with it, but I'm not 100% sure.

Yeah, you can use C code for Android as well, but I don't know.

How then if the talking to the native drivers and whatever, how seamless that works. But that might be an option.

Yeah, but of course you always have some necessary work for the adaptation for a different. I don't know what's easier actually, to port a windows application to macOS or iOS application to Android. I'm not sure. You don't want to be in the same ballpark. No.

But for, let's say for prototyping or trying things out. Do you think web, web based technology is an option? If so, how? I mean, occasionally you just rewrote the better life index to work on tablets as well.

Oh yeah, that's happened and you got it to work.

So I think there is, if you work hard, an option to make it work with web technologies as well.

Yeah, you can do that. Of course you lose all the niceties of things like SVG for example, where you just say, okay, this object needs some type of interactivity, do it browser, but you have to derive where a touch landed and things like that. So yeah, optimization is everything. And I guess it's gonna be better pretty soon. So I guess next year's devices are, will be able to run all these web based visualizations without a problem, right?

Yeah, yeah. So this one was done in canvas in the end, so we tried out a few alternatives in the beginning and in the end I think you threw out most of the third party libraries.

Yeah, we actually started with paper js as a graphics library that sat on top of canvas, but that was too slow. Even. So now everything's happening natively.

So this is what migration of the better life index on mobile devices, right?

Yeah. So there's a complete HTML five rewrite. At the moment it's just the fallback option, but soon it's going to be the default because it's very, very close to the flash version.

So this is something you guys are still working on or it's something that is out.

It's basically done, but it's not out.

Okay.

Limbo again?

Limbo again.

Okay, so another thing I wanted to ask to Dominicos. I mean, I'm really interested on the research side, as you may guess. And I'm wondering, I know that you know research really well, so are there any other results around or other than your, your work that we've been talking about before? Every researcher trying try to tackle this kind of issues and understand what works when, how, stuff like that.

You mean for mobile devices?

I mean for visualization on mobile devices.

Okay, so there's, there's been this, this really old line of research, so old in terms of computer science, you know how it is. So this five year old line of research that has been going for visualizations on these personal digital assistants and feature phones. So that was really interesting. So you had actually scatter plots on feature phones that you could control using your numpad and then that line of stopped for some reason. But I guess visualization was more of a topic in the tabletop community. So with these really huge touch enabled table like interactive surfaces. And there was a lot of research that went into this direction, especially if you think about collaboration. For example, tabletops are huge enough that more than one person can use them at the same time. So multitouch was always a topic just because you had multiple people interacting with the visualization. And then the question became, okay, how do you solve problems like the orientation of the interface, for example, that labels are still readable even if people are standing around the table? So yeah, there's a lot of research in this regard. The question of course is how much of it is applicable to our current state of affairs. So the current mobile devices and touch enabled tablets?

Sure.

Because, well, I don't know. Enrico, do you have a tabletop at home? An interactive one?

I just ordered one also. That was you? No, honestly, what I would really like to do is to. So I have an interest for touch displays and I think I've been discussing this thing with Moritz before offline and I think, I don't know, I know that quite a lot of researchers have been spending time observing people interacting on top of tabletops, but I would be really curious to see people interacting on touch displays that are not too, too big and they are vertical. So my gut feeling is that there is quite a big difference there and I have no idea why. Oh, maybe it's, I mean, maybe this research is taking place and I don't know anything about it, but my feeling is that standing in front of a touch display is going to be a much richer kind of experience. So, for instance, the problem that you just mentioned, the fact that you, you have an orientation problem, that's not going to happen if two persons are standing in front of a screen, right?

Sure. Yeah.

And at the same time, I think that it's even kind of a more natural kind of interaction between people. People are very much used on standing in front of a whiteboard. So it's a very natural transfer from the whiteboard to a touch display. Right. And the other thing I've seen quite a lot in research is people interacting with very large displays, but very, very large, and again, with indirect devices. Right. So I'm actually really wondering why there's not a lot of research, as far as I know, on people interacting on this medium sized kind of touch displays. Because to me it looks something like really, really interesting and much, much more natural than all the rest. I don't know.

It's also more realistic in an apartment.

It's much more realistic.

Or an office to have like something big on the wall and something big in the middle of the room.

Yeah, yeah, yeah.

If you think about it, yeah, no, yeah, I totally agree. And I mean, these like 50 inch type displays, they are totally coming and they're getting much more, much cheaper and they're really nice things to have, like many of the new tvs also have now this type of size. And, yeah, why not touch enable all of them and yeah, yeah.

Especially as you can just buy, buy and touch overlay for, I don't know, $150 or so to refit the whole thing to touch. Yeah, it somehow seems that all of these vertical interactive types of displays are. They always had to be huge. So really wall size, everything on the.

Wall size is a big ego boosting kind of problem here. No, see, I'm serious. I mean, I think it's the fact that having a huge display, looks, having shows how powerful you are or something like that. I mean, I can totally sympathize with this thing. I would like to have my own huge display, but I cannot afford it yet. But, yeah, but I think it would be much, much interesting.

And what do you do with information? Like on the tip of your knees? It doesn't help you that much.

Yeah, that maybe might have been an outcome of this whole notion of the more pixels we have, the more data we can display.

Yeah.

So if we have this wall full of data, then we can get all the insights.

Even though it must be related to.

This big data hype that it has.

To be like life size. You should put the big data on a big wall. And that might solve all the problems. Yeah, yeah.

But then it's crazy, because you have the problem that you have to. You cannot stand too close, and if you are too close, you. You lose the context. Right. So you have many more peaks running.

Around, like back and forth.

Yeah, you have many, many more pixels, but your field of view is. I mean, the interaction with your field of view is much, much more complex. Right, sure.

Well, the cute thing about these walls is that you actually make zooming and panning a physical thing, because you have to walk in front of the wall to and fro.

Sure, sure.

I don't know. Maybe that was the idea.

Yeah, yeah, yeah, yeah. Well, when I was in Konstanz, we had this huge power wall. There is an old room that is called power wall room, actually. I think they actually just kind of renovated it a few weeks ago. This is what I heard. And it's. It was stunning. Definitely. It was really stunning. I mean, and it's. It's really nice when you can come. When you can come closer. You can see a lot of details. It really scales very well. But in terms of interaction is a nightmare.

Yeah, yeah, yeah, yeah. Well, when I was in Konstanz, we had this huge power wall. There is an old room that is called power wall room, actually. I think they actually just kind of renovated it a few weeks ago. This is what I heard. And it's. It was stunning. Definitely. It was really stunning. I mean, and it's. It's really nice when you can come. When you can come closer. You can see a lot of details. It really scales very well. But in terms of interaction is a nightmare.

I mean, did it have touch interaction? No, it's.

No, of course, you cannot even. You're not even allowed touch it because you. It can get dirty and the power is too powerful. Yeah, yeah. It's a totally cool piece of technology. If you go in the. In the back room and you see the projectors that are working there to project these images that are very high resolution.

So it's a back projection.

Yeah, it's back projection, and it's a whole array of projectors. And I I think they had to develop their own infrastructure to pilot these projectors in a way that they are very well aligned and that they synchronize themselves.

It's nuclear reactor in the yard.

There is a huge fan that is taken care of.

The whole city of console is probably heated.

Yeah. Plus, I think even the glass itself is a very special kind of glass. It's a very fine piece of technology. Really, really stunning. Yeah. But then, in terms of interaction is really complicated.

Yeah. So you mentioned before this interaction between multiple different types of devices, and of course, such a powerball might become much more usable if you have smartphones or tablets that people can then use to.

Point at stuff, interact with it like this. Yeah, that totally makes sense.

But then why do you have this huge wall?

Yeah, yeah.

Maybe just for, like, having this landscape in context, and then the details happen on more high res, smaller devices.

Maybe.

But there's also like this, you know, minority report. I don't know if we open like this box now, but I mean, these are the.

It's getting late.

Yeah, it's getting late for that. But I mean, no, there is an actual problem there that it's very fatiguing just to interact with these big things. Yeah, you need to make big gestures which cost you big energy and are big time. Very unusual. And, yeah, after ten minutes you're like, what?

Yeah, and not only with big things. I mean, we also have that with touch enabled laptops, for example, where you actually have to reach out and move stuff on the display. And I think people still prefer mice to do that just because it's much more ergonomical. So I mean, that's a huge downside of the whole touch thing, that you often have to lift your arms and that is very tiresome.

Makes strange types of movements as well.

Yeah, so I mean, of course, minority report, it's very impressive. And he only has to use the interface for, I don't know, 30 seconds per day, so that's fine.

Exactly. Yeah, yeah. But there's one since like oblong industries, they actually, they built these interfaces and they sell them and also these big screen like combinations of different screens and. Yeah, it's very interesting. Also, we could also have an episode on these types of interfaces at some point, gestures like in space, but it's also. Yeah, it's a whole new type of sport you have to become good at.

And speaking of big screens, why not having your personal dome? That would be fun. Okay, there is one last thing I wanted to ask to Dominicos. So I'm wondering maybe some of our listeners might want to start doing visualization on mobile devices. So can you give some suggestions on how to start in terms of technology, even in terms of design, things to watch for? What are the main principles or what are the main technologies to start with? So what's the best way to start with visualization on mobile devices?

And speaking of big screens, why not having your personal dome? That would be fun. Okay, there is one last thing I wanted to ask to Dominicos. So I'm wondering maybe some of our listeners might want to start doing visualization on mobile devices. So can you give some suggestions on how to start in terms of technology, even in terms of design, things to watch for? What are the main principles or what are the main technologies to start with? So what's the best way to start with visualization on mobile devices?

Okay, so I guess it depends on what type of visualization you want to do. So if you want to just port your run of the mill type of visualizations to a tablet, then I would use the web for that. If you already have a background in D3, for example, then you can just use that. And there's a lot of resources on how to program how to do JavaScript for mobile safari, for example, or also for chrome on Android. And well, basically you just have to replace your mouse events with a touch events and you're all set.

So it's make your buttons bigger, of.

Course, and make your buttons bigger. It's a takeaway message from this episode, but I think if you want to do something more interesting and also more intense from a graphical point of view. So if you want to have animations and things like that, then there's no way around actually learning one of those two major frameworks. So either programming for iOS or Android and for the whole design thing, there's lots and lots of resources on the web for interaction design, for mobile, and there's great stuff out there. I mean there are all these conferences even about this type of things. So. Yeah.

Is there any specific book or resource on the web that you might want to mention? I don't know, people might be curious about that.

So of course I always had this plan of having a blog about touchable visualizations and that's coming, I promise.

Okay, cool. So just go to Dominico's blog in the future and keep an eye.

Go to my future blog.

Yeah, that would be nice because I think, yeah, there's no, especially for visualization, there's no clear, not even a clear point where you would follow the latest projects, right?

Yeah, you can find them on the usual blogs, of course. Yeah.

But I think there should be more. If you're really into the mobile and follow that, you should maybe find more than what's showing up on flowing data or info statics.

Yeah, I'll make sure to put that on my blog. That's cool. Very cool. Now we have it on a podcast.

Nice. Bitch.

You totally committed yourself.

Yep. Shit.

Now you cannot escape Dominicos. And this whole blog thing is a nightmare. I can tell you.

It's worse than academia.

What did I do?

I think we had a nice. Oh, Dominicans. I think we had a nice meeting together. Talking about visualization blogs. Right? Some time ago.

Yeah, we had that two years ago or so.

And did you agree never to write blogs again or.

Well, that was fun. I think Robert was there as well. It was kind of like 20 people.

The usual suspects?

Yeah, the usual suspects. It was quite lively actually. And that was.

Yeah, it was nice.

Yeah, yeah, yeah, it would be a.

Great topic for a blog. So I wouldn't mind at all if you would start that. Okay, I agree. Cool. Excellent episode. I think we're basically through, unless you have something more you want to mention. Anything else you want to get rid of?

Great topic for a blog. So I wouldn't mind at all if you would start that. Okay, I agree. Cool. Excellent episode. I think we're basically through, unless you have something more you want to mention. Anything else you want to get rid of?

Any secrets?

Well, any more commitments, that's enough commitment.

You know, we are specialist in destroying people's career here.

Oh, nobody told me before. So maybe one last thing just to close this whole thing off here. So I think it's this whole mobile and touch enabled visualization stuff is very, very interesting. I think we barely scratched the surface there, especially what's possible from an interaction design point of view. So this whole notion of having visualizations is something that you can manipulate using your hands. I find it very exciting and I'm really looking forward to future developments there.

Fantastic. Thanks a lot, Dominicos. It's been great having you on the show.

Yeah, thanks for coming.

We're looking forward to your blog and your future developments. We want to see the life better life index coming up soon.

Yeah, it's pretty cool.

Okay, thanks. Take care. Bye.