Visualizing Data with RAW

The idea was from the beginning to create a tool that should make the life of data designers and people who work with data a little bit easier.

Data stories is brought to you by Qlik. Are you missing out on meaningful relationships hidden in your data? Unlock the old story with Qlik sense through personalized visualizations and dynamic dashboards, which you can download for free at clic Datastories. Hey, everyone. Welcome to a new episode of Data stories. Hey, Moritz.

Hey, Enrico. How are you?

I'm good. Good.

Very good.

Yeah. Speaking from warmy, New York, very weird winter.

Very good. And back in Germany, I spent a few lovely days in the Bay Area. But now I'm tired as a dog because I'm super jet lagged.

But I saw that.

That's the price you pay.

San Francisco is always passion, right? Yeah.

I had such a good time.

Such a lovely place. Yeah. Any news?

No big news.

No.

But, yeah, I spent a few days consulting in the company. Met a lot of people, even got recognized on the street. On the street corner. Made my day. Shout out to Luke, who spotted me. Thanks, Luke.

Fantastic.

Oh, you're Morris and famous in San Francisco.

Total. You should start hiring guards, Moritz.

Yeah, I need bodyguards and everything. And a stretch limo.

Oh, my God. Okay, so before we start with the guests that we have today, I just want to briefly thank all those of you who already signed up to our Patreon crowdfunding initiative. Yeah, we have 28 of you. Of 28 patrons. So thanks so, so, so much. We are not so, so, so far from our, from our goal. There is still some. Some way to go. So just as a reminder, we. We are trying to reach $600 per episode. Then we are at 200. So we are almost one third. And.

Oh, my God. Okay, so before we start with the guests that we have today, I just want to briefly thank all those of you who already signed up to our Patreon crowdfunding initiative. Yeah, we have 28 of you. Of 28 patrons. So thanks so, so, so much. We are not so, so, so far from our, from our goal. There is still some. Some way to go. So just as a reminder, we. We are trying to reach $600 per episode. Then we are at 200. So we are almost one third. And.

Yeah, keep going, guys.

Keep going, guys. I mean, what else can we say and do? If you like data stories and you want us to. You want help us to go switch to this model. Yeah, go to Patreon and help us. Moritz, do you have anything to add about that?

No, you're absolutely right. But it's great to see so many people already joining, and that's fantastic.

Okay, so today we have three people actually on the show, and we're going to talk about a very nice data visualization tool called Raw. And we have actually three people from Italy. Three Italians, which doesn't happen every day, as you can imagine. I like that. So maybe we can switch to Italian for a while. So we have Giorgio Caviglia, Michele Mauri, and Giorgio Ubaldi. So this time, I'm actually not butchering people's names. And welcome on the show, guys.

Hi. Thank you.

Thank you. Hello.

Thank you. Hello.

Hi all.

So can you guys briefly introduce yourself so our listeners know a little bit about you, what's your background and what you're doing right now? And then we can switch to describing raw. Thank you.

Okay, so I can start. So I'm Michele and we are all part of the original raw team. And I'm a designer and currently I'm a postdoc at Politecnico di Milano and actually the last one still at Density Design.

And I'm Giorgio Uboldi. I am also from Milan and I was part of the Density Design team until last year when I co founded with Matteo Azzi. That is the only original co author that is missing now during this episode. And we funded a small design studio called Calibro (calib.ro) and we still work mostly on data driven projects and digital projects.

And I'm Giorgio Caviglia. I'm a designer and researcher who is passionate about products and experiences to work with data, especially data transformation and exploration. I've spent several years in academia. I was a density designer, Politecnico di Milano, with the other guys. Then I moved to Stanford and now I am at Trifacta in San Francisco as a principal designer.

Nice. So can you guys briefly describe what Raw is for those of our listeners who are not aware of it?

Nice. So can you guys briefly describe what Raw is for those of our listeners who are not aware of it?

Okay, so raw, that is, first of all, is an open source and free data visualization tool that you can find on the new website that is at Rawgraphs IO. And so the idea was from the beginning to create a tool that should make the life of data designers and people who works with data a little bit easier. And so it was conceived as a kind of a link between spreadsheets, applications, so Excel, Google spreadsheets and so on, and vector graphic softwares like illustrator. And since, and we always conceive it as a tool from designer to designers in a way. And it has two main goals. One is to give the possibility of quickly explore data sets in a visual way. And the other goal is to give the possibility to designers and let's say these geeks in creating semi finished vector graphics visualization that then can be reflected find, for example, in illustrator, it's a kind of easy tool, so if you go on the website you should be able to use it, but just quickly. Basically you take your data set, you copy and paste it inside raw. You then have the possibility to choose between around 20 charts. Some are common charts, like a scatterplot or a bar chart. Others are less common and a bit unusual, like alluvial diagrams or horizon charts that are a bit more difficult to create with other tools. And once you select your chart, your layout, you can map the dimension of your data set to the visual variables of the chart and instantly see the visualization. Then, depending on the type of chart, you have the possibility to change some options. And in the end, you can download your visualization and then go on with your project, let's say.

Yeah, and I have to say, I always show raw when I'm teaching because it's just such an easy way to, you just paste in your data, you try out a few templates and export the chart you like, and you already, you know, it looks pretty good already. And then you can spend some time to clean it up, maybe in illustrator. So going to the beginning. So why did you start the project and how did it get started? Is there like an origin story to the whole thing?

Yeah, and I have to say, I always show raw when I'm teaching because it's just such an easy way to, you just paste in your data, you try out a few templates and export the chart you like, and you already, you know, it looks pretty good already. And then you can spend some time to clean it up, maybe in illustrator. So going to the beginning. So why did you start the project and how did it get started? Is there like an origin story to the whole thing?

Yeah, sure, I can talk about that. So Ra was born a Density Design lab at the Politecnico di Milano. And Density Design, we say, played a lot with data, with data visualization, we experimented, we pushed the boundaries a little bit, tried to experiment with new layouts, and also we do this together sometimes with students, try to teach them how to use the tools already there, but also question sometimes the limits and the possibility. And also for some projects we were working on, so often we had the need to explore the data, to have a quick visualization and to have especially visualization that are not so common. So custom layouts, and at the same time having visualization in vector formats that we could eventually refine in illustrator or in other tools. So, and of course, D3 was out at that time. So this is about 2012, 2013. I was doing my PhD at Density Design at that time. And so I used D3 a lot. And so often I had to say, create some visualization for the other, either students or people at density. And this, of course, as you. No. And you can imagine it's time consuming, and also it's frustrating because the actual people that want to work, they cannot, they have to pass through somebody else. So I thought, oh, maybe there is a nice way we could put some sort of UI on top of D3, or in general, on top of this process of data exploration and visualization. That's a, that's how I started prototyping a little bit with very, very rough ways with j's D3, and creating the first version of raw in which you could upload a tabular data or copy and paste, and then from there, selecting, I think in the beginning were very, most basic, but still not so common. I would say visualization layout like tree map or trees key diagrams, which audacity was extremely popular back then, so still is.

Yeah, it's true.

So, yeah, so that's how it started. And then at the time, I think the other Georgia also came in and working more on the UI and then Michele and Matteo, like, introducing new visualization layout or again creating also the community gradually. So the tool started to be a little bit less an internal tool and becoming more, say, something that we felt comfortable to in a way, release. And yeah, that's how it happened. I initially was very, very close as tool. So I think I spent quite some time to make it possible for people to contribute to the tool, either adding layouts or data models. So, yeah, yeah, that's the. In a nutshell, yeah, absolutely.

I think what I really like of raw is that it's one of those rare instances where some people develop visualization tools that actually everyone can use and no coding or very basic coding is needed. So when you look at the landscape of existing visualization tools, most of them require some heavy coding. And I am a big, big fan of visualization tools that are actually for everyone. I think there is a huge empty space, a big gap out there that needs to be filled. And raw seem to me to go exactly in this direction. And it's free, by the way. Right, because one could in principle use Tableau, but I mean, it's pretty expensive. Yeah, exactly. Yes. I think it's a great project. Can you give us a little bit of details about the geeky stuff behind it, say what technology you used, maybe whether even it changed over time just for those listeners who wants to hear the geeky stuff?

I think what I really like of raw is that it's one of those rare instances where some people develop visualization tools that actually everyone can use and no coding or very basic coding is needed. So when you look at the landscape of existing visualization tools, most of them require some heavy coding. And I am a big, big fan of visualization tools that are actually for everyone. I think there is a huge empty space, a big gap out there that needs to be filled. And raw seem to me to go exactly in this direction. And it's free, by the way. Right, because one could in principle use Tableau, but I mean, it's pretty expensive. Yeah, exactly. Yes. I think it's a great project. Can you give us a little bit of details about the geeky stuff behind it, say what technology you used, maybe whether even it changed over time just for those listeners who wants to hear the geeky stuff?

Sure, I'm happy to. So in the beginning, I think the first prototype was, say, purejs. No, framework was actually a bit crazy because as you can imagine, there are a lot of, say, binding and interaction between the data, the visualization, the UI. So I remember crazy events going all over the place and to manage was crazy. So then I think at that time, angularjs came out as one of the more, let's say, robust framework. And I started to play with that and yeah, it worked very well. So of course the main visualization part, the charts, came from D J's, which worked in a sense as a first base, especially to have the first charts there then. So the idea of raw is actually, if you want, I think the main aspect are two parts. One is, say the process to transform your tabular data into something else. So if you're familiar with D3, I think D3 is very powerful and, but he requires that in a certain shape, right. If you want to work with certain layouts like I don't know, force graph, you need nodes and edge and stuff. And I found that's mostly, actually the hard part for a lot of people is like you have, you don't have that kind of data. So in a way I think there is this part that if you want is like more data structuring, data transformation. And the good news, I think that if you, the data models are not too many or if you want, with few of them, you can create a very, very different layout. Right? So if you have, I don't know, points, if you have a network kind or if you have a tree, you can create a tree map or hierarchy or, or Sankey. So there is that part that I think also it's probably more than the visualizations that actually are pretty, I would say, straightforward from what D3 examples or in general how D3 was conceived. But I think that part probably, I don't know personally I think is the most valuable in a sense of raw is this sort of UI on top of these very. If you want complicated processes and annoying at the same time. So the last part is the charts and the layout. And actually the way raw works is we always say we use D3, which is of course true, and a lot of layouts come from there. But the thing is that raw potentially can accept anything. So it simply has this function which you pass through the data and inside that you can do whatever you want with your dom. It's just to say using SVG is just one option of in this way is very free and open, I would say. And that's something by design compared for example to other sort of dictionary or languages. I don't know. I think about like Vega for example, in which you need to define the visualization in a very formal way, which is very good, I think, but at the same time can be a little bit limiting.

So, so in a way each visualization is like a little plug in to that bigger framework, right. And it talks to the framework with a, with a. In a clear and defined way, but what happens inside doesn't really matter, right?

Absolutely. Yeah.

Yeah. And so you can write your own plugins. Do people write their own chart types? Do you know that? Did you get some contributions from the community or are they maybe even companies who write their own custom charts they can use? What was the pickup there.

Well, we get some contribution on this and. Well, actually many people created their own charts, but, well, mostly they drafted a visualization, so the code was not good enough to include in our project. And this is something that we discovered about open source projects, that it's not so simple actually to involve someone else to contribute in a good way to your projects. And if I can add something to the technological choices on rule, Georgia spent a lot of time actually trying to figure out a way to make a solution which doesn't require a server. Doesn't require a server.

It's true, actually.

Yeah. Because it's something really valuable for people, the fact that data stays on your computer and is not sent anywhere. Yeah. So I think that this was part of the complexity that Georgia was.

Yeah, that's a really good point.

Yeah.

And it's just one single long website and all the tools and all the codes is there in one page. Right. Which is great, of course, but can make it hard to develop also.

Yeah, yeah. We have some limitations like that. Some of them are actually, I remember one thing that we decided in the tool was to, you know, potentially you can add a lot of, say, features and option to change the charts, but we decided if you could actually do it, let's say in illustrator, you should do it in illustrator. So we like, I don't know, changing the fonts. Right. Or something like that.

Yeah. And one feature I really love, if I understand correctly, you can just cut some data from somewhere and paste it in the tool. Right. It's one of those little, little things that make a lot of difference. Right. Just cut and paste there and it works. That's amazing. I love it. Okay, so can you tell us about, I guess you have learned something from the process and the project. So I think if I remember correctly, you started in 2012. So it's been a few years you've been working on that. Do you have any, do you have any major insights or lessons learned from the project that you want to share with our listeners?

Yeah. And one feature I really love, if I understand correctly, you can just cut some data from somewhere and paste it in the tool. Right. It's one of those little, little things that make a lot of difference. Right. Just cut and paste there and it works. That's amazing. I love it. Okay, so can you tell us about, I guess you have learned something from the process and the project. So I think if I remember correctly, you started in 2012. So it's been a few years you've been working on that. Do you have any, do you have any major insights or lessons learned from the project that you want to share with our listeners?

Well, I can start with this. The first thing that came in my mind is about open source projects. And as I said, it's not simple to do a good open source project because, well, in my original idea, maybe quite naive, is that you put your code on GitHub and then other people will write the code and do the stuff for you. But actually it's really not like that. It requires really lot of effort in following the community and while checking all the contribution and decide which one you want to keep and which one you have to exclude and also following the user groups, because they were asking many questions, from the most complex stuff to the simplest one. And so this is something that I discovered that actually open source projects are cool, but require a lot of effort. And a second thing is about visualization. And, well, we discovered that it's quite difficult to formalize a visualization. So, I mean, the same, you can create the same visualization in several different ways. And this project was really interesting for us because we were forced to think about the essence of each visualization. So also the most simple one, I don't know, a bar chart. What is bar chart? And I think that it was really enriching this process.

So you mean that even for a simple chart, there are lots of variations, and you have to decide which one to choose, right?

Yeah, exactly. Exactly. You have many variation on the visual side, but also on the data structure, on the model that you use. And so we had to choose just one. And we have to identify a sort of current way of creating all the charts. So also find out which are the common parts, which are the common features shared among different kind of visualization. That was challenging and. Yeah, yeah. Third thing that we discovered about this kind of project is that it's quite difficult to keep everything, to keep it simple, let's say, as Giorgio was saying. And at the beginning, when we discovered that, because at the beginning, you just want to add any kind of function to your software and it become a mess quite, quite quickly. And so he gave us three rules, like, so the first one is ask yourself if you can achieve that particular thing with another software.

That's a really good one. Yeah.

If you can do it in excel, if you can do it in illustrator, where we. Well, we won't add that feature.

That's smart.

And also, we had to choose one kind of data. So we said, okay, we just use tabular data, one table, no, more than one table, and we will produce just static images, no interaction, no movies.

Come on.

And it's difficult, actually, to actually follow this rule, because every time you try. Come on, let's add a little bit of interactivity.

This time only. This time only once.

No, Michael, these are really good points, and I'm actually learning myself some of these things. There is a tool that we are developing in my lab, and we are planning to go public someday. And I can already see that. So your first point is clearly a really good one. If you decide to make a tool public, you have to be ready to really work on it in terms of promotion and supporting. The community and all the rest, and also make it simple. I'm always, always tempted to say, oh, but there is this other cool thing that we could add here, right? And yes, and then I recognize that is not a good idea. Okay, so we just want to conclude by asking you what is happening next. Right? So I guess you have some plans for raw. I saw that you've been promoting it quite a while recently. So what's going to happen next?

No, Michael, these are really good points, and I'm actually learning myself some of these things. There is a tool that we are developing in my lab, and we are planning to go public someday. And I can already see that. So your first point is clearly a really good one. If you decide to make a tool public, you have to be ready to really work on it in terms of promotion and supporting. The community and all the rest, and also make it simple. I'm always, always tempted to say, oh, but there is this other cool thing that we could add here, right? And yes, and then I recognize that is not a good idea. Okay, so we just want to conclude by asking you what is happening next. Right? So I guess you have some plans for raw. I saw that you've been promoting it quite a while recently. So what's going to happen next?

So here comes a little bit, the more exciting part about the next steps. And so as you probably some users of raw notice that we have been pretty, I mean, we haven't been active in the last couple of years because as we said, it was a self initiated project, a self side project, and we didn't have the time and the resources to work on it and improve it. And what happened is that some months ago, an Italian company that works in the field of digital communication programs and digital marketing that is called Contact Lab decided to sponsor Raw. And this was like a super good news for us because it was a good possibility for us to keep the project alive, but also give the possibility to use more time and resources to work on the project. So what we have done basically is since in the last two years anyway, we worked a little bit on Raw, on ourselves in our spare time, we decided to clean up the code a bit and release like some weeks ago, a new version, the 1.2, with some new visual layouts and some new features. And we also decided to change the license of the code. The project is still open source, clearly, but we decided to move towards Apache two license in order to make it simpler for other people to reuse it in commercial solutions. And then we started thinking about a roadmap of features and possible things that we would like to do. And first of all, we have to update the old technological stack, since we paused it for a while, but the word of coding and developing moved forward. So we have to update all the framework and libraries. The other thing that we would like to do is to add new visual models. So from more basic charts like line charts or rudder graphs, to other ones that I don't know which one will be, we want to add them and release them to the public. And then we want to try to not follow one of the rules that Michele said before. This is the most challenging thing actually, and we are really thinking about this is to move forward just from static visualization outputs to something that is more interactive. And this is something that, I mean, we are thinking about it and we have to take some time and maybe in the next month you will see some updates about that. Then one last thing that I think is really important, and Michele said about open source project is the fact that we want to engage more with users and the community. So with the new website, we are launching a part that is about creating tutorials and easy how to do guides for users and also involve more the community in terms of explaining how with some case studies how you can use Raw and hack it for your own project. Another thing that we would like to do is to open another section on the website that is about showcasing the best and more interesting projects about Raw. So we're gonna create this somehow, this call to action where people can send their work or tweet it to us and we're gonna display it on our website so the people that are new to the tool can see the possibilities.

Yeah, that's a great idea too. And you see already how much goes into managing and developing and designing and curating such a complex project. And I think it's amazing what you have achieved so far already. And I was really excited also to see the new release with new cool chart types with horizon graphs and all the good stuff. That's amazing. And to our listeners, if you're not a coder but more a designer, check out the tool. You can create great looking graphs and modify them. Maybe later in illustrator, if you're a coder, you should also check it out because you could maybe contribute or use it. If a client, for instance, asks for a very customizable thing, you could take that as a starting point as well.

Exactly.

Yeah.

And let's keep the ball rolling there. Let's keep raw running. Check out the project on raw graphs IO and on the corresponding GitHub page. I guess this is where you would also start discussions or contribute, is that right?

Exactly. If users or contributors want to engage with us, they can either use the GitHub page or they can use also the Google group if they have questions about how to use it, or they want to have suggestions from other users.

Very good. Great. Thanks so much, guys. This is amazing.

Thank you, guys.

Thank you very much.

Thank you.

Thank you. Very nice having you on the show. Ciao. Bye bye.

Ciao.

Bye bye.

Hey guys, thanks for listening to data stories again. Before you leave, here are a few ways you can support the show and get in touch with us.

First, we have a page on Patreon where you can contribute an amount of your choosing per episode. As you can imagine, we have some costs for running the show and we would love to make it a community driven project. You can find the page@patreon.com Datastories and.

If you can spend a couple of minutes rating us on iTunes, that would be extremely helpful for the show. Just search us in iTunes store or follow the link in our website.

And we also want to give you some information on the many ways you can get news directly from from us. We're, of course, on twitter@twitter.com. Datastories. But we also have a Facebook page@Facebook.com, data stories podcast and we also have a newsletter. So if you want to get news directly into your inbox, go to our homepage data stories and look for the link that you find in the footer.

And finally, you can also chat directly with us and other listeners. Using slack again, you can find a button to sign up at the bottom of our page. And we do love to get in touch with our listeners. So if you want to suggest a way to improve the show or know amazing people you want us to invite or projects you want us to talk about, let us know.

That's all for now. See you next time, and thanks for listening to data stories.

Data stories is brought to you by Qlik. Are you missing out on meaningful relationships hidden in your data? Unlock the old story with Qlik sense through personalized visualizations and dynamic dashboards, which you can download for free at Qlik Datastories.