Data Nation

Munther Dahleh (00:04):
Welcome to Data Nation. I'm Munther Dahleh and I'm the director of the MIT's Institute of Data Systems and Society. Today on Data Nation, Liberty and Scott are diving deep into the metaverse.
Liberty (00:23):
When we decided to do an episode on the metaverse, I have to be totally honest and say that I had to Google what is the metaverse? And what I saw is that most people have tons of questions about it. Is the metaverse our future? Are we going to be living with virtual reality headsets and completely disregard the real physical world? But before we can answer those bigger questions, we have to figure out and define what actually is the metaverse.
Scott (00:57):
Well, hopefully it's Ready Player One, but I guess if you'd ask the average person on the street what the metaverse is, you'd get a lot of blank stares and wildly different answers. But I guess that's why we're doing this episode. So, to be honest, we're probably much closer to the metaverse than we realize.
Liberty (01:12):
We're going to start by figuring out what that actually is. What we mean when we say the metaverse. And we are going to take these questions to Joe Paradiso. Joe is the Dreyfoos Professor at MIT and directs the MIT Media Lab's Responsive Environments group. On a quick side note, the MIT Media Lab has in the past really defined the future for society when it comes to media and technology. Joe has also written over 350 publications and really specializes in how to augment and mediate the human experience.
(01:50):
In the fall of 2021, Facebook renamed itself Meta, which was really the first introduction I had to the word metaverse. But I at least understand that the metaverse isn't Facebook. From your perspective, from the MIT Media Lab perspective, what do you see the metaverse as?
Joe Paradiso (02:11):
These ideas go way, way back. So the basic ideas are not new. When we think... We don't really call it metaverse because that's become such a loaded term now. You say it, everybody knows what it is, but that in the way hems it in to this kind of narrow definition maybe of a network game environment where you kind of play games and can get a call in virtual reality. It's so much deeper than that. We call it X-verse at the Media Lab and we've got lots of programs that have been exploring the edges of this. We're looking at now kind of theme-ing them a little bit more than we have. But it's about information coming from you and coming to you in different ways at different times. And already, we're living a bit in that. I have my watch that's connected to my doorbell. Somebody walks by. It may buzz or I'll see a picture. So there's a whole continuum there that's really intriguing and I think that's how it's going to play out. It's not just going to be this virtual reality headset we put on. Suddenly, we're somewhere else. It's going to be a whole continuum of really abstracted presence. I think that's one of the interesting things about the future is we become more and more connected. It is about connection. We start blurring the sense of where we are.
Liberty (03:19):
A lot of people are actually already familiar with this blurred sense of reality. When COVID-19 pandemic swept the globe, many individuals started working from home. And for many, I guess the typical workday looked like this. You'd sign into Slack. You'd update your status to active so your employer knows you're there. You make sure you stay productive because your workplace monitors your project progress and senses for computer movements to make sure that you haven't left your desk. You join an afternoon video call via your laptop with colleagues who are also sitting in their homes. And finally, you end your workday by signing out of Slack, shutting down your computer, and you immediately head back into your home life after having just worked an entire day from your kitchen table.
Scott (04:10):
So the line certainly became blurred between what was real and what was virtual. So, Joe, I'm curious. Is this the kind of direction the metaverse is going? Because I know that there are other examples that I might qualify as a tool of the metaverse. For example, if I have my Apple iPhone and I've got a filter on it and it's superimposing a dog over a picture or something I'm looking at, am I using a tool of the X-verse or metaverse or is that part of that blurred reality that you're talking about? Would that fit the definition?
Joe Paradiso (04:41):
It's definitely the on ramp to it and I'd say it's a part of it. Especially when it's networked. Because you do it in Snapchat like my daughters do. Then, all your friends see you as that, so that's your image at that point.
Scott (04:54):
So using the dog filter on my Snapchat is an on ramp to the metaverse. This would be an example of what we call augmented reality or AR. AR uses things like real landscapes and digital scenes to theoretically enhance our current reality. Augmented reality or AR really came to the world's fingertips in 2016. With the emergence of Pokemon Go, players would open the app, which would then use their camera and location to display whatever the camera pointed at. The app would display Pokemon on screen as though they were right there in front of them. Players could then catch the monsters. Eventually, developers added trading, where players could directly interact and trade Pokemon with each other from the comfort of their own homes. It's funny to say the metaverse and the future of humankind may look a lot more like the world of Pokemon Go than you might think, where the metaverse uses reality to support its platform rather than purely be a virtual experience.
Liberty (05:46):
The most common way people are proposing this future is through the use of special AR glasses. These glasses are being developed by brands like Meta or Apple or Snapchat or even RayBan, but I can't help but wonder if existing in these AR and VR landscapes will actually be what the metaverse consists of. Or is this just some idea purported by science fiction?
Scott (06:12):
So, Joe. When you say the word metaverse, the average person on the street, their first thought will probably be, oh, like Ready Player One? Or, oh, I get an Oculus and I'm going playing here. You might get a more sophisticated nerd like myself and be like, yeah, I've been part of the metaverse before. I play a game called EVE Online. Or, yeah, I've owned a house in Second Life. The definition is fluid, so to speak, right? I don't think we would agree that Facebook is inventing the metaverse. I think we should say that Facebook is trying to participate in the concept of the metaverse.
Joe Paradiso (06:45):
Yeah. They're trying to give it a big kick. Exactly what the concept is, they're still figuring out as we are. You made some good points. Of course, you could have your house on Minecraft. You could have it also in Animal Crossing, which is very interesting because people come and visit it. You work on your property. You share it. And also the weather in your island can depend on the weather that's happening locally. So you already see this folding of reality into this other universe. And I think that's... At least that's where a lot of my work has been. I think that's where there's a whole interesting frontier. Interpreting the real world through virtual constructs so you can actually have a closer connection to it.
Scott (07:24):
I like that. I'm going to steal it. Interpreting the real world through virtual constructs. Which, again, it's not quite a direct link to the augmented reality we talked about before.
(07:34):
So, there's this concept of uploading your brain to a computer in the future. When I hear something like this, I envision we're going to upload our brain to the metaverse via an Elon Musk brain link into a Zuckerberg owned community and play virtual football owned by the NFL. Are we basically building the metaverse that I can do all the things in a nonphysical world that I can't do in the real world?
Joe Paradiso (07:55):
That's a very loaded question. You can break it down 20 different ways, all of which are kind of interesting, actually. At the first level, uploading your brain is complicated. Making a connection to your brain is still complicated, but it's doable. People have done it and there's going to be much more of it. So, if you want to look at... You go out... Let's say you talk 50 years. This is a comfortable number. Could even be much sooner. We don't know. People will connect to virtual realms directly, I suspect. Lots of different takes on it. Already, you people that are paralyzed that can move things with their motor neurons. Elon's very involved in that, but there are lots of other people working that problem. Including people with the Media Lab and at MIT. But yeah. This is going to be first for people with disabilities. Eventually, I think it will have broader impact. Like in The Matrix and in [inaudible 00:08:45] and World on a Wire. Stuff like that.
Scott (08:48):
When we talk about the metaverse, we're talking about creating another environment in which people interact. Everything from economies to how we live. Do we think this is a good thing for society? That this technology is coming? Are you excited for it or are you afraid?
Joe Paradiso (09:01):
It's a bit of both, right? Even here at MIT, Tom Malone used to talk a lot about collective intelligence. At least to our point of view. We're afraid of any kind of a socialism. I think identity and individual identity is very important. So if we approach this kind of a collectivism, what's the role of the individual? I think it's critical. Or could individuals be dynamic things that just form out of combinations of people? I don't know. It's going to be very different if you go out far enough. Maybe even in the near term of a decade or so. I think it's going to lead to a lot of capability. On the other hand, it leads to tremendous opportunity for indoctrination because you know a lot about people. You know a lot about their state. You can approach them with something at the right time. And for learning, I think that's great. Because you learn, the cloud learns, we all learn together. It's another future for this. It's already happening to some extent, but now it's brought up close. So that whole division of what the cloud learns what I learn and exactly how I learn it and what context is something that's going to be very dynamic.
Liberty (09:56):
To wrap this up, we've just talked about the negatives. What are some examples of the true incredible positives that are going to be able to come out of this for society?
Joe Paradiso (10:06):
When we did our Here There project at Tidmarsh, the cranberry farm... It's a wetland. We have sensors all over the place and tunnel them through the virtual world. We had an app where you wear bone conduction headphones. It didn't cover your ears. It just pushed into your head. So you would hear through the vibrations on your skull essentially. It was fairly innocuous. You really didn't notice it, but you heard these things as well as hearing through your ears. And we had it channeled into the microphones we had all over the place to pick up the wildlife. It knew where you worked because you had GPS. It knew the angle of your head because we inertia measurement unit to track it. And then we'd tunnel in the audio from these sources as if you were hearing them. They would move with your head, so on and so forth.
(10:45):
It was remarkable. I was expanded. I was not diminished. I didn't want to take it off. I put on a VR headset. It's kind of fun to play some stuff, but I get fatigued. I get sick. I'm relieved when I take it off. I didn't want to take this off. It augmented me rather than diminished me. And I think we have a chance to develop interfaces in systems that really augment people in different ways and don't diminish them. In all aspects, I think that's what we have to go for. I think there's tremendous promise there.
Liberty (11:14):
Okay. I think we definitely have a clear view on the metaverse, but I also think it's fair to say that a lot of people have mixed feelings about how this will change society. Joe mentioned in his experience with virtual reality that he felt that technology added something refreshing to life. He didn't feel that it took away or cheapened reality. But are there valid reasons for people to be concerned about the increasing use of metaverse technology?
Scott (11:48):
That's a good question. One of the things people are concerned about is how hard will it be for people to access the metaverse from a financial standpoint? In order to make the metaverse inclusive, businesses need to be able to securely exchange funds that aren't exclusive to any one country or continent. So it seems likely that the metaverse will turn to cryptocurrencies like Bitcoin or Ethereum, and some transactions may also take the form of non-fungible tokens or NFTs. And cryptocurrencies aside, will the cost of the equipment like VR and AR glasses create an equity gap?
Liberty (12:18):
Another thing that people are really worried about is obviously safety. Recently, Meta was testing their VR platform called Horizon Worlds when users alerted them of this totally new kind of threat. This virtual physical and sexual harassment. Meta responded by adding a feature that created a four foot virtual bubble between non-friend players. But what happens when a user does what's called modding or modifying the game and they create a program that allows them to violate this four foot barrier and impose on the player in some way? How will the metaverse punish virtual crimes?
Scott (13:04):
There are clearly a lot of questions and concerns for what the metaverse would actually look like day to day. So we're going to take these questions to Eric Ravenscraft. When you Google what is the metaverse, Eric's article is the first one to come up. Eric is a writer for Wired, where he has guided readers on how to use technology for nearly a decade for publications like Lifehacker, OneZero, and The New York Times. The metaverse is such a big term and it takes a bit to wrap our minds around what the word metaverse actually means, so we're asking everyone. We talked with Joe Paradiso from MIT's Media Lab about what the metaverse means to him. We wanted to talk to you as well. How would you explain the metaverse?
Eric Ravenscraft (13:43):
I sort of have two branching answers to that. One is that it is sort of a collection. Companies are referring to a collection of technologies ranging from virtual reality, augmented reality, a collection of different types of sensors and networking devices that enable motion tracking or whatnot that go beyond the capabilities of just what we typically think of as the internet, which is your phone or your computer connecting to a website. And that is a very wide array of technologies. Some of which even work in opposing directions to each other.
(14:17):
The second definition that I think is more accurate for the metaverse as a phrase is that it is a marketing term. The conversation around the metaverse kicked off when Facebook announced that it was rebranding to Meta because it's planning to build the metaverse. This idea of a virtual world where everyone will socialize and buy stuff and there will be a lot of transactions for digital clothes and a merchant to take a cut of those sales is a narrative that Facebook has a vested interest in promoting. But the narrative of what the metaverse is and the technology that actually exists are very often divergent from each other.
Liberty (14:59):
One of the problems that we've seen... If we describe the metaverse, just for the purposes of this question, as what I imagine it to be, where it's like VR headsets or Oculus headsets and you're living in this virtual reality, I would imagine that, to do interesting things, you're going to need stuff. You're going to need equipment. You're going to need software. You're going to need stuff. And that there's a huge cost as things goes on and on to that. Is there a chance that the metaverse, or what I imagine the metaverse to be, is actually going to end up creating an equity situation? It's going to make it so that people who have money or who can buy stuff are going to be able to participate in these things and people who don't can't? Is there any issue with the metaverse in terms of the socioeconomic aspect of our country or our world?
Eric Ravenscraft (15:54):
Oh, absolutely. Yes. We already see that a little bit in the internet of today. You need a laptop or a phone, you need a cell connection, in order to connect. And those things are relatively cheap and relatively portable. You can put a laptop on your lap. You can carry a phone in your pocket. But let's say that the metaverse is going to be VR based. If you have a VR headset today... I have a Quest 2 in my house. In order to use that, I need about a living room's worth of space to set up and use that, which means, among other things, I need a living room. I briefly lived in DC and there are very small apartments there. I'm not sure I can allocate space for a metaverse workstation. And headsets are already... They're getting less expensive, but they're not useful enough to justify the expense in the same way that a phone does. And they're on top of a phone. They're on top of a laptop.
(16:45):
It's one more device to buy. There's space needed in order to use them. We don't know what bandwidth requirements would be for these if we got to a really high detailed virtual analog for a physical world. So I do think that there are absolutely socioeconomic challenges to getting people to use the hardware for this. And then it sort of creates a chicken and egg problem. That if not everyone can use the hardware, how do you develop the software for it? And if no one's developing the software, why buy the hardware for it? There are a lot of challenges in getting even a hypothetical virtual world like this up and running.
Scott (17:22):
But as we defined earlier, the metaverse is not just a headset you put on. It's augmented reality phones, glasses. If you and I and Liberty are talking about this 10 years from now, where do you think it's going to be? Is it going to be closer to Ready Player One or is it going to be closer to we have a lot of individual gadgets that augment our world and allow us to interact differently, but not necessarily ensconce and envelop our lives?
Eric Ravenscraft (17:47):
I love talking about what may happen, but the one thing I am comfortable saying is we will not be living in Ready Player One. It's challenging because part of the issue is that we don't really have a good sense of scale of the technological requirements for a Ready Player One style metaverse. Intel came out recently and described that the amount of computing power needed for something like that is like a thousand fold beyond what we have now, but in the next five years we might get tenfold. It's just the scale required. And as processing power gets more powerful, we tend to fill that up. There's a reason that video games don't load instantly even though we have massively more powerful systems now than we had 20 years ago. It's because, when the computers get more powerful, what we ask of them gets more intense. Fortnite could be photorealistic, but you couldn't do it with a hundred people in an instance.
(18:43):
So, that I don't think is likely or even possible, but I do think that we have a lot of exciting technologies that could help us augment our world. One thing I'm really interested in is a photogrammetry. There's an app for an iPhone called Polycam. You basically take a few hundred photos of an object or you just take a video walking around it and it will spit out a 3D model of that object. You could take that into a video game. You could make a movie out of that. You could build your own virtual sets. Those kind of things are really exciting because that lets you connect your phone to a virtual environment to real practical things you can make. And that I think is very exciting. But will that rise to the level of Ready Player One? I don't know if it's technically possible in the next decade, and I'm not even sure people want that.
Liberty (19:35):
Just to get back to this idea of... My research area is photogrammetry. So just to get back to this concept of, okay, the metaverse could allow you to take a 3D image of your... create a 3D model of yourself and all of a sudden... or of anything. There's really cool aspects to that. But you also then get into the really potential dangers of what we already see with deep fakes or people making it look like our president is saying one thing or doing one thing. You get into some serious dangers. Could you talk to us a little bit about what may be the dangers that we have to be aware of as we move forward into the next 10 years or whatever it might be?
Eric Ravenscraft (20:19):
I think that's especially an interesting problem because... For example, we already have a problem today with people going on Twitter and pretending to be someone else. Someone pretending to be the president is obvious. If the president starts saying a bunch of swears on Twitter, it's probably not the president. But it is very easy to impersonate someone else. Like someone you don't like at your school. Someone you don't like at work. You could create dozens of accounts impersonating someone with just a photo and a couple of tweets. In a virtual environment, we haven't really considered how... Could you pretend to be someone else?
(21:00):
There's also new forms of digital harassment. It took almost no time for Facebook to have to create new rules about how close two people could stand next to each other in a virtual environment. Because originally, someone could walk right up to you and just be right in your face, making it impossible to see what's going on through your camera. So Facebook said... Or Meta, I suppose, said, "Well, now, we'll create a barrier." You have to be X distance away from each other and you can't push into that space. But then that raises questions of could five people surround you and make it impossible to move in a virtual space? Could they physically blockade? These are just questions that the systems, assuming they even take off at all, have to be built with the possibility of harassment and impersonation and abuse and crime.
Scott (21:54):
I guess my last question is, are you hopeful? Do you have a positive outlook on what the concept of the metaverse bring or are you more worried? Not that it's all rosy. But do you think it's going to be a net positive, the influx of these technologies and these experiences, which is what the terms I will use to describe the metaverse, or are you more worried about the problems they bring, which we've been kind of talking about for a little while?
Eric Ravenscraft (22:19):
I think in terms of the technologies that we see today... Augmented reality, motion tracking, photogrammetry, for example. Those are all technologies that excite me. I love seeing what you can do with those. I've been obsessed with virtual production. I'm somewhat of a film guy and I'm obsessed with virtual productions and stuff. That excites me. I also get excited about platforms that allow people to express themselves creatively. If there becomes a virtual reality platform that enables creative expression in a way we haven't seen before, that does excite me. However, I think that when we talk about the metaverse, that is where I get a little more cynical, I'll admit. Because when we talk about technology in vague and conceptual terms, it becomes easier to obfuscate problems with systems like harassment or discrimination or whatnot. I don't think we discussed the concept of discrimination in the metaverse. And it makes it difficult for people who have to buy this stuff, people who have to sign onto the platforms, to know what they're being involved with.
(23:25):
I've talked to a lot of people about the metaverse and, just like you, they have almost no idea what it means. And that's a tough place to be in when someone's asking you to buy a product, to use it every day, to sign onto a service, when you don't know what it's going to mean. So what I try to do in my work is focus on the specifics of what actually exists as opposed to grand visions. The tech industry as a whole generally relies on futurism. They rely on being able to tell you that this is the future, this is what's coming next, this is the grand shift that's happening. And sometimes there are big shifts that happen, but a lot of times there's a lot of technology that's just good for some things and that's fine. It's okay for VR and AR and all these technologies to be useful for a few cool tasks rather than being a big shift that fundamentally changes society every 10 years. Because we do need to consider the challenges society already faces before rushing to apply technology to a social task it's not suited for.
Liberty (24:43):
We asked today, what is the metaverse? While I feel that we know a lot more about it than we did at the start of the episode, it is very clear that the metaverse itself is still being defined.
Scott (24:57):
The experts we talked to agree we aren't going to be living in a Ready Player One world anytime soon.
Liberty (25:02):
I know that's hard for you, Scott.
Scott (25:03):
Yep. Shedding tears right now. It's not because of lack of vision or missing link in the technology it's just because we're in the very early stages, where computers and software haven't yet caught up to the vision. It's a path many researchers and companies are working on today. Just hopefully they can move faster.
Liberty (25:21):
I think that the more the metaverse becomes part of our reality, the more we're going to have to consider the challenges it's going to present. How could it manifest similar problems we see in the real world like harassment or theft or discrimination? And being ahead of these problems is a huge part of the battle. I think we're going to see companies like Meta handle it going forward.
Scott (25:51):
So while disappointing we're not going to be in Ready Player One immediately, in the meantime or at least the near future, it's interesting. The metaverse could allow disabled people to interact and experience things like seeing, hearing, interacting with others, in a way they currently can't in their non-metaverse lives. And eventually, the metaverse will fully define itself and technology will start living up to the science fiction novels. Based on what we learned today, I'm expecting and hoping a much bigger impact on how we work, play, and go about our day to day lives.
Liberty (26:23):
Thanks much for listening to this episode of Day Nation. This podcast is brought to you by MIT's Institute for Data Systems and Society. And if you want to learn more about what IDSS does, please follow us @mitidss on Twitter or visit our website at idss.mit.edu.