Interviewing Epic Games Founder/CEO Tim Sweeney and Author/Entrepreneur Neal Stephenson — MatthewBall.co

Ball: So when I was writing this book, I was reading a bunch of Richard Bartle’s old published works, and he had reported on a statistic that I thought was remarkable, which was as late as 1993, a full 10% of all global internet traffic was just for MUDs.

Sweeney: That was to the point where you realized that the graphics can help the experience, but they’re not essential. Our brains are willing to fill in massive amounts of detail. In my memory of reading Snow Crash, I actually see images of it though it was just a book. From the early MUDs to playing Doom with 320 by 200 pixels of real estate on the screen, my memories of are in fully formed photorealistic fashion. And I think that’s the big lesson that we don’t need hardware that plugs into our brain and creates a sense of immersion that’s indistinguishable from reality. What we can do with just the devices right in front of us is totally not the limiting factor now.

Ball: Neal, I’d love to get a little bit deeper into Lamina1. You co-founded it in 2022 with one of your colleagues from Magic Leap, Rebecca Barkin. Can you explain what Lamina1 is, how it works, what it’s trying to do, and its criticality to the Metaverse as you imagine it?

Stephenson: If we’re going to have a Metaverse that millions of people use, we need to have experiences there that people enjoy having, which seemed like kind of a obvious statement to make, especially in present company. But the people who know how to make those experiences are the creators who by and large are employed in the video game industry or more and more in the motion picture and TV industry. They’re the people who know how to run game engines and who know how to run the tool chains that feed assets into those engines. So the question is, what’s the revenue model that allows those people to get paid? And there’s various answers to that question. We have conventional payment systems that work, but in our minds, there was an overlap there with some of the qualities and capabilities of blockchain systems. And so Lamina1s a new chain that’s optimized to help creators build things, to help them get paid for building things and creating experiences or components of new experiences in an open Metaverse.

Ball: Crypto has become one of those more controversial elements among Metaverse aficionados. There are those who are ardent believer’s in blockchain technology’s criticality to the Metaverse. You have those who are entirely skeptical about the technology, not just unconvinced of its relevance. And then comparatively few people in the middle who largely say, “not sure; we’ll see.” What in your mind makes blockchain such a viable, helpful, if not essential technology for building an open Metaverse?

Stephenson: I think a lot of the skepticism and hostility that we’ve seen, particularly in the game development industry, just comes from this clash of mindsets between people who are strongly ideologically motivated coming out of a libertarian crypto kind of mentality, versus the people who build these experiences and who actually understand how game engines work. And there’s a dream of interoperability, which is the idea that you could essentially drag and drop assets from one game into another, which is quite reasonably seen as threatening and even kind of insulting to people who spend their lives crafting beautiful AAA games. And it’s also technically ridiculous. If you actually understand how these things work, you can’t just take an asset from one game and somehow drop it into another. So I think that it is possible to engineer new experiences from the ground up that are designed to support interoperability and that there’s some overlap between what that is going to look like and the kinds of payment systems that you can construct, and the more importantly, the smart contract systems that you can construct on top of blockchains.

For me, money is kind of the least interesting of applications on blockchains. It’s the one that’s gotten the most attention because they were mostly started as financial instruments. But with the rise of NFTs, we saw this idea that you could essentially put pieces of art up on a chain. And what was then discovered was that the smart contracts that governed these NFTs weren’t smart, and they weren’t contracts. They actually weren’t enforceable. And so some work has been going on in the last couple of years to bring the NFT market into the realm of legal enforceability. The late Josh Kramer at Grapevine developed some technology in this area, but unfortunately passed away last fall. Mattereum is a company in the UK that’s building royalty systems based on legitimate UK law.

And our friends at Shrapnel have been developing an actual running AAA game that embodies some of the features I’m talking about, where creators can post things on a chain that establish a kind of chain of IP development, IP ownership that’s traceable, and that should in theory, allow them to get rewarded economically if their stuff succeeds. So that’s I guess kind of the quick overview of a really complicated topic.

Ball: Tim, I’m curious about your perspective on blockchain technologies.

Sweeney: The underlying idea of blockchains is awesome nerd technology. There’s great use of cryptography, great protocols for distributed agreement on events, and a really interesting foundation for the future of distributed computing systems of all sorts, including the Metaverse. It strikes me as very unfortunate that it didn’t have another couple decades to be nurtured in the purely nerd community before it was adopted as a financial instrument, because the currency has been greatly undermined by speculation and scams and regulatory uncertainty and so on. And I feel like it’s a very unfortunate artifact of this decade. But in the future, the ideas from various blockchains such as zero knowledge proofs, the idea of cryptographic consensus protocols and so on, should be a key component of a lot of systems. And if we would only stop clawing the money and stop building financial scams around them, then they could be a great part of a future society.

I think that it takes a lot of discipline in the minds of technologists to separate the good from the bad of crypto. There is actually a great deal of good in the technology, separate from the bad uses of it that we’ve seen over the past, and I think we should be open-minded to the learnings to be made from there. Perhaps in a decade or two, we’ll look back and be like comparing the cryptocurrency period we went through now to the 1990s dot-com crash. Underneath all of that, the internet was really solid technology and there were some companies built up in that timeframe, like Amazon actually did extremely well and thrived, but there are also scams layered on top of it. But the world recovered from that and is doing great now. And I think the world will end up on the right side of blockchain technology in the long-term future. And in the meantime, it’s still a rather wild west so buyer beware.

I think the future of the Metaverse has to be built on open protocols, open standards and interoperability of all forms. We need both technological interoperability so that any creator, any hosting provider, any ecosystem operator, any brand and any content can interoperate freely without being forced into any one company’s walled garden of any sorts. And this isn’t really even an attempt to paint a utopian picture, but just that there are going to be lots of places in the Metaverse and a lot of these places will be owned by companies, but we mustn’t allow any one company to dominate or control the thing overall. And this both goes for the underlying technology and standards, but also for commerce and notions of ownership and economic interoperability in the Metaverse.

When we designed the Fortnite Creator Economy 2.0, the key principle is realizing there are two things happening in the Fortnite economy. There is value being created through engagement, people building fun experiences. Third-party creators as well as their own teams are creating value engaging players. And because players are engaging and are happy, they’re spending money in the Item Shop. And so the key became to share the Item Shop’s revenue, which is a source of spending, with experiences, which are the source of engagement, and build an economy that scales based on that. And from the very beginning, the idea was that we’re operating a Fortnite version of this initially, but in the long run, there’s no reason that this Creator Economy 2.0 couldn’t be extended into a Creator Economy 3.0 where any company could participate however they choose.

And besides participating in technical standards, if another ecosystem with similar or compatible visual aesthetic and respect for games ratings and so on wanted to participate, then perhaps we could connect our economies where if you spend in the Fortnite Item Shop and then play in a third-party economy, then we revenue share to them. And if a third-party item shop sells something and then it’s used in Fortnite, they revenue share to us. And just as internet hosts agree on peering arrangements to connect their fiber optic lines, that revenue sharing can enable an open Metaverse economic model. And I think this is one of the exciting things that we’ll see happen within this decade. And Epic has been on a very long-term trajectory to build out all of this tech and to do it in a thoroughly open way.

We have Unreal Engine, which is a huge engine, but we expect the ultimate Metaverse technical standards will be engine agnostic and that you could participate in the open Metaverse in the game that’s built using the Unity engine or the Godot permissively licensed open source engine.

And while you’re doing that, you could use any number of online backends. You could use the Epic Online Services social backend for voice chat or you could use Sony’s PSN or Microsoft’s Xbox Live or Valve’s with Steamworks. And that perhaps if these companies would actually cooperate in the right ways, then we could build an entire economy that links all the major games and all of the platforms together into an economy. I think it’s actually in everybody’s best interest. This isn’t like the smartphone walled gardens. What’s happening in the Metaverse is Metcalfe’s Law at a huge scale. Players want to be in a place where they can play with their friends. If we can connect our voice chat systems and our economies and players can move seamlessly together with their party and with their purchases where they’re compatible, move from say, Fortnite to Roblox to Grand Theft Auto to PUBG Mobile, and then to a pure chat type of application as well, then the world would be a better place.

But not only that, but the companies participating would actually make more money because we’d see an engagement lift from the ability to reach more customers through this interconnected economy. And all of our competitors would gain more revenue too because of the increased opportunity they have to actually reach customers and players would play more and they would spend more, because buying an outfit in this future open Metaverse could be owning it everywhere they went and not just in the one Roblox experience or one Fortnite ecosystem that they’re participating in as is currently the case. I really think that this is going to happen. And it doesn’t rely on any altruism for those participants, but it’s in everybody’s interest and that this world of the open Metaverse will just be purely better than the separate game worlds that we have today.

Stephenson: You don’t want to have to stop at the exit of every Metaverse experience and take all your clothes off and then step across the threshold into a different experience and then put on the clothes that you’re allowed to wear in that experience. You want to just walk through. And that seems so obvious that people just assume that’s the case. It seems like you shouldn’t even have to mention that, but it takes a lot of technology to actually make that work.

Ball: The seventh anniversary of Fortnite comes up next month. Seven years from now, how do you think of Fortnite as being different? I think in the typical fan’s perspective, Fortnite right now is Battle Royale, which they see primarily as a game. They see the Metaverse play primarily around UEFN. And the interaction model for UEFN feels a little bit like an app store or Netflix or YouTube. You’ve got thumbnails and rows and rows of them, and that’s how you navigate these different 3D worlds that are lightly connected through avatars and aesthetics and user identities. What do you think the platform looks like another seven years from now? What’s your dream experience?

Sweeney: It’s going to evolve a lot. And when you look at what’s in Fortnite today, some of it you have to recognize as artifacts of the limitations of the current technology that we’re working within. Why is Fortnite: Battle Royale 100 players? Well, because at the time we launched it we couldn’t make 200 players work on a server. Computers in the data center were just too slow. The reason we have Fortnite divided into a huge number of different islands, many built by third-party creators and some built by Epic, is because we don’t yet have the entire technology stack needed to robustly enable every creator to put their content together into a big, seamless open world if they wanted. And so a lot of the things you see in there are not the permanent end state of what we see this medium being, but are just current crutches that we’re using to hobble by as we work towards the ultimate capabilities of the thing.

And so I think we need to expect really significant changes in a number of areas. One is being able to build an interoperable economy that works with other games and other ecosystems is a key that will be really freeing for people, being all of the systems for voice chat and account interoperable, federated so that you can participate with any of the major platform company services rather than each Fortnite using ours and every other game using their own is going to be a key part of it. Another is the networking model, which is extremely limited. If you look at what’s in Unreal Engine 5 today, it’s remarkably similar to the networking model I built for Unreal Engine 1 in 1997. It shipped in Unreal and Unreal Tournament, and it’s been incrementally improved ever since without dramatically upending it.

But the problem with this network model is it doesn’t enable our servers to talk to each other. A Fortnite Battle Royale session is 100 players. There might be at peak hundreds of thousands of these servers running and there might be at peak over 10 million concurrent players online all at once, but they’re each in their own separate sharded copies of the world and they can’t see each other in that space. They can’t go anywhere to find each other all at once.

So one of the big efforts that we’re making for Unreal Engine 6 is improving the networking model, where we both have servers supporting lots of players, but also the ability to seamlessly move players between servers and to enable all the servers in a data center or in multiple data centers, to talk to each other and coordinate a simulation of the scale of millions or in the future, perhaps even a billion concurrent players. That’s got to be one of the goals of the technology. Otherwise, many genres of games just can never exist because the technology isn’t there to support them. And further, we’ve seen massively multiplayer online games that have built parts of this kind of server technology. They’ve done it by imposing enormous costs on every programmer who writes code for the system. As a programmer you would write your code twice, one version for doing the thing locally when the player’s on your server and another for negotiating across the network when the player’s on another server. Every interaction in the game devolves into this complicated networking protocol every programmer has to make work. And when they have any bugs, you see item duplication bugs and cheating and all kinds of exploits. Our aim is to build a networking model that retains the really simple Verse programming model that we have in Fortnite today using technology that was made practical in the early 2000’s by Simon Marlow, Simon Peyton Jones and others called Software Transactional Memory.

The idea is that you write normal code and it’s our job as the implementors of the engine and the language runtime to make your code scale, so the game can run on a vast number of servers and to do all of the necessary coordination and to provide the guidelines. If you optimize your code in a certain way like you optimize for cache coherency today, then we want your game to be able to run in a much larger simulation than we’re running now. This is one of our focuses for Unreal Engine 6, and it’s going to consume an increasing portion of our engine team’s efforts as we work on this. And the other is the ability to combine as much of the content together into a seamless world as players want. Some experiences will be better by themselves. If you want to build an awesome bespoke story-driven, single player or a co-op game, you might build it off in its own little corner of the world, no connections to the outside, but an awful lot of what we’re doing would be a whole lot better if it were all seamlessly connected.

As Disneyland is itself, you get on all of these elaborate transport systems like the People Mover and the different cars go from place to place. You can go anywhere in this connected world and participate in any experience there. And what are creators doing instead of creating their own little isolated islands? They’re taking over a portion of space in the world and they’re defining the game roles there in different parts of space.

We’ve done little experiments here and there along the way. Back in Fortnite Chapter 2 (Note: This eight-season chapter began in October 2019 and ended December 2021), there was a period where there was a bubble appearing around certain parts of the world. It changed the gameplay in that part. Imagine that writ large in the scale of a simulation with hundreds of millions of players and hundreds of thousands of creators.

The final bit is interoperability of content and code. Battle Royale is mostly code written by Epic. Every creator’s world is a mashup of their code and Epic’s code. But things become really interesting when every creator’s code can interoperate with every creator’s code.

Everybody’s out creating their own really interesting objects and creating them using protocols that are provided by the system to enable them all to work together. So you might be riding a mount or an animal built by one creator and your friend might be driving a car built by another creator. You might be carrying a weapon built by a third creator, and you might be in a world maintained by dozens of other creators, and you might be moving seamlessly from place to place with all of these interactions happening. And you really expect that to work. An awful lot of the reasons that we’ve built Verse and our ecosystem the way we have is to allow for these future usage cases.

If you just wanted to get Roblox style experience with a bunch of sharded islands deployed as quickly as possible, there were much faster ways we could have done that and much simpler trade-offs we could have made in the language and in the engine to achieve that. But we’re building for the long term, and by the end of this decade I think an awful lot of this will have come to fruition and you’ll see the ability of creators of all sorts to build things that are qualitatively different and better than they are today.

Ball: Neal, we talked a little bit about Lamina1, and you have a novel coming out later this year that I want to get to, but you’re also an advisor to Inworld.AI and a co-founder at Whenere. What are those latter two companies about?

Stephenson: Yeah, so this has to do with what would I want to experience in the Metaverse. – where is the first place I would go. My co-founder at Whenere is Karen Laur, who was employee number 17 at Valve. She worked on Half-Life 1, and we ended up working together at Magic Leap building creative projects there. And towards the end of that time, we were messing around quite a bit with Sequencer, which was a component of Unreal Engine that is there to help people who want to make cinematic experiences. And through that, we got to know Kim Libreri and some of the team at the Northern California branch of Epic who were working on making Unreal Engine a tool for people who work in film and TV, not just games. And so what came out of all of that is this realization that graphics hardware and game engines have reached the point where you can now build immersive environments that most people would identify as of cinematic quality. If you’re a professional movie director, you might see that it’s not quite up to that level, but most people are essentially going to consider it a photorealistic environment. So that was kind of the first element of this.

And then what came on a little bit later was Inworld.AI, and they’re building a system that essentially makes it easy to connect large language models on the back end to avatars in the game engine. In effect, you can use a simple interface to essentially build the brain of a character. You can specify what this character knows, what they don’t know, and what their personality traits are. You can map them onto a voice from a different company. The one we are using at the moment is called ElevenLabs. And then you can wire that in to an avatar in the game engine. We’re using Unreal, we’re using MetaHuman, which is another kind of cinematic feature that’s been added to the engine in the last few years, which basically makes it easier for people to create, again, nearly photorealistic human avatars. So what it all adds up to is that to game in Whenere, all you have to do is talk.

We have a speech-to-text subsystem in there that will transmit whatever you said to the character’s brain in the backend. The brain will generate an appropriate response based on the knowledge base and the personality of that character, and send it back to the game engine as an utterance, which is it’s text, it’s the sound file generated by ElevenLabs, and it’s a set of visemes, so the atoms of facial animation. So that the character will exhibit the right facial expressions and even the right emotional expressions that have been generated by the brain.

And we first encountered this when we were working with an internal production team at Inworld to make a character Virj, who was part of the extended Snow Crash universe timeline. And we went in sort of with modest expectations and were just astounded by how well this thing worked and how interesting the results could be. And so I think zooming out for a second, the door that this opened for us is that the way that we interact with video game worlds has tended to be pretty limited, and most games still revolve around shooting things because that is a perfect match for the UI input devices that we’ve got. You use a mouse to put the crosshair where you want it, you click the button to fire the weapon or whatever, and you see something happen in the world, the world responds in a way that makes sense, and that’s been a very powerful UI paradigm. But here for the first time, we were able to just sit and talk like normal people to this character and have an answer back.

So based on that, we, Karen and I and Jamil Moledina co-founded Whenere last summer and we’ve been working since then on building a system that sits on top of Unreal Engine and on top of Inworld.AI, and its purpose is to enable users to immerse themselves in story worlds that they love. You go to Comic-Con, you go to any kind of fan environment, you see people who’ve traveled thousands of miles, they’ve spent thousands of dollars to make elaborate cosplay outfits, and it’s all in the service of I love Game of Thrones or Harry Potter or Lord of the Rings or whatever, so much that I just want to spend more time in it. And we think that there’s a way now to give those fans the ability to enter into such worlds, and not only to interact with them, but to mod those worlds, to come up with new ideas, new fan fiction storylines, or just to change the way the furniture’s arranged or the way a character looks or talks.

So we’ve been working away on that since about August and have been fairly quiet up to now, but we’re going to be talking about it more in the coming weeks.