2.24.2010

History of the Unreal Engine




The Epic evolution of gaming's most influential engine, from Gears of War to Mass Effect 2.

February 23, 2010 - Every good story starts with one person alone in a room deciding to try something new. Tim Sweeney was just twenty-one years old when he designed ZZT, the first official game from Epic Mega Games. Sweeney, then a student at the University of Maryland, had already spent ten years as a novice programmer, but the completion of ZZT was a milestone. In the interim Epic has gone from a bootstrap shareware developer to one of the pre-eminent design studios in the world. They've also led the industry in building one of the most widely-used and recognizable game engines in the world. Their technology has been used for everything from Deus Ex to Harry Potter. During the generational transition to Xbox 360 and PlayStation 3, Epic's Unreal Engine 3 became the development tool of choice for the world's biggest publishers and franchises. This is how it happened.


The ABC's of ZZT
ZZT was a shareware puzzle game with simple graphics even by 1991's standards. The game was an abstract dungeon puzzler that had you guiding a small white sprite through a maze of locked doors, treasure, and enemies that often resembled letters of the alphabet. The content of the game wasn't so revolutionary, but it was the approach to programming that would ultimately germinate into something much bigger.

"I wrote ZZT in Turbo Pascal, using an object-oriented programming style, and designed ZZT-OOP to provide easy control over gameplay objects without the complexity of a 'real' language," Sweeney told me. The idea for object-oriented programming, a way of defining quantities of data as an "object" and creating a system for how different objects should respond to one another, dated back to the mid-sixties. By 1991, the games industry was still a relative ether of potential energy. Even the biggest games were made by a handful of people and each new game was basically built from the ground-up in a process that required the precise work of engineers and programmers as much as it did creative daydreamers.

Sweeney's implementation of object-oriented language in ZZT changed all of that. In making it a central part of the game's code he allowed significant user modification above and beyond the simple map editors of the day. More importantly, this approach laid out the conceptual framework for the idea of a game engine.

"ZZT served as a conceptual blueprint for Unreal," Sweeney said. "A game engine with a high-productivity, what-you-see-is-what-you-get tools pipeline, bundled with a programming language aimed at simplifying gameplay logic."

In the intervening years, Sweeney slowly built a reputation for Epic in the primordial ecosystem of shareware games, with titles like Epic Pinball, Jill of the Jungle, and Jazz Jackrabbit while adding key staff members like Cliff Bleszinski. Steve Polge, and Mark Rein. After the creative explosion around first person shooters like Wolfenstein 3D, DOOM, Duke Nukem 3D, and Quake, Epic began work on their own shooter with 1998's Unreal.

Developed as a serious competitor to id's Quake II, Unreal let players control a marooned prisoner on an alien world, trapped in between warring alien species. The game was arguably the best-looking shooter of its time, handling large, highly detailed indoor and outdoor areas with uncommon ease. Fans of history will enjoy noting that George Broussard was so impressed with Unreal that it prompted him to move development of Duke Nukem Forever from the Quake II engine to Unreal Engine, the very first of many delays in the game's still unfinished production.

Look, I'll Pay You For It
The idea of using borrowed technology was not new in 1998. Capcom and Konami had both been recycling internal technology with their various platformer and brawler variations (e.g. DuckTales, Rescue Rangers, Teenage Mutant Ninja Turtles, The Simpsons Arcade Game). Likewise, id had made its tools available to a carefully selected number of partners (including Valve).
As with object-oriented programming, Epic didn't invent the idea of licensing their engine, but they put the idea to better use than anyone had yet done, creating a massive money-maker from the seemingly simple idea.

"After we began showing an early version of Unreal, our two first licensees – Legend Entertainment and Microprose – called us up and asked us about the possibility of using our engine in their games," Sweeney said. "We were thrilled by the opportunity and our early collaboration with those partners defined the style of our engine business that remains today: a community-driven approach, and open and direct communication between licensees and our engine team."

While the end results of Epic's first version of the Unreal engine set a technological standard, the focus was more on creating cohesive tools and streamlining the technical hurdles for creativity. "The Unreal Engine is really focused on providing great tools and workflow for artists and designers," Steve Polge, Lead Programmer at Epic, told me. "By doing so, it really removes barriers to achieving their creative vision and dramatically improves their productivity."


At the time, Unreal was stunning.
Unreal was released just before the revolutionary shift to dedicated graphics cards. All the games of the day had to be rendered in software, meaning development of the Unreal toolset was as much a process of refinement and efficiency of the rendering models of the past. "Over the course of 3.5 years, I rewrote my core rendering algorithm at least 7 times, finally settling on a two-level scheme that enabled soft shadows, dynamic lights, volumetric fog, and efficient texture filtering," Sweeney said.

Another huge breakthrough was the ability to change the level layout on the fly, something other level editors at the time didn't allow. "The major initial challenge was the Unreal Editor's support for real-time geometry editing, including performing Constructive Solid Geometry operations in real-time," Sweeney said. "I was new to 3D programming, and found the problem mind-boggling, but there was one 24-hour stretch of programming where I got all of the cases right and CSG went from not working at all, to working perfectly. It was immensely gratifying."

In the second half of 1998 Epic continued to focus on developing the Unreal universe with the expansion pack Return to Na Pali. In 1999 Epic released Unreal Tournament (developed with Digital Extremes), taking all the art and lore of the original Unreal into a competitive online shooter that became the center of one of the most active and loyal modding communities in the world.

But Will It Play on PS2?
For Unreal 2.0, Epic shifted its strategic focus to include support for PlayStation 2, Xbox, and Gamecube, prefiguring the migration from PC to console development that would happen just a few years later. For years console and PC development had seemed like two separate tracks that would be irreconcilable for the ages. PCs would perpetually be more powerful and specialized than their laggardly little brothers in the living room. Unreal 2.0 was an attempt to build a shared tool that would make multi-platform development possible.

It was also a sign of the times that Unreal 2.0 debuted with America's Army, a licensee and not an internally produced title. In the same year, the artist Mary Flanagan also turned to Unreal 2.0 for "Domestic," which offered a haunted passage through the memory of a burning house, mixing poetry, cinema, and nostalgia into an interactive first person exploration. It's a testament to the emerging flexibility of Epic's toolset that two such dramatically different experiences, one a military recruiting device and the other an emotional reverie, could be created using the same technology.


America's Army on Unreal Engine 2.0
Unreal 2.0 also moved the business of engine licensing forward with an unprecedented volume of deal making at the highest level. Unreal 2 was used for Tom Clancy's Splinter Cell, Deus Ex: Invisible War, EA's Harry Potter games, Red Steel, BioShock, and Mirror's Edge (to name only a few big ticket games). It was also during this period that the very idea of a game "engine" began to stretch. In the mid-90's "engines" were often used to make clones, or DOOM-likes as the FPS was once called. With the advent of Unreal 2.0, saying something built on a particular engine didn't necessarily dictate the kind of experience you would have.

"An engine has nothing, or at least very little, to do with how a game looks visually," Sjoerd de Jong, long-time Unreal modder and developer of The Ball, told me. "You could produce an Unreal Engine game that looks like a 2D retro game, a realistic war game, or a brightly colored cartoon game for kids."

The additions to Unreal 2 were largely focused on integrating a wider variety and depth of the available tools. The engine had integrated tool to support Karma's physics system, the rendering system was rewritten for even more efficiency, a new particle system was added, and the Unreal Editor was updated for usability and performance.

"The Unreal Engine is not a static piece of software," Geremy Mustard, of the Epic-owned Chair Entertainment, said. "Epic is constantly adding new features, optimizations, and improvements."

While continuing to develop its own internal projects, Epic has worked hard to build a support network and make its engineers accessible for licensees working with their technology. Likewise, a huge body of shared community knowledge has made the engine approachable for the new-comer. "The Unreal community is over ten years old. It has a good mix of experienced people who have been around since the early days, and many new people," de Jong said.

"When we did get stuck there was the ever helpful Unreal Developers Network, and the dozens of tutorials written by the community."

But Have You Played it in HD?
Having already made great headway in establishing engine-licensing as a major part of their business, Epic was even more ambitious with the release of Unreal Engine 3. While UE2 supported home consoles, it was released several years into their various life cycles, meaning the engine had to adapt to the out-dated hardware. With the third official iteration of the engine Epic had a chance to tailor the experience for the coming generation of HD consoles and harness their power from the outset. In 2005 Epic announced Gears of War as the graphical showpiece for what the Xbox 360 could do using their technology. Epic's Vice President Mark Rein later boasted that he and Sweeney were able to convince Microsoft to double the onboard RAM in the system (from 256MB to 512MB) after seeing the equivalent difference in graphics in a screenshot from Gears.

An early look at Unreal Engine 3
Sony was given a timed console exclusive with the PlayStation 3 version of Unreal Tournament 3 to fire a similar sense of graphical wonderment in its partisans. In the build-up to the launch window of each system, there was a steady string of announcements from Epic about new licensees for its nascent tech. Square-Enix, EA, Ubisoft, Disney, THQ, SEGA, Activision, Midway and more signed on for Unreal 3 licenses. Many of the most-anticipated titles in the early days of the 360 were powered by Unreal 3, including Gears of War, John Woo's Stranglehold, Medal of Honor: Airborne, Blacksite: Area 51, and Brothers in Arms: Hell's Highway. It seemed that Unreal Engine 3 might come to define development on high definition consoles.

But then rumors of development teams struggling with the new tools began to roll in. The big increase in technological capacity made stunning visuals possible, but it also made development significantly more expensive and complicated than it had been. As Epic continued to add updates to its engine in the run-up to launch, developers with milestones to complete often struggled to keep pace. In a post-mortem published in Game Developer magazine, team members from John Woo's Stranglehold lamented their time lost trying to make new versions of the engine work with content made using earlier versions. Then there were reports of the difficulty optimizing the PS3 versions of multi-platform games. After lack-luster receptions for The Last Remnant, Square-Enix president Yoichi Wada said the company's future use of the engine would be made on a "case-by-case basis."

The rumors all came to a head in mid-2007 when Silicon Knights, which had proudly announced their licensing of UE3 two years earlier, filed a lawsuit against Epic claiming UE3 never did what they had been contractually promised. They declared they would instead have to build their own engine for Too Human, causing major delays and millions of dollars in lost development resources.

In spite of the tumultuous start to the generation, the turbulent reports of struggle have abated. While the early years of this generation might have pointed to a near ubiquitous adoption of UE3, a healthy variety of different technologies have emerged. Meanwhile, Epic's technology has quietly continued in big performers like Army of Two: The Fortieth Day, Mass Effect 2, BioShock 2, and Splinter Cell Conviction. Some of those games are built on heavily modified versions of UE 2.5, but the core suite of tools that Epic built remains consistent.

"I think our tools have always been what makes Unreal Engine so powerful," Rein told me. "Recent improvements like UnrealKismet and Matinee have been a big factor in our ability to make games that are competitive on a world stage even with smaller teams than many of our competitors."

Chair Entertainment found all the new features in UE3 made it easier for them to transition to small team development after going all out with Advent Rising. "UE2 required programmer support for most gameplay tasks, whereas UE3 moved most of that power into the hands of level designers and artists," Mustard said. "This allowed us to make Undertow with only one full-time programmer in a very short period of time."

de Jong had a similar experience working with UE3. "It allowed us to get going on the game right away," he said. "We got a one-level-long prototype up in two and a half months, with three people on a part time-basis back in Spring 2008. A lot of the tools allow you to quickly see the end result, without touching a line of code."

After having worked with UE3, Mustard said it would be hard to imagine moving back again to UE2. "With [Matinee and Kismet] level designers can prototype almost any scenario in the game without the assistance of a programmer, which allows so much freedom and creativity," he told me. "I would cry if I had to make a game without these tools."

Take Me to Tomorrow, Please
One surprise that's caught many in the industry flat-footed is the advent of simpler browser-based and portable games that don't require powerful HD toolsets to captivate players. The explosion of the DS and iPhone, combined with Facebook's Flash-based gaming, has shown there's a lot of profitable game territory not well served by HD consoles. Epic recently announced a version of Unreal Engine 3 will be coming to the iPhone.


Undertow was made quickly by designers, not engineers.
"Mobile platforms like iPhone are very interesting," Sweeney said. "I'd expect, a decade from now, more users worldwide will be using advanced mobile devices for computing and entertainment than fixed computing devices like PCs and mainstream consoles."

While the iPhone version of UE3 won't let developers magically shrink HD games onto Apple's portable, there will be crossover in the tools. "On mobile platforms UE3 is able to run content created on the PC with the normal Unreal Engine 3 tools," Rein said.

"That means the investment our licensees have in learning the toolset and pipeline can also apply to these mobile platforms in the future. As the platforms get more and more powerful the need to have tools like UE3's will increase dramatically."

Epic has also worked closely with NVIDIA to get Unreal Engine 3 to run on their netbook-based Tegra chipset. "As Moore's Law takes over it will become a better and better gaming PC over the coming years so it can participate in the PC gaming market that already exists," Rein said. While there is a potential market rivalry heating up between netbooks and Apple's newly announced iPad, expect Epic to support both sides. "I'm confident the iPad will be successful and I expect our engine will be running on it before too long," Rein continued.

Unreal Engine will be coming to a mobile device near you.
In the meantime, Epic has continued its support of grassroots developers by releasing a free version of UE3 as the Unreal Development Kit. The tool allows anyone to download the basic toolset from UE3 and begin making their own game from scratch, not as a mod dependent on someone else's existing code. Epic continues to host its Make Something Unreal contest (last year's grand prize was $1,000,000), which seems only appropriate for a company with a history of discovering talent in the mod community (both Polge and Bleszinski won jobs on the strength of their own mod designs). And true to its core concept, Unreal Engine continues to power a fantastically diverse group of games, big and small. NASA is developing an MMO using UE3 and the US Army continues to use Epic's tech in interactive simulations. And then there's the Korean dance game MStar.

It's hard not to want to look back on the image of Tim Sweeney writing lines of code in his parent's house as a young twenty-one year old and tell him two decades later NASA would be using his technology, as would every major publisher in the video game industry. The years between then and now have been momentous ones, bringing breakthroughs and adversity. All the while, the idea of giving a better tool to those moved to create something for an audience remains the core of Unreal, and it accommodates everyone from Mary Flanagan to Hironobu Sakaguchi. Creating a platform through which those two people could create such wonderfully different experiences deserves a rank among the great historical achievements in the gaming industry.






MovieMiguel.com


blog comments powered by Disqus