AI is about to transform the future (and past) of video games

The Final Fantasy IX HD remaster should've been my jam. I've been a Final Fantasy nut since the Super Nintendo, and was full-blown obsessed with Japanese RPGs by the time Final Fantasy IX came out 20 years ago. Getting to replay an improved version designed for my HDTV should've been the perfect way to revisit Zidane, Garnet, and all my other favorites from Final Fantasy's 32-bit swan song.

But it wasn't.

Oh, how it wasn't.

Previously, I wrote about 240p Test Suite, which helps gamers get the best out of their old consoles. But as retro gaming continues to find new heights of mainstream popularity, publishers have been looking for new ways to sell their old games to a newly eager audience. So we get full-blown, AAA releases like Final Fantasy VII Remake, HD remasters like Crash Bandicoot N. Sane Trilogy, and spit shines like Final Fantasy IX for Steam, Nintendo Switch, mobile, and everything in-between.

Despite having access to my original copies, and a PlayStation hooked up to a CRT for gaming at home, I eagerly bought the HD remaster of Final Fantasy IX when it released for Switch in February of 2019. I only got about 30 minutes in before quietly shutting it down, disappointed and frustrated at having spent $25. Square Enix billed this version as a way to revisit the original with improved graphics, an updated interface, and HD support. What I got, however, featured a horrible new font, up-rezed character models, blurry, pixelated backgrounds for those models to skate over, and an infuriating soundtrack bug.

Truth is, there's a huge divide between the treatment offered to remasters of many older games. Final Fantasy IX was one among many recent official remasters that left fans feeling empty and realizing — maybe for the first time — that the corporations who created their favorite childhood games might not love those games as much as the fans do.

Fans have taken it upon themselves to improve games for decades, with major fan hacks like Neill Corlett's localization of Seiken Densetsu III dating all the way back to August of 2000. Hop back to the early 80s, and you'll find Super Missile Attack — a physical hack performed on Missile Command cabinets to add new features to the arcade hit. Even official releases like Ms. Pac-Man began life as a fan project. Over the past few years, groups of dedicated fans have undertaken their own remasters, often in the form of free downloads that require a legitimately purchased version of the game to play. For all of the million dollar budgets afforded to the official remasters, these fan efforts often become the definitive way to experience retro titles on modern hardware.

Moguri Mod: The little mod that could

The gold standard of fan remasters is Moguri Mod, "a faithful revamp of the PC version of Final Fantasy IX helped by deep learning techniques," according to the official website. "The most important changes are in the background arts, that are now cleaner, more detailed and higher resolution."

Moguri Mod is the result of hundreds of hours of work by a team of modders led by Belgian developer Gaël Honorez — better known online by his handle, Ze_PilOt. I spoke with Honorez and others about this mod, deep learning technologies, and the passion of the gaming community, in an effort to understand how small fan groups manage to beat large, money-laden corporations at their own game.

Just like the official products, fan remasters can vary wildly in quality, which is part of what makes Moguri Mod so appealing, Marc Lauzon-Bragg told me for this piece. Better known online as Stry8993, Lauzon-Bragg is a retro gaming fan from Alberta, Canada. "It respects the content, the game, and the artistic approach, and adds clarity to an old classic," he said. "It's lovingly done."

After installing Moguri Mod — a free download — players launch the Steam version of Final Fantasy IX, and are prompted with a settings panel loaded with options — from 16:9 support and orchestral music, to quicker battles, the original PlayStation font, and cheats. They can tailor the experience to their liking, and even pick up a save created without the mod.

“It respects the content, the game, and the artistic approach, and adds clarity to an old classic.”

Remasters of more recent games in the series like Final Fantasy X have more successfully captured the beauty of the originals, but PlayStation-era Japanese RPGs like Final Fantasy IX liberally used pre-rendered backgrounds that looked gorgeous on CRT TVs, but scale poorly on modern HD displays.

Honorez wasn't initially interested in playing the Final Fantasy IX remaster. While the quality of life upgrades were tempting, he could get similar graphical results from emulating the game on his old PSP. So he had no intentions of modding Final Fantasy IX until he began messing around with a machine-learning image upscaling technique called ERSGAN (Enhanced Super-Resolution Generative Adversarial Networks) to upscale the original low-res backgrounds. Honorez was so impressed with the results that began to look into how he might actually insert the backgrounds into the game.

ERSGAN is a recent technology leveraging machine learning to upscale low resolution images. It's being put to use by modders like Honorez to improve old, low resolution game assets—like the backgrounds in PlayStation-era Japanese RPGs — to make them more appealing at the higher resolutions of modern displays.

"My initial goal was to upscale the whole game," Honorez told me. He got the screens working on his own, "but managing all the layers and animations was a much bigger task" than he anticipated. Final Fantasy IX contains 70+ unique backgrounds, which are split into 11,506 static and animated layers. "All of them have to be upscaled separately," he said, likening the process to assembling dozens of puzzles from thousands of pieces.

Honorez connected with a fellow fan called Snouz who offered to work on cleaning up the background masks (that define the edges between each background layer). This process took almost a year to complete. "We never really worked together at the same time," Honorez reflected. "A bit sad, but it helps a lot for modding because you don't have to rely on each other's free time."

"The technique used to upscale the backgrounds uses two deep neural networks," Honorez explained to me. "The first one is called the generator, which is trained to do a task." In this case, the generator upscales a low resolution image. "The second one is the discriminator. Think of it as a specialist that is trained to detect counterfeits." The discriminator has been trained to know what the upscaled image should look like, and uses this knowledge to rate the work produced by the generator. "When the discriminator can't tell the difference between a real image and the upscaled one, you're done."

How are fans beating major corporations at their own game?

To train these neural networks, Honorez begins with two pieces of existing material: a high resolution image utilized for the discriminator to base its comparisons on, and downscaled version of that image for the generator. Since most of Final Fantasy IX's original high resolution backgrounds were lost after development, Honorez trained the networks with similar art from games like The Witcher 3 and Dragon Age: Inquisition.

Training is the most important step. "The general look of the high resolution image will depend on what the AI learned," Honorez said, and told me that modders often use pre-trained AIs that aren't adapted to the specific work they are doing, which will often pick the upscaled image that looks better, but not most accurate. "I think training the AI myself [by using art from similar games] was the key point to the success with Moguri Mod."

Moguri Mod is an open source project, and Honorez says there are no plans to monetize it or accept donations from other fans. "I've contacted [Square Enix] to clear the rights, but of course, they can't legally endorse a third party modification of their game." Honorez wanted to make sure Square Enix — who has a history of shutting down fan projects — was aware of the project, so they could issue a cease and desist order as early in the process as possible. That hasn't happened yet.

Fan revolution

So how are fans beating major corporations at their own game? It comes down to a combination of community expertise and passion. No corporate entity can match the breadth or speed of an organized, open source community unshackled by quarterly earning expectations, nor the love of fans who grew up playing the games.

"I think it's really rare for a company to understand what it is about their older legacy products that appeals to fans," said Frank Cifaldi, Founder and Co-Director of the Video Game History Foundation, when I spoke to him for this piece.

"Fans often know what is so beloved about the games to begin with," echoed Stry8993.

If you asked a creator like Stan Lee what made X-Men so compelling 25 years ago he would have a completely different perspective than the fans watching the cartoon or reading the comic books, says Cifaldi. "[Gaming] is in this awkward teenage phase of figuring that out as a commercial industry. And I think, for now, it's really the fans that have a better grasp of how to treat games with respect."

Many official remasters or remakes end up feeling like they're designed by committee, said Stry8993, often losing the magic of the original. "No one really appreciates being on the other end of a quick cash grab." Stry8993 brought up 2017's Final Fantasy XII: The Zodiac Age (an HD remaster of the 2006 PlayStation 2 game) as an example. "It had some great qualities," he said, "but ultimately felt overpriced [at $49.99 USD]. And even then, fans had to go in and clean up issues."

“In most cases the fan efforts will trump the commercial ones.”

"What I've often said is that when it comes to the preservation of older titles is that in most cases the fan efforts will trump the commercial ones," said Cifaldi. At their heart, remasters, whether they're official or fan-made, are preservation efforts, but Cifaldi doesn't think the industry has figured out "how to commercialize true preservation efforts." Major corporations are businesses first, and have to scale projects according to budgets and shareholder profits. This requires trading thoroughness for efficiencies.

Durante's Dark Soul

Fans aren't just saving old games, though. When Dark Souls was first released for PC a few years ago, fans were disappointed when it failed to leverage the greater horsepower of their kits. Responding to the outcry, a fan named Peter “Durante” Thoman started work on DSfix — a miniscule .dll mod players could apply to their game to adjust the rendering resolution, add anti-aliasing options, improve the depth of field effect, and many more improvements. The first version of DSfix was released just 23 minutes after the game's official launch on Steam.

"It's kind of pathetic how a single person was able to fix such glaring issues, unpaid, in spare time," said Jim Sterling for Destructoid at the time, "while the people who were employed to port it either didn't know how or lazily left the issues untouched. It's both a testament to the excellence of PC modders, and a damning indictment of the professional industry's attitude toward the market."

The line between fan and professional has been blurring for years, with many modders working in games or as programmers by day. Durante started programming as a teen in high school, and eventually worked his way up to a PhD in computer science and a gig as lead developer on Insieme Compiler. He went pro with his game optimization work when he co-founded consultancy and engineering firm, PH3 Games. Since then, he's worked on PC ports of Rez Infinite and The Legend of Heroes: Trails of Cold Steel II, among several other games.

It's dead, Jim

"The idea of the video game remaster is very new," said Cifaldi. Over the past five years or so, there's been a growing understanding that popular games will likely be remastered for more future consoles, and game makers are planning ahead. This wasn't always the case, however, and that's made it more challenging for fans and developers to remaster old games, particularly from the PlayStation era, which relied heavily on low resolution pre-rendered assets. Honorez and his team would have an easier time with their work if they could get their hands on the uncompressed backgrounds created during Final Fantasy IX development — but they were trashed by Square Enix after the game released.

The move from 2D sprite-based graphics into 3D space changed the way games were made in the 90s, requiring more and more memory to store development assets. Storing them after launch was expensive. "I think the commercial video game industry was still in the mindset that you ship a product and it's done that there's no revisiting those assets," Cifaldi said. "A lot of pioneering engineers kept their materials — inherently understanding [the value in keeping them], but the commercial industry itself did not." DOOM creator John Carmack released the source code for the legendary first-person shooter in 1997, and fans have been using it to create impressive hacks and restorations ever since.

“The idea of the video game remaster is very new.”

Data storage is not easy or free, though, especially back in the 90s when storing several gigabytes of data required tape backups, said Cifaldi. Even when measures were taken, they've often become difficult or impossible to retrieve. "We've talked to companies who did tape backup, but the employee responsible hasn't worked here in 20 years, and they don't know where the tape is stored."

A major part of the Video Game History Foundation's preservation efforts involve locating and archiving source materials that are "floating around in the ether." Cifaldi wants to make sure they're available in the future for historians and researchers to access and learn from — and for fans looking to learn from the past. Among those who've come to the Video Game History Foundation are publishers looking for assets they lost years ago. Cifaldi's helped several track down what they need for modern re-releases.

"We often liken it to early cinema," Cifaldi told me. In the earliest days of film there was no sense of a secondary market. You'd make a movie, you'd sell prints to theaters, and you'd be done with it forever. There was no home video, right? There weren't TVs yet, or broadcasting. We lost something like 80% of American cinema from the earliest days because there was literally no reason to keep it."

Cifaldi is concerned video games are following the same trajectory. Games are a home media, so final retail copies exist, often in abundance, but the missing master files make proper preservation challenging. "I think we're better now," he said. But he also admitted he's doubtful anything produced before the 2010s has surviving source repositories. So fan restorations, fan-run databases, and emerging technologies like ERSGAN are the best bet for remasters and preservation for most of those games.

Fan forward

In addition to his work at the Video Game History Foundation, Cifaldi is also the Head of Preservation at Digital Eclipse, a developer specializing in re-releases of retro games. As challenging as it is to preserve official releases, preserving fan work is a different beast, according to Cifaldi. The goal of preservation is to capture a snapshot of collection items as they originally existed, but fan efforts like Moguri Mod are interested in revitalizing old experiences and making them more appealing for new generations who never played the originals.

Cifaldi referred to RomHacking.net as a good repository for game hacks focusing on the 16-bit era and earlier, but even its database is only a partial snapshot of all the fan hacks that have been released. Even Kajar Laboratories's Chrono Trigger: Crimson Echoes — an impressive fan-sequel to the popular Super Nintendo JRPG — isn't listed on RomHacking.net due to a cease and desist order from Square Enix when it was 98% complete. Crimson Echoes eventually leaked onto the Internet, but still remains obfuscated, and requires some Google deep digging to recover a working hack file to apply to a legitimate Chrono Trigger ROM.

So what does game preservation look like in the future, and how do archivists handle fan remasters like Moguri Mod alongside official releases? It's a tough question to answer, said Cifaldi.

We're only just at the beginning of understanding what can and needs to be done to preserve these games in a playable state.

"Game preservation right now tends to be about finding the physical artifact that was sold to people in stores and making a copy. That's like 95% of video game preservation right now."

If we're talking 50 years from now, Cifaldi continued, we're only just at the beginning of understanding what can and needs to be done to preserve these games in a playable state. "We have The National Videogame Museum, and they put vintage hardware out for people to play so that they experience things in that way," but The Strong National Museum of Play in Rochester, NY has a more conservative approach, eyeing the longevity of the physical media and consoles. "They keep original hardware and software archived, and they will set it up for a researcher."

Access to original hardware and software is only part of the preservation effort, though, and does nothing to help with preservation of fan work. "It'll take a combination of historians preserving the idea that they ever existed, and things like the Internet Archive preserving the data for future gamers and researchers to access."

Awkward age

With the release of the PlayStation 5 and the Xbox Series X, and many Wii U Remasters released for the Switch, we're seeing major game companies recognizing the value of bringing older games to new platforms. Hopefully this signals the start of a trend of future proofed games and archived development assets. Nvidia's DLSS (deep learning super sampling) technique uses AI to upscale lower resolution images to higher resolution displays—similar to what Honorez accomplished with Moguri Mod, but applied on the fly by the computer's GPU.

But these emerging technologies don't spell the end for projects like Moguri Mod or similar projects for Morrowind or Silent Hill 2. Honorez and his team continue to work on improving Moguri Mod, and many other fan groups are pooling their expertise and passion to reinvigorate the old games they love. Given the relative newness of the technology, it feels like it's the only beginning of truly impressive fan restoration projects.

I'm playing through Moguri Mod now, and what I've discovered isn't just a modern splash of paint on a classic game, but an experience that looks the way I remember the game looking when I first played it 20 years ago on my tube TV. Preservation means retaining knowledge in its original form, but projects like Moguri Mod make me wonder how we can approach the preservation of experiences — how a kid playing Final Fantasy IX can experience the same joy I did long after the last PlayStation has died, CRTs are nothing more than a memory, and technology has advanced toward Star Trek's utopia.

"Fans have the passion that drives them to dig and scrape for tiny incremental improvements wherever they can, to make a better experience," Cifaldi said. "Once it's commonly understood that fan restorations are the definitive versions of the games, I think the commercial industry may start catching up."

As games leave their adolescence, fans like Marc Lauzon-Bragg, developers like Gaël Honorez, and archivists like Frank Cifaldi are eager to find ways not just to keep the past alive, but to improve the way we experience it. Fan restorations like Moguri Mod prove that the success of remasters isn't predicated on a huge budget, but rather a solid vision of what made the original so beloved in the first place. Combine that with a fanbase that's becoming progressively more skilled and experienced in game creation and a proven love for the classics, and the future of retro gaming is looking better than ever.