Imagine a future where our games learn us, instead of us learning them.
Jason Evangelho , Contributor News and reviews from the crossroads of PC gaming & consumer hardware Opinions expressed by Forbes Contributors are their own.
Wouldn't it rock if your Skyrim followers could adapt to your changing behaviors and needs?
Microsoft announced Windows Machine Learning, an advanced AI platform for Windows 10, earlier this month. WinML can improve security, increase productivity and even aid cameras in picking out faulty chips before hardware is boxed up and shipped out. But Microsoft has lofty ambitions in the gaming department, too. In fact, they want to see dynamic neural networks radically and dynamically influencing the next generation of video games.
A very simplified explanation of Dynamic Neural Networks (DNNs) is this: a set of algorithms that are modeled after the way a human brain behaves, trained to recognize a wide range of patterns and behaviors and then adapt.
You see this in your everyday life without realizing it. Machine Learning, a subset of artificial intelligence, is responsible for detecting what emails are spam, and is behind speech-to-text on your PC and phone (whether you love it or hate it, at least it's learning over time). Machine learning can even pick out certain human gestures or identify unusual behavior. It's a pretty diverse and interesting category, but how can this improve games?
In a blog post being released this morning, Microsoft acknowledges that all gamers come to the table with different expectations: "...some want to spend time with friends or to be the top competitor, and others are just looking to relax and enjoy a delightful story. Regardless of the reason, machine learning can provide customizability to help gamers have an experience more tailored to their desires than ever before."
What Microsoft is talking about is basically shifting the very framework of the game itself to revolve around you, instead of you being locked into the game's rigid AI ruleset.
AI, Machine learning and neural networks
I have to assume this would interfere with the creative vision of certain developers, but in many genres of games, it could be a thrilling improvement.
Microsoft will speak in depth about using artificial intelligence and Windows Machine Learning at this week's GDC.
Dynamic Neural Networks: The ultimate customization tool?
I loved Ubisoft's Rocksmith because it adapted to my talent (or lack thereof) dozens of times throughout each song. If I sucked it would back off and decrease the intensity of notes it wanted me to play. If I started nailing riffs, it almost immediately added more notes and challenges. With Windows Machine Learning, dynamic neural networks can be created to do much, much more than that.
How many times have you yelled at your boneheaded AI companions in Ghost Recon Wildlands, or been annoyed at the relative uselessness of followers in Skyrim? What if they could start learning your habits and change their combat style based on yours? What if they noticed your fondness for a bow, and started hitting nearby shops for the latest-and-greatest crossbows or looting dead enemies for explosive arrows? What if you changed your mind mid-game and they adapted to that? Then it's not just an NPC, it's a character you feel closer to.
Microsoft uses the example of a player loving the loot chase, someone who enjoys finding treasure but doesn't particularly love combat. A dynamic neural network could "prioritize and amplify those activities while reducing the amount of difficulty of battles."
It should even be possible for these dynamic neural networks to learn a player's tastes and preferences across a wide library of games and apply tweaks based on your behavior in the other games. Now that would be amazing. Perhaps not something everyone wants, but I love the potential.
How Neural Networks Can Help Developers
In their blog, Microsoft argues that neural networks could be used to generate the terrain of a vast open world, and do it with the same quality level as an artist creating it by hand. This would allow the game's visual artists to focus on making that world more beautiful and interactive.
They also cite a real-world example of this technology in a game you may have already played: Quantum Break. Developer Remedy used machine learning to generate facial animation using a variety of audio and facial inputs. They then developed a model that moved its face based solely on new audio inputs. In a nutshell, this kind of tooling alone created facial movement that was 80% complete out of the gate.
While I'm not well versed in facial animation techniques, it matters to Microsoft because they believe studios can save time and money with this technology, passing that down to gamers in the form of earlier release dates or simply better looking experiences.
Better Visuals, One Pixel At A Time
This image is an example of machine learning dynamically improving an image when it's zoomed in.
I like eye candy. I like super crisp, high resolution scenery where I can just melt away and enjoy the world. What about the sad reality of an in-game object that looks lifelike from afar, but then turns into a blurry mess when you're nose-to-nose with it? That's commonly known as the "jaggies" caused by aliasing. Well, dynamic neural networks can also greatly enhance a game's visuals. That's because models can be trained to pick the best color for every single pixel on the screen. The benefit here would apply to smaller images that are upscaled.
The Neural Network Flow
My understanding is that developers can create these trained models that identify and change certain behaviors and traits, and convert them to the Open Neural Exchange Network format. That's something co-developed by Microsoft, Facebook and Amazon to provide broader usage of neural networks. Then they stick it into their Windows 10 game and that's it. It sits on the gamers machine and learns. Adapts. Improves.
Becomes Skynet? Just kidding.
Seriously though, this is cool, geeky stuff that I can barely wrap my brain around. But one thing is certain: this feels like a glimpse into truly personalized games. A future where our games learn us, instead of us learning them.
Right now it seems like a blank canvas of possibilities, and I can't wait to see how developers paint it. Just don't expect these dreams to become reality for several years.
gamestar games gamestorrents games workshop gameshop gamestop gamesdeal games with gold gamespot games online