If you've been following the next-gen console news from the past year or so, then you probably noticed a lot of chatter about ray tracing. The graphics technique isn't new, but providing hardware ray tracing is a big next-gen feature for Nvidia's GeForce RTX line of cards. Several major titles, including Cyberpunk 2077, Call of Duty: Modern Warfare, Watch Dogs: Legion, Control, Wolfenstein: Youngblood, and Dying Light 2 are all supporting ray tracing in some fashion.
Nvidia has been pushed ray tracing hard over the course of Gamescom, with one highlight being an impressive demo of Minecraft. Light shines through the treetops and holes in cave walls; gold and diamond shine in the face of your torches, and water is actually transparent and reflective. It's old, steadfast, blocky Minecraft, but the addition of real-time ray tracing means it almost feels realistic. The unsaid statement surrounding that demo: "If ray tracing can make an eight-year-old game like Minecraft look great, imagine what it can do for future games!"
With Nvidia behind it, ray tracing is the "next big thing" in graphics technology, like tessellation, ambient occlusion, bump mapping, and anti-aliasing before it. It's the buzzword to let you know that the next-generation is more snazzy and shiny than the one before. Microsoft and Sony have already revealed that the Xbox Scarlett and PS5 will support ray tracing. Even AMD has weighed in on ray tracing, mentioning hardware ray tracing as a feature for its next-generation graphics architecture. Given that, it's worthwhile to dive deep into ray tracing, and the potential benefits players can actually see in the future.
What is Ray Tracing?
Ray tracing is a graphics rendering technique that's meant to simulate how light actually works in the real-world. Rays of light begin at a light source and then travel through space, becoming absorbed, diffused, refracted, or reflected by different objects along its path. By calculating distinct rays of light from a light source and how they react with various objects in a digital scene, we can more closely mirror the real-world with better lighting and more accurate shadows.
Technically, computer ray tracing is working backwards, tracing a path from the camera back towards a light source. This is for efficiency's sake: instead of tracing rays from every light source, it's better to only simulate the rays that the player is actually seeing. Ray tracing itself is already computationally intensive; this method lowers the strain on your system.
Proper ray tracing requires developers to be specific about the surfaces of various digital objects. Light reacts differently when it comes into contact with wood, metal, or dirt. Certain cloth absorbs more light that leather, which has a reflective sheen to it. Water is translucent and reflective; you should not only be able to see through it, but also reflections depending on your point of view. All of this needs to be defined properly for ray tracing to work.
Visual effects houses for film and television have been using this technique for a long time, but they require either lots of time or lots of power. Properly rendering a scene with ray tracing needs hours, days, or weeks, or the combined power of huge server farms. Ray tracing is just complex math, but it's a lot of complex math. In place of ray tracing, games have generally used another technique called rasterization, which takes 3D models and converts them into pixels for display. Each pixel has a certain color, based on the model itself, and developers use programmable shaders to determine the final color of the pixel. Rasterization is closer to guesswork than simulation.
What's new is we're finally able to do real-time ray tracing in hardware. Nvidia's RTX cards are still primarily built on rasterization, but they also have GPU cores set aside specifically for ray tracing calculations. In addition, Nvidia has crafted its own ray tracing libraries to take the busy work off of developers. It's not entirely doing what film and television visual effects artists are doing, but it's one step closer.
"For game creators, ray tracing helps to automate and simplify the creation of realistic visuals without having to deal with multiple and often highly approximate rendering techniques," says CryEngine principal rendering engineer Vladimir Kajalin. "With ray tracing, both the complex and more typical day-to-day examples work out of the box, without the need for developers to combine multiple approximate algorithms or find a workaround for visual errors."
Why is This Important?
As I noted before, the result of using real-time ray tracing is more realistic lighting and object surfaces, without developers having to handle certain visual bits through workarounds. In some of the biggest AAA titles, real-time ray tracing isn't going to wildly change your graphics, but it is going to clean up some of the details.
How developers use ray tracing is ultimately up to them. Some games, like Metro: Exodus, use ray tracing for its global illumination, offering more accurate lighting. Shadow of the Tomb Raider uses the technology for more precise shadows. Battlefield 5 leans heavily on it for reflections. Newer titles like Control utilize ray tracing for multiple bells and whistles, including shadows, illumination, and reflection. For developers, it means being able to create more realistic scenes at the cost of fewer resources.
"With ray tracing, we are free to use very reflective surfaces without thinking about the drawbacks of the previous methods," Hunt: Showdown principal 3D artist Ron Frolich said in an interview with Crytek. "We can place large mirror-like surfaces into the level without worrying about whether the scene will change to the extent that will make them look out of place. We can also place animated objects across from mirrors knowing that the animations will be reflected accurately in these mirrors. Essentially, we can worry less about reflections not matching the current state of the world, which allows us to make the world feel more vivid and dynamic."
"RTX is not an overhaul of the way you develop games in general. It's something that fits in alongside it. That's why it's so great – it goes in nice and easy, takes your existing data and just represents it in another way," Metro Exodus rendering programmer Ben Archard told RPS last year.
Ray tracing doesn't have to only be based around lighting, either. Sound propagates through a scene in much the same matter, and similar calculations can be used to create more realistic lighting. Mark Cerny, the architect behind the PlayStation 4 and its upcoming successor, touted ray tracing for this purpose.
"If you wanted to run tests to see if the player can hear certain audio sources or if the enemies can hear the players' footsteps, ray tracing is useful for that. It's all the same thing as taking a ray through the environment," Cerny says.
Looking at something like Minecraft, you can see the dividend something like ray tracing can provide to even older titles. Imagine Uncharted or Resident Evil with proper real-time ray tracing. The next generation look of the previous generations is going to be stunning.
Who's Pushing It Forward?
Nvidia has been at the vanguard of this push into real-time ray tracing. (Alongside another feature called Variable rate shading.) That's primarily because it allows them to have further differentiation over its competition, AMD. But as Nvidia goes, so does the rest of the industry seemingly.
Epic Games revealed that real-time ray tracing was coming to Unreal Engine 4 at GDC 2019. The feature is only available in Direct X12, but it may expand further in the future. Unity Technologies has released an experimental build of its engine focused around real-time ray tracing. CryEngine showed off a Neon Noir demo with real-time ray tracing, though the tech hasn't come to the engine yet.
Basically, real-time ray tracing is what's next, even if you won't have to do anything about it. Like tessellation and bump mapping, it will eventually become a part of the landscape that you don't notice while you're exploring new worlds and putting holes in digital monsters. But the future is here, especially if Gamescom 2019 is any indication. And the digital world is looking a little brighter with all those rays.
If you're trying to keep up to date on the next-generation of consoles, you can read our look at the PS5's announced specs, including the secret sauce that Sony has yet to reveal. There's also our thoughts on the Xbox Scarlett, which is focusing on improved frame rates for future games. and if you've got the cash and want to get in on Nvidia's real-time ray traced future, check out the Super line of RTX cards.