Nvidia has been a market leader in PC gaming graphics for years, and with the rise of machine learning, AI is playing an increasingly important role in how we experience our games. One of its most revolutionary advancements has been Deep Learning Super Sampling (DLSS), an intelligent upscaling solution that opens the door to increased performance, especially at higher resolutions. This technology has undergone multiple iterations over the years, and with the launch of the RTX 50-series, it’s making its biggest jump forward yet with DLSS 4.
In this guide, I’ll explain everything you need to know about how this tech works, why it matters to you and the future of PC gaming, and why it’s something you may want to keep in mind with your next PC upgrade.
What is DLSS?
DLSS, or Deep Learning Super Sampling, is Nvidia’s proprietary system for intelligently upscaling games. The company has consistently developed and built upon DLSS since its debut in 2019. Throughout that time, its core purpose has been to improve performance by rendering games at a lower resolution and then upscaling that content to your monitor’s native resolution. Rather than leave you with the soft and potentially blurry image you would experience turning up the resolution yourself, DLSS applies its scaling through a neural network that has been trained on thousands of hours of video games. Alternatively, if you don’t want or need to upscale, you can instead enable Deep Learning Anti-Aliasing (DLAA), which provides image enhancements to your native resolution.
These features are only available on Nvidia graphics cards with Tensor Cores, which began with the RTX 20-series. This is because the upscaling and enhancements taking place are the result of thousands of hours of neural training on Nvidia supercomputers. Its neural network ingests and learns from huge data sets to learn how to upscale and reconstruct/enhance images with the least quality loss and, in fact, even provide additional clarity in some situations.
As time has gone on, Nvidia has enhanced the system with additional features. One of the biggest is Frame Generation, which uses artificial intelligence to create an additional frame between each rendered frame, increasing frame rate. When used in conjunction with Nvidia Reflex (which is also being enhanced with the 50-series), these additional frames can blend in with a minimal impact on latency. The result of which isn’t just “better performance,” but allowing lower-performance graphics cards to reach previously unattainable frame rates with aspirational graphics settings.
Which brings us to the current day with DLSS 4.
TNN vs. CNN (The Transformer Model)
This generation marks the most significant change in the feature’s history and includes a completely different, and much more capable, AI model.
The Transformer Model
So far, DLSS has used a model of AI known as a CNN, or Convolutional Neural Network, to deliver its benefits. This type of model analyzes an image to determine key elements, like lines, edges, and spatial relationships to determine how to apply its enhancements. It’s a very common type of neural network specializing in image analysis, which is why it’s not surprising that it was the foundational model behind DLSS up to this point.
DLSS 4 instead utilizes a Transformer model. A Transformer is a different form of AI model that’s able to calculate twice the number of parameters to better understand each frame of the scene. Put another way, it’s able to understand what it’s looking at and what’s taking place much better, and then apply more sophisticated calculations to deliver a higher quality image.
This new model is core to DLSS 4 and impacts each of the different pieces that allow DLSS to look and perform so much better than the prior version.
Multiple Systems in One
With DLSS 4, Deep Learning Super Sampling is much more than a simple upscaler (and never was, really). Instead, it’s a series of systems that work together to improve performance, enhance image quality, and reduce latency. In addition, frame generation has been significantly upgraded and can produce three times as many frames as DLSS 3.5.
It’s this network of cooperative systems that allow DLSS to look and perform better than it has in the past. One system handles upscaling (DLSS Super Resolution), while another handles lighting and shadows (DLSS Ray Reconstruction). DLSS Frame Generation multiplies the frames, while Reflex 2.0, which is it’s own feature separate from DLSS, keeps latency numbers low so you’re not noticing input lag as you play your games.
DLSS Super Resolution is the sub-system that handles upscaling. If you’ve ever noticed DLSS in an in-game menu, there’s a good chance that there were quality options to choose from: Ultra Performance, Performance, Balanced, and Quality are the typical presets. Each of these levels adjusts the game’s rendering resolution, the base resolution that the neural model will need to upscale.
Through the new TNN model, DLSS Super Resolution is able to deliver much sharper results and retain much more detail that the previous CNN model would lose, including in motion. The results can look nearly as, equal to, or even slightly more crisp than native resolution. While any enhancements beyond native resolution are subject to discussion and scrutiny, when it works, it can look fantastic. The impact of the TNN is especially noticeable in fine detail, like textures, fine edges, and lettering.
DLSS Ray Reconstruction is the second core image enhancer and one that has seen big improvements. Replacing traditional denoisers (systems that remove graininess and “noise” from a scene), this portion of DLSS is focused explicitly on analyzing and reconstructing lighting and shadow information. Like DLSS more broadly, it has been trained on thousands of hours of data to improve its understanding of different lighting conditions and how they should appear in real-time renders.
The TNN model provides DLSS Ray Reconstruction with much better understanding than it was previously able to have, and the results are easy to spot. DLSS Ray Reconstruction in the CNN model often struggled with fine lines and moving shadows. In the image above, the results speak for themselves, but it’s also evident elsewhere.
The flickering that was previously evident in distant shadows and lines (like telephone wires) is dramatically reduced. Objects in motion, like ceiling fans, retain more clarity. The strange action of “bubbling shadows” isn’t nearly as evident. While scenes in games are variable and more testing and iteration is necessary to come to any hard conclusions, it’s difficult to argue that it’s not a leap ahead for DLSS as a system.
Deep Learning Anti-Aliasing, or DLAA, is an alternative to DLSS Super Resolution. If you don’t need the upscaling features, DLAA allows you to significantly enhance your native resolution through TNN-enhanced anti-aliasing. DLAA smooths out edges much better than traditional anti-aliasing models and maintains those improvements in motion. The result is a very sharp image that looks noticeably clearer than gaming at native resolution with standard anti-aliasing.
For gamers with older GPUs, you’ll be able to upgrade to the new Transformer model, as well as toggle DLAA or DLSS Ultra Performance mode, within the Nvidia App.
Frame Generation and Multi Frame Generation
DLSS Frame Generation made its debut with the RTX 40-series. It was contentious at the time, though has largely been accepted as an effective way to improve your in-game performance. This system, which I’ll refer to as single frame generation (SFG), allowed the GPU to leverage its Tensor Cores to create an artificial frame based on the details of the previous frame. Using this technology, gamers could play at higher resolutions and frame rates than may otherwise have been possible, make better use of their high refresh rate gaming monitors, and enjoy smoother action overall.
With DLSS 4 and the new Transformer model, the system is now able to generate up to three artificial frames for every true frame that’s rendered. This new capability is DLSS Multi Frame Generation (MFG). DLSS is able to accomplish this thanks to the improved performance capabilities of the new TNN, as well as shifting optical flow into a neural network instead of relying on the hardware-based Optical Flow Accelerator on the RTX 40-series. Because this AI Optical Flow system is unique to the RTX 50-series, DLSS Multi Frame Generation is exclusive to this generation for now.
Optical Flow, in general terms, is the AI’s ability to interpret the composition and motion within a scene to determine what should be rendered into its neural frame. Because the transformer is able to analyze each scene more thoroughly, ingesting more data points, it’s able to more accurately anticipate what will occur further into the future.
While it would be easy for things to get messy with the TNN rendering 75% of the frames when set to its max, the RTX 50-series also introduces flip metering. The important thing to know here is that flip metering controls frame pacing to ensure that gameplay remains smooth. However, Nvidia does recommend that MFG only be set as high as it takes to reach your monitor’s refresh rate. Overshooting to get the highest FPS possible can introduce visual artifacts due to the mismatch.
Wrapping Up
DLSS 4 is only one piece of the AI-enhanced PC gaming future Nvidia has promised us, but it’s an exciting one. While it’s clearly designed to have something for everyone, its benefits are likely to be especially keen for gamers playing on mid- to low-performance GPUs, extending the usable life of the hardware and opening the door to higher resolutions and graphics settings than would otherwise be possible. Time will tell how DLSS 4 ultimately shapes up, and if prior generations are any indication, Nvidia can be expected to build upon this foundation throughout the generation. As a starting point, however, DLSS is poised to impress.
