Why NVIDIA RTX Tensor Throughput Is Locked—and How Unlocks Work

Why NVIDIA RTX Tensor Throughput Is Locked—and How Unlocks Work

Two open-source Linux projects have demonstrated that selected Tensor Core operations on certain NVIDIA GeForce RTX graphics cards can achieve substantially higher throughput by modifying GPU configuration during initialization. In a reported RTX 4090 experiment, FP16 and BF16 operations with FP32 accumulation nearly doubled their measured instruction throughput, rising from approximately 165 TFLOPS to 331 TFLOPS.

The results expose an important distinction between a GPU’s physical computing resources and the performance made available through its default operating configuration. The underlying hardware does not change, yet particular matrix-operation paths can execute at a higher rate after the relevant speed-selection settings are modified.

The work spans NVIDIA’s Ada Lovelace and Ampere architectures, with the broader open-source research ecosystem also exploring older Turing GPUs. Reported results include substantial gains in selected matrix multiplication workloads, prompt prefill, and BF16 image generation.

However, these improvements do not apply universally. Gaming performance is not expected to improve, and memory-bound large language model decoding may see little benefit. The nearly 2× figures primarily describe selected Tensor Core instruction throughput, not a doubling of end-to-end application performance.

Understanding why these limits exist requires examining precision modes, streaming multiprocessor configuration, firmware execution, and the difference between theoretical compute throughput and actual application performance.

🔍 1. What Is NVIDIA’s Tensor Core Throughput Limit?

#

FP16 accumulation versus FP32 accumulation

#

Tensor Cores accelerate matrix operations commonly used in deep learning, scientific computing, and other highly parallel workloads.

Their effective performance depends on several factors, including the input format, accumulation precision, supported instructions, and the GPU’s operating configuration.

Two FP16 operations can therefore exhibit very different throughput depending on how their results are accumulated.

• FP16 input with FP16 accumulation: Uses reduced-precision inputs and accumulation.
• FP16 input with FP32 accumulation: Uses reduced-precision inputs but accumulates intermediate results in FP32.
• BF16 input with FP32 accumulation: Uses the Brain Floating Point 16-bit input format with FP32 accumulation.
• TF32 operations: Provide a Tensor Core computation mode commonly used for matrix workloads that require a wider exponent range than FP16.

FP32 accumulation is particularly relevant to workloads that benefit from reduced-precision matrix inputs while retaining higher-precision accumulation behavior.

For selected GeForce GPUs, NVIDIA’s published specifications show that certain Tensor Core operations with FP32 accumulation have lower throughput than corresponding operations with FP16 accumulation.

The RTX 4090 illustrates the difference. Its nominal FP16 Tensor Core throughput with FP32 accumulation is approximately half the rate associated with FP16 accumulation, consistent with the measurements reported by the gpu-unlock-ada project.

The limitation does not mean that the GPU lacks Tensor Cores capable of executing matrix operations. Instead, the effective execution rate depends on the operation mode and the configuration under which the hardware runs.

Why the restriction matters

#

Matrix-compute throughput is a major performance factor in several AI workloads.

A GPU that can execute more FP16 or BF16 operations with FP32 accumulation per unit of time may accelerate compute-intensive tasks such as diffusion-model inference, model fine-tuning, selected training workloads, and prompt prefill.

However, the benefit depends on whether the application actually uses the affected instructions.

A workload based on INT8 operations, a model whose kernels already run at the available maximum rate, or a task constrained primarily by memory bandwidth may see little or no improvement.

The distinction between the theoretical capability of a compute unit and the execution rate exposed by a particular configuration is therefore central to understanding the unlock.

⚙️ 2. How the Open-Source GPU Unlock Works

#

The two principal projects use a similar high-level strategy, but target different GPU architectures.

• gpu-unlock-ada targets the Ada Lovelace architecture, with demonstrated results on the RTX 4090 and experimental validation on selected additional Ada cards.
• gpu-unlock targets selected NVIDIA Ampere GPUs, including cards built around GA102 and other supported GA10x chips.

Rather than modifying the graphics card’s physical hardware or patching the official NVIDIA driver, the projects change relevant GPU speed-selection settings before the standard driver takes control.

SM speed-selection configuration

#

NVIDIA GPUs contain configuration mechanisms that control the effective execution rates of particular instruction classes.

The projects focus on speed-select settings associated with the streaming multiprocessors (SMs), the GPU’s primary compute units.

The relevant configuration includes register controls commonly identified as SS0 and SS1. The unlock implementations modify these selectors to enable higher issue rates for selected Tensor Core operations.

This changes the effective instruction throughput without adding compute units, increasing the number of Tensor Cores, or raising the configured power limit as part of the benchmark comparison.

The observed behavior also helps explain why some operation modes improve while others remain near their original performance. The modification targets particular execution paths rather than applying a universal multiplier to the GPU.

Why the modification happens before the NVIDIA driver loads

#

Both projects use an early initialization mechanism because the relevant configuration must be established before normal GPU driver initialization.

The general sequence is:

1. Prepare the required firmware inputs and platform-specific unlock component.
2. Execute the configuration procedure before the NVIDIA driver binds to the GPU.
3. Modify the relevant speed-selection settings.
4. Perform a PCIe Function Level Reset (FLR).
5. Allow the original NVIDIA driver to initialize the graphics card.
6. Verify the resulting configuration and benchmark the affected operations.

The exact implementation differs between projects and GPU generations.

The Ada implementation uses a GPU firmware execution path to perform the required register transactions. The Ampere project builds on earlier GPU-unlocking research and incorporates a Linux kernel module that performs its work before the normal NVIDIA driver binds.

The important architectural feature is the timing: the modification occurs during initialization, not as an ordinary CUDA kernel or application-level setting.

No modified video BIOS or patched NVIDIA driver

#

A notable aspect of the projects is that the reported method does not require flashing a modified VBIOS.

The Ada project also generates the relevant firmware locally from NVIDIA-provided input files, using SHA-256 verification, rather than distributing NVIDIA firmware binaries directly in its repository.

After the configuration transaction and reset, the unmodified NVIDIA driver can take control of the card.

This does not mean that NVIDIA officially supports the altered configuration. The resulting state differs from the normal supported operating configuration, and successful driver initialization does not establish universal compatibility or long-term reliability.

📊 3. RTX 4090: Nearly Double the Measured Instruction Throughput

#

The most widely discussed results come from the RTX 4090 and the open-source gpu-unlock-ada project.

The GPU uses NVIDIA’s AD102 processor and has 128 streaming multiprocessors in its standard configuration.

Original benchmark results

#

The project reported the following results for targeted Tensor Core operations:

The developer subsequently extended the benchmark loop from 20,000 to 40,000 iterations.

The longer test recorded 330.8 TFLOPS for FP16 input with FP32 accumulation and 330.3 TFLOPS for BF16 input with FP32 accumulation.

These results remained consistent with the initial measurements.

The experiment retained a 450 W power limit and did not rely on manually increasing the power target or applying conventional overclocking settings to produce the reported difference.

Independent validation and its limitations

#

An independent RTX 4090 owner subsequently reproduced the configuration change and compared the same card before and after unlocking.

The reported results were:

The retest used Linux kernel 7.2.11-arch1-1, NVIDIA driver 615.78.08, and a 450 W power limit.

This provides additional evidence that the effect is reproducible on the tested configuration. However, the available validation remains limited to particular cards, software versions, and benchmark implementations. It should not be interpreted as comprehensive validation across every RTX 4090 board or firmware revision.

The measurements also distinguish the original instruction-throughput experiment from later matrix multiplication tests. The two benchmark categories measure different aspects of execution and must be evaluated separately.

Other operation modes remain largely unchanged

#

The unlock does not double every Tensor Core or GPU operation.

The results are consistent with a targeted change to selected instruction execution rates rather than a general increase in the GPU’s computational resources.

For more detailed measurements and the project’s technical evidence, see the RTX 4090 speed-select experiment report.

📈 4. Ampere GPUs: RTX 3090 Ti, RTX 3080, RTX 3070, and RTX 3060

#

The Ada project builds on earlier research involving the Ampere architecture.

The gpu-unlock project uses a Linux kernel module to adjust selected instruction-rate settings on supported GPUs before the standard NVIDIA driver initializes.

Its reported results include substantial improvements for BF16 and TF32 matrix multiplication.

RTX 3090 Ti benchmark results

#

Under a 450 W power limit, the RTX 3090 Ti produced the following results in dense matrix multiplication benchmarks:

These figures are especially useful because they describe matrix multiplication performance rather than only the issue rate of an isolated instruction sequence.

They show that a change to the execution-rate configuration can improve selected real compute kernels, although the size of the gain varies substantially with the operation mode.

Results across other RTX 30-series GPUs

#

Community testing has reported improvements on additional Ampere cards, including the RTX 3080 10GB, RTX 3070, and RTX 3060.

For example, the RTX 3080 10GB reportedly increased BF16 matrix-operation throughput from approximately 65.4 TFLOPS to 113.4 TFLOPS, an improvement of around 73%.

The reported gains for the RTX 3070 and RTX 3060 were approximately 81% and 76%, respectively, although exact absolute results depend on the specific test configuration.

Compatibility is not universal, even within an architecture. Some cards may complete the configuration procedure without achieving the expected increase in a particular operation mode. GPU identifiers alone do not guarantee that a given board, firmware, or memory configuration will behave identically to a previously tested model.

The available results therefore support a narrower conclusion: selected Ampere cards can benefit significantly from the technique, but every model and workload requires independent validation.

🧪 5. Real-World AI Performance: Where the Unlock Helps

#

The difference between instruction throughput and application performance is critical when evaluating the usefulness of GPU unlocking.

A compute microbenchmark can achieve nearly 2× throughput while a complete application sees a much smaller improvement because it spends time on data movement, synchronization, kernel scheduling, or other operations unaffected by the configuration change.

RTX 4090 matrix multiplication gains are smaller

#

The RTX 4090 tests demonstrate this difference.

Although the measured FP16 and BF16 instruction throughput nearly doubled, cuBLAS matrix multiplication tests showed more modest improvements.

For FP16 matrix multiplication, the reported gain ranged from approximately 12% to 34%, depending on matrix size. A 4096 × 4096 test improved from 153.2 TFLOPS to 206.0 TFLOPS, representing an increase of approximately 34.4%.

BF16 results varied more. Some matrix sizes improved by approximately 16% to 20%, while another tested size showed little change.

This behavior is expected when a benchmark includes more than the targeted instruction sequence. The fraction of total execution time affected by the unlock depends on the kernel, matrix dimensions, data movement, scheduling, and utilization.

Consequently, the instruction-level result should be treated as an upper-level indicator of the available compute-path improvement, not as an application speedup guarantee.

Prompt prefill can benefit more than token generation

#

Large language model inference offers another example of workload-dependent performance.

Inference commonly consists of two distinct phases:

• Prompt prefill: Processes the input sequence and builds the initial model state through substantial parallel computation.
• Autoregressive decoding: Generates output tokens sequentially, repeatedly processing model weights and intermediate state.

Prompt prefill often uses large matrix operations that can benefit from increased Tensor Core throughput. Community reports involving unlocked RTX 3090 and RTX 3080 configurations have described prompt-prefill improvements in the range of approximately 14% to 36%, depending on the model and configuration.

By contrast, per-token generation speed has often remained largely unchanged in the reported tests.

Decoding can be constrained by memory bandwidth rather than arithmetic throughput. If the GPU spends most of its time moving model weights and cache data, increasing the issue rate of selected Tensor Core instructions may not materially improve generation speed.

For local AI deployments, it is therefore important to benchmark prefill latency, time to first token, sustained token-generation speed, and overall throughput separately.

BF16 diffusion and image generation

#

Diffusion workloads can be more favorable when the model relies heavily on BF16 matrix operations affected by the modification.

One reported RTX 3090 Ti ComfyUI experiment reduced the duration of a BF16 image-generation task from approximately 72 seconds to 45 seconds.

The reported times correspond to an improvement in throughput of roughly 60%, with the task completing about 37% sooner.

This is a meaningful application-level result, but it remains specific to the tested model, workflow, and hardware configuration. It should not be generalized to every diffusion pipeline or image-generation model.

Fine-tuning and selected training workloads may also benefit if their computational kernels rely heavily on affected matrix-operation modes.

The central requirement is that the workload must use the instructions whose effective throughput has changed.

🎮 6. Why Gaming Performance Does Not Improve

#

The unlock is primarily relevant to selected AI and matrix-compute operations, not conventional graphics rendering.

Most games do not rely heavily on the specific FP16/BF16 Tensor Core execution paths with FP32 accumulation targeted by these projects.

As a result, the modification is not expected to produce a meaningful increase in gaming frame rates.

The same distinction applies to many graphics workloads that are limited by rasterization, shader execution, memory bandwidth, geometry processing, or other portions of the rendering pipeline.

It would therefore be misleading to treat the unlock as a general-purpose GPU overclock or a universal performance upgrade.

The difference between compute unlocking and conventional overclocking is important here. A typical overclock adjusts frequencies, voltages, or power targets. These projects instead change a selected execution-rate configuration while keeping the physical GPU and configured power limit unchanged during the comparison.

For a different example of GPU performance modification, PUDN’s report on a shunt-modded RTX 4090 Laptop GPU describes a hardware-level power-sensing modification. That approach is fundamentally different: it changes the effective power constraints and introduces additional electrical and thermal risks rather than targeting Tensor Core instruction-rate settings.

⚠️ 7. Deployment Requirements, Compatibility, and Risks

#

The published projects remain experimental low-level modifications. A successful benchmark is not equivalent to a supported production configuration.

Linux initialization requirements

#

The Linux kernel modules must execute before the NVIDIA driver binds to the GPU.

This requires correct integration with the system’s boot and initramfs configuration. If another component initializes the GPU first, the unlock may fail or be unable to establish the intended configuration.

Some initial deployments have also required a complete power-off cold boot rather than an ordinary restart. Behavior varies with the implementation, hardware, and software environment.

Kernel upgrades can affect module builds and boot configuration, so the initialization order must be rechecked whenever the operating system or kernel changes.

GPU-specific compatibility

#

The Ada implementation has demonstrated the technique on an RTX 4090 and has documented experimental validation for selected additional Ada models, including the RTX 4070 Ti SUPER and RTX 4060 Ti 16GB.

These additional targets are not universal endorsements of all cards using the same GPU architecture. The experimental targets have separate compatibility requirements, and other Ada models—including workstation and data-center products—must not be assumed to work.

Ampere compatibility also varies across board configurations and GPU identifiers.

In all cases, benchmark results from one card should not be treated as proof that another card will behave identically.

Stability and firmware risks

#

The projects interact with privileged GPU firmware execution and protected configuration registers.

An incorrect configuration or unexpected initialization behavior can cause driver failures, system instability, or loss of GPU availability.

The fact that the stock NVIDIA driver loads after the modification does not mean the operating state is officially supported or guaranteed to remain compatible with future releases.

For systems used in production, preserve an unmodified boot entry and a reliable recovery path. Avoid making an experimental configuration the only route to a working system.

Numerical correctness and reproducibility

#

The throughput figures are encouraging, but numerical correctness must be validated separately from performance.

A limited benchmark can confirm that a particular matrix operation produces expected results under tested conditions. It cannot establish that all kernels, input values, matrix dimensions, and software environments remain correct.

For training or scientific workloads, verification should include the relevant output comparisons, convergence behavior, model-quality checks, and stability tests required by the application.

The current evidence is best treated as an open-source hardware research result, not as a universal production optimization.

📋 8. Summary of the Reported Performance Gains

#

The table below summarizes representative results from the reported projects. The entries cover different benchmark types and should not be compared as though they measured the same thing.

These results show that the largest gains occur in the operation modes directly affected by the configuration change. End-to-end performance depends on how much of the application workload uses those operations and whether other bottlenecks dominate execution.

For a broader overview of the related architecture-specific projects and reported benchmarks, see KAD’s analysis of NVIDIA GPU Tensor Core unlocking.

🌌 Conclusion: A Firmware Configuration Difference, Not New Hardware

#

The open-source NVIDIA GPU unlock projects demonstrate that selected Tensor Core operations can deliver significantly higher measured throughput when the GPU’s effective instruction-rate configuration is changed.

On the RTX 4090, FP16 and BF16 operations with FP32 accumulation approached twice their original instruction throughput. Ampere GPUs such as the RTX 3090 Ti have also shown substantial gains in selected matrix multiplication workloads.

The mechanism does not add compute units or increase the GPU’s physical capabilities. Instead, it changes the speed-selection configuration applied to particular execution paths, allowing existing hardware to deliver higher throughput for supported operations.

The practical benefits remain workload-dependent. Compute-intensive matrix operations, selected prompt-prefill workloads, and BF16 diffusion tasks may improve substantially, while gaming and memory-bound token generation are unlikely to benefit meaningfully.

The technical significance is the distinction between advertised operating characteristics and the performance achievable under a different configuration. The engineering challenge is determining whether that additional throughput can be accessed reliably, safely, and consistently across real workloads.

For now, these projects are best understood as experimental demonstrations of GPU configuration behavior and potential compute improvements. They offer valuable insights into NVIDIA’s Tensor Core execution model, but broader hardware validation, workload-specific benchmarking, and long-term compatibility testing remain essential before treating the technique as a dependable optimization.