NVIDIA AI PCs Can Now Run 30B MoE Models Locally
NVIDIA has quietly expanded its local AI platform, pushing consumer and workstation hardware beyond conventional model inference toward persistent, locally hosted AI agents. The latest update covers RTX PCs, DGX Spark, DGX Station, Jetson, RTX PRO workstations, and other Blackwell-powered systems, with improvements spanning large language models, agent orchestration, speculative decoding, video generation, and multi-node deployment.
The centerpiece is the ability to run open-weight Mixture-of-Experts (MoE) models with around 30 billion parameters locally. Combined with NVFP4 and GGUF support, optimized inference runtimes, and new clustering tools for DGX Spark, NVIDIA is positioning its AI PC ecosystem as a platform for increasingly capable private and persistent AI workloads.
🧠 30B MoE Models Move to Local AI PCs #
The most significant addition is NVIDIA’s open-weight Nemotron 3.5 Lightning, a 30-billion-parameter Mixture-of-Experts model designed for specialized local agents operating within multi-agent systems.
NVIDIA claims Nemotron 3.5 Lightning can achieve token-generation speeds up to four times faster than comparable open-source models while reducing overall agent task completion time by approximately 30%. Because the model uses open weights, developers can fine-tune it for domain-specific knowledge, coding conventions, writing styles, and other specialized workflows.
Potential local-agent applications include:
- Email and calendar assistants
- Smart-home agents
- Pair-programming systems
- Domain-specific research assistants
- Private document-processing agents
- Specialized creative and engineering workflows
The ability to fine-tune and execute these models locally is particularly relevant for workloads involving proprietary data, credentials, private documents, or long-running autonomous processes.
NVFP4 and GGUF Expand Local Deployment Options #
NVIDIA is working with vLLM, Ollama, llama.cpp, and LM Studio to simplify local deployment. The ecosystem supports both NVFP4 and GGUF, giving developers multiple paths for optimized inference across different software stacks.
Unsloth also released an optimized quantized version through Unsloth Desktop, an open-source desktop environment combining local model inference, fine-tuning, agents, web research, code execution, and image/video generation.
This ecosystem approach is important because local AI performance increasingly depends on the interaction between the model, quantization format, inference runtime, GPU architecture, and application framework rather than on raw GPU specifications alone.
Blackwell Extends Across the Client and Workstation Stack #
The supported hardware range spans NVIDIA RTX PCs, DGX Spark, OEM GB10 systems, Jetson devices, RTX PRO workstations, DGX Station, GB300 desktop systems, data centers, and cloud infrastructure.
The common denominator is NVIDIA’s Blackwell architecture, allowing the same broader software ecosystem to scale from individual developer systems to larger AI infrastructure.
OEMs including Acer, ASUS, Dell, HP, Lenovo, and MSI already offer Blackwell-based systems, further expanding access to local AI workloads.
🔀 NeMo Switchyard Adds Model Routing for AI Agents #
NVIDIA also introduced NeMo Switchyard, an open-source routing library designed to select the most appropriate model for individual steps within an agent workflow.
Instead of forcing an entire workflow through a single large model, Switchyard can route tasks according to factors such as:
- Accuracy
- Latency
- Computational requirements
- Cost
- Task complexity
This architecture is increasingly important for production-grade agents. Many workflows do not require a frontier-scale model for every operation, so intelligent model routing can reduce inference costs without substantially affecting task completion quality.
According to NVIDIA’s internal benchmarks, Switchyard can maintain frontier-level task completion rates while reducing costs to roughly one-third of running the workflow exclusively on Opus 4.8.
💻 Meta’s Muse Glimmer Targets Local Coding Agents #
Another notable 30-billion-parameter model highlighted in the update is Meta’s Muse Glimmer, which provides a context window exceeding 120K tokens and is designed for coding and local-agent workloads.
NVIDIA has optimized Muse Glimmer for RTX GPUs, DGX Spark, DGX Station, and Jetson. On a GeForce RTX 5090, NVIDIA reports throughput exceeding 200 tokens per second.
Muse Glimmer can also be integrated into local agent workflows through NemoClaw, while NeMo Automodel provides a path for local fine-tuning. Keeping inference and customization on the local machine makes the stack particularly suitable for workloads involving sensitive documents, credentials, private codebases, and persistent multi-step automation.
DFlash Accelerates Speculative Decoding #
The latest llama.cpp integration also adds support for DFlash speculative decoding, providing another significant inference optimization for RTX hardware.
On a GeForce RTX 5090, NVIDIA reports that DFlash can increase decoding performance by up to 3.1× compared with the baseline, reaching approximately 233 tokens per second.
DFlash is a lossless inference acceleration approach that uses a block diffusion model as a draft model before having the larger target model verify the proposed tokens. This separates token generation from verification in a way that can substantially increase effective decoding throughput.
The technique was proposed by the Z Lab team at UC San Diego in February 2026 and represents the broader trend toward improving LLM performance through inference algorithms rather than relying exclusively on larger GPUs.
🎬 LTX-2.5 Brings Faster Local Video Generation #
NVIDIA’s update also extends beyond language models with a major upgrade to local video generation.
The open-source LTX video-generation stack has been upgraded to LTX-2.5, with optimizations targeting RTX GPUs, DGX Spark, and DGX Station.
The new version introduces multi-shot generation designed to improve consistency between clips, while a new diffusion video decoder improves visual quality and reduces artifacts.
Better Prompt Adherence and Lower VRAM Requirements #
LTX-2.5 introduces a prompt-enhancement pipeline using Gemma4 E2B alongside a customized Gemma4 12B text encoder. The combination is intended to improve the model’s ability to interpret and follow detailed generation instructions.
NVIDIA reports up to a 20% performance improvement and 40% VRAM reduction on the RTX 6000 PRO.
The stack also supports NVFP4, FastVideo, and ready-to-use ComfyUI workflows, making the optimized pipeline easier to integrate into existing local generative-AI environments.
🖥️ DGX Spark Moves Toward Multi-System Local AI Clusters #
NVIDIA is also addressing one of the biggest limitations of desktop AI infrastructure: scaling beyond a single machine.
The company introduced NVIDIA Sync Cluster Assistant for DGX Spark. The application automatically discovers multiple DGX Spark systems and connects them into a high-speed cluster using ConnectX-7 networking.
Rather than requiring developers to manually configure each system, the tool handles network setup, workload scheduling, and system-health monitoring.
This makes it possible to combine multiple desktop-class DGX Spark systems for workloads involving models that exceed the practical memory or compute capacity of a single node, including models such as GLM 5.2 and DeepSeek V4 Flash.
Resource Monitoring Becomes a Built-In Capability #
NVIDIA is also adding Sync Resource Monitor, which provides real-time and historical CPU and GPU utilization data across individual DGX Spark systems or entire clusters.
Because monitoring is integrated into the Sync environment, developers can inspect cluster utilization without deploying separate monitoring infrastructure.
NVIDIA is additionally preparing native ARM64 Linux support for Google Chrome on DGX Spark. The planned release includes one-click installation through DGX Dashboard, account synchronization, and support for Chrome extensions.
⚙️ From Local Model Inference to Persistent AI Infrastructure #
The broader significance of this update is that NVIDIA’s AI PC strategy is moving beyond simply running quantized LLMs on desktop GPUs.
The combination of open-weight 30B-class MoE models, optimized inference formats such as NVFP4 and GGUF, llama.cpp and vLLM acceleration, speculative decoding, local fine-tuning, video generation, model routing, and DGX Spark clustering creates a more complete local AI software stack.
Instead of treating the PC as an endpoint that occasionally executes an AI model, NVIDIA is increasingly treating Blackwell-powered systems as persistent AI infrastructure capable of hosting agents, coordinating multiple models, processing private data, and running increasingly complex workloads without sending every operation to the cloud.
For developers, the most important shift is therefore not any single model or benchmark. It is the convergence of local inference, agent orchestration, fine-tuning, multimodal generation, and multi-GPU or multi-node execution into a single client-side ecosystem.
As these capabilities mature, RTX PCs and DGX systems are becoming viable platforms for private AI agents and increasingly sophisticated local workloads that previously required dedicated data-center infrastructure.