DeepSeek released version 4 after 15 months of incremental updates, leaks, and rumors, but without the fanfare of earlier releases. The quiet reception reflects a shift in AI discourse from model capabilities to infrastructure and cost per token. Architecture changes in v4 are engineering wins rather than capability leaps, and NVIDIA and Lambda co-designed infrastructure to reduce serving costs, as shown in MLPerf Inference V6 results.
Why it matters: The release signals that the AI industry's focus is moving from model breakthroughs to practical deployment and cost optimization.
Falcon-180B, the largest open-source LLM to date, requires 400GB of VRAM to run unquantized. RunPod explains how to deploy it using A100 GPUs.
Why it matters: This provides a practical guide for deploying a massive open-source model, highlighting the hardware demands and accessibility via cloud GPU services.
Together AI published real-world inference benchmarks for coding agents, reporting 31% higher throughput than TensorRT-LLM, 2× better time-to-first-token at saturation, and 76% lower cost than Claude Opus 4.6. The benchmarks focus on scaling inference for agentic coding workloads.
Why it matters: This demonstrates significant performance and cost improvements for deploying coding agents at scale, which could accelerate adoption of AI-assisted development.
Nvidia has launched a new platform that applies its expertise in autonomous vehicle safety to physical AI, aiming to make robots safer. The system leverages Nvidia's experience in self-driving car safety to address safety challenges in robotics.
Why it matters: This move extends Nvidia's safety technology from autonomous vehicles to the broader robotics industry.
Together AI has made NVIDIA Nemotron 3 Super and Nemotron 3 Nano Omni available on its platform. Nemotron 3 Super offers efficient multi-agent reasoning and a 1M-token context window, while Nemotron 3 Nano Omni is a single open model that can process video, images, audio, and text for agentic workloads at scale.
Why it matters: These launches provide developers with production-grade, multimodal AI models optimized for agentic reasoning and scalable deployment.
RunPod has introduced a vLLM worker on its serverless GPU platform, allowing users to deploy Meta's Llama 3.1 efficiently. The company offers step-by-step guides for model setup and emphasizes performance benefits. This update enables users to run large language models without managing complex infrastructure.
Why it matters: It lowers the barrier for developers to deploy advanced LLMs like Llama 3.1 with optimized inference on serverless GPUs.
The Beijing Academy of Artificial Intelligence has released Orca, a world model that predicts abstract world states instead of tokens or pixels. Trained on 125,000 hours of video without any action labels, Orca matches the specialized π0.5 on five robotics tasks. This approach could help ease the field's chronic data shortage.
Why it matters: Orca demonstrates that world models can achieve competitive performance on robotics tasks without requiring expensive action-labeled data, potentially accelerating progress in robotics.
Together AI has announced an integration with Goose that allows users to deploy any Hugging Face model in a single session using Dedicated Container Inference. This approach removes setup complexity, enabling models to run in a production-grade GPU environment immediately upon release.
Why it matters: This integration streamlines AI model deployment, making it more accessible to developers without requiring infrastructure expertise.
RunPod has launched a redesigned website and refreshed its brand identity, aiming to provide a clearer and faster user experience. The platform continues to focus on powering real-time inference, custom LLMs, and other AI workloads.
Why it matters: The redesign highlights RunPod's ongoing commitment to supporting AI inference and model deployment for developers.
RunPod has introduced updates to its serverless platform, with a focus on supporting faster and more scalable deployments for large language model (LLM) workloads. The 2025 update is designed to improve efficiency and scalability for users deploying LLMs. More information is available on the RunPod blog.
Why it matters: These updates are important for developers and enterprises seeking efficient, scalable serverless infrastructure for LLM deployments.
Together AI has launched DeepSeek-V4 Pro, featuring a 512K context length and controllable reasoning modes. The model offers cached-input pricing for long-context workloads such as code agents, document intelligence, and research synthesis.
Why it matters: This release provides developers with a powerful, cost-efficient model for complex reasoning tasks requiring extended context.
RunPod published a guide on transitioning from Pods to Serverless for model inference after training. The guide discusses the trade-offs involved and offers advice on optimizing for fast deployment. It aims to help users determine the right time to switch deployment strategies.
Why it matters: This guide helps AI developers make informed decisions to optimize inference costs and performance.
Together AI published a blog post detailing how it serves MiniMax-M3 efficiently, enabling 1M-token context and multimodality. The optimizations include KV-block-major sparse attention, paged MSA decode, optimized index scoring, and a Rust-based multimodal gateway.
Why it matters: This demonstrates practical techniques for deploying large multimodal models with long context windows, which is critical for enterprise applications requiring processing of extensive documents and multiple data types.
Anthropic's Claude models are now available in Microsoft Foundry, providing enterprises with broader access to build domain-specific, autonomous AI agents. This integration enables businesses to utilize Claude's capabilities within Microsoft's AI platform.
Why it matters: This move expands enterprise access to advanced AI models for developing custom autonomous agents.
RunPod has introduced cost centers, a new feature that helps teams monitor and allocate GPU spending. This tool enables users to track GPU expenses across different projects or departments.
Why it matters: This feature supports better budget control and resource allocation for teams using cloud GPUs.
RunPod's blog introduces SGLang, a framework for structured LLM workflows designed to boost inference performance and enable response customization. The post explains how SGLang can be used to enhance LLMs, targeting developers interested in optimizing their models.
Why it matters: SGLang provides a new approach to improving LLM inference efficiency and customization, which is important for deploying responsive AI applications.
Together AI published a blog post detailing the inference systems work required to serve DeepSeek-V4, which supports million-token context. The post covers compressed KV layouts, prefix caching, kernel maturity, and endpoint profiles for long-context workloads on NVIDIA HGX B200 hardware.
Why it matters: This highlights the growing importance of inference infrastructure as models scale to million-token contexts, a key challenge for enterprise AI deployment.
Google has announced an expansion of managed agents in the Gemini API, introducing support for background tasks and remote MCP. These enhancements are part of a new feature bundle launch aimed at developers.
Why it matters: This update gives developers enhanced agent capabilities within the Gemini API.
RunPod published benchmarks for its Overdrive inference optimization, testing four models across sixteen workload profiles. The results detail performance measurements for various AI inference tasks.
Why it matters: This provides developers with concrete performance data to optimize AI inference workloads on RunPod's infrastructure.
Together AI has announced an $800 million Series C funding round aimed at accelerating the shift to open-source AI. The company emphasized that the economics of closed models do not scale and shared plans for future development.
Why it matters: This significant investment highlights growing market confidence in open-source AI as an alternative to proprietary systems.