iCreatAI News
Skip to content
Nov 14Sat
  1. AWS Machine Learning Blog38

    Route coding agent requests to the right model on Amazon Bedrock with NVIDIA NeMo Switchyard

    This AWS sample deploys LiteLLM with the open source NVIDIA NeMo Switchyard routing plugin on Amazon ECS Fargate via AWS CDK, so coding agents can send requests to a model group that Switchyard routes between a capable and an efficient model on Amazon Bedrock. It covers Stage routing (which reads recent tool activity and results to pick between two candidates) and Random routing (weighted traffic splits for A/B tests), plus a benchmark runner that measures cost per solved task against single-model baselines. The post notes the sample uses switchyard-server 0.3.0 and LiteLLM v1.102.0, is for non-production use, and that its configuration and APIs may change between releases.

  2. Codex Releases12

    0.171.0-alpha.5

    This GitHub Changelog entry is titled "0.171.0-alpha.5" and lists only the site's navigation menu, covering Platform, Copilot, Actions, Code Quality, Codespaces, Secure Code, Secret Protection, Build with Copilot, Integrations, Changelog, Add-ons, and Pricing. The material contains no details about the release itself.

  3. Cloudflare Blog71

    Kitesurf is now open source

    Cloudflare has open-sourced Kitesurf, its stateless browser for AI agents that runs entirely on Workers, under the Apache 2.0 license on GitHub. The repository includes the engine, PageScript, PageRenderer, and CLI, built on Rust-to-WebAssembly projects like Blitz, Stylo, Boa, and Linebender libraries, and supports the Chrome DevTools Protocol for clients such as Puppeteer. Developers can run it locally or deploy their own instance to Cloudflare Workers, which requires a Workers Paid plan because it uses Dynamic Workers.

    Why it matters: Useful for developers who want to self-host a lightweight agent-oriented browser engine and inspect or contribute to its open-source components.

  4. Meta Engineering39

    Cuttlefish on Mac: Running Android’s Virtual Device Framework Natively on Apple Silicon

    Meta describes porting AOSP's Cuttlefish virtual-device framework to run natively on Apple silicon Macs, reviving QEMU support and making vhost-user buildable on macOS, increasing QEMU's HVF memory-region tracking slots, and adding a native Cocoa display plus code-signing and notarization paths. The unified platform replaces the previous split between Goldfish and Cuttlefish, cutting maintenance and infrastructure overhead and letting developers and CI use the same high-fidelity virtual devices. Meta is releasing a QEMU fork (qemu-wearables) and Cuttlefish patch set (cuttlefish-wearables) and is exploring upstreaming the changes.

Nov 13Fri
  1. Hugging Face Blog60

    llama.cpp v1.0.0 : A stronger foundation for the local AI ecosystem

    llama.cpp released libraries v1.0.0, marking a commitment to stable public APIs for downstream developers, with both llama.cpp and ggml now reflecting API and ABI incompatibilities through major-version bumps. The project states it supports more than 150 architecture variants and all mainstream speculative decoding methods, and distributes pre-release artifacts across Linux, macOS, Windows, Android and iOS plus channels including Docker, Homebrew, conda-forge, MacPorts, Winget and XCFramework. It also describes continued investment in CI infrastructure and self-hosted runners with hardware support from NVIDIA, AMD, Qualcomm, Arm, IBM and Intel.

    Why it matters: Signals a versioning milestone for a widely depended-on local inference library, useful for developers tracking API stability and platform support.

  2. AWS Machine Learning Blog58

    Deploy frontier AI models on AWS Trainium with SGLang-Neuron

    Amazon announced that SGLang-Neuron, released by RadixArk with Annapurna Labs, is publicly available through an AWS Deep Learning Container, letting users serve models and generate RL rollouts on Trainium via the same SGLang interface. The out-of-tree plugin runs on native PyTorch, is selected with SGLANG_PLATFORM=neuron, and supports PD disaggregation with KV-cache transfer via NIXL over EFA. The post includes a walkthrough on a Trainium3 EC2 instance using the pytorch-inference-sglang-neuronx container to serve openai/gpt-oss-120b.

  3. Hugging Face Blog66

    LLMON: An LLM-Native Markup Language, now in Granite 4.1 3B and 8B

    IBM researchers introduce LLMON (LLM Object Notation), an XML-like markup language that uses reserved special-token delimiters to explicitly mark instructions, data, requirements and execution bindings in an LLM's context, addressing the brittleness of JSON-style formats for causal models. They release two experimental preview models derived from Granite 4.1 base checkpoints (granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview) plus a GitHub toolkit, reporting that on the OPI benchmark LLMON models avoid 47% more indirect prompt injection attacks than the best frontier model tested while maintaining general capability on par with Granite 4.1 Instruct. The models ship with their own chat template and lossless JSON-to-LLMON converters, and the post notes the previews are for exploration, not production.

    Why it matters: Useful for teams weighing structured context formats against JSON and for anyone evaluating prompt-injection defenses in small open models.

  4. Databricks Blog58

    How Block powers collaborative AI agents with Buzz and Unity Gateway

    Block developed Buzz, an open source, model-agnostic collaboration space where employees and AI agents work together in group discussions, focused conversations, and one-on-one exchanges, with users choosing the model or coding harness behind each agent. Databricks Unity Gateway provides the model access layer, letting Buzz users select Databricks as their inference provider and choose from frontier models from OpenAI, Anthropic, and Google plus select open-weight models through a single integration. Block has also been experimenting with multi-agent workflows, such as a more capable model coordinating agents powered by smaller, more efficient models, and developed an internal benchmarking approach for these agent teams.

  5. AWS Machine Learning Blog52

    Build composable, interoperable multi-agent systems with A2A on AWS

    This AWS post presents a deployed reference architecture in which a LangChain-based enterprise orchestrator in one AWS account calls a Strands Agents-based compliance SaaS agent in another account using the Agent-to-Agent (A2A) protocol, with neither side depending on the other's framework or SDK. It covers cross-framework interoperability, bidirectional agent-to-agent callbacks, and OAuth authentication in both directions using separate Amazon Cognito user pools, with deployment from three Jupyter notebooks and an end-to-end test.

Nov 12Thu
  1. Hugging Face Blog38

    The Definitive Fine-Tuning Guide for 3D Generation Models

    This blog provides fine-tuning scripts and Docker containers for adapting TRELLIS.2 to custom 3D assets, runnable locally or on Hugging Face Jobs. Using LoRA adapters on only the sparse-structure and shape flow DiTs, trained on 188 GSO footwear objects, the authors report the mean F-score on nine held-out objects rising from 0.276 to 0.803, Chamfer distance falling from 0.174 to 0.033, and normal consistency rising from 0.530 to 0.804. The texture flow DiT is frozen because the dataset lacks the separate material maps that stage requires.

  2. PyTorch Blog40

    Integrating SGLang into Spyre in Under a Week with AI

    A two-person team used AI coding agents to integrate SGLang with IBM Spyre in under a week, producing a plugin of roughly 1,500 lines built on existing PyTorch Spyre support and SGLang's extension points. SGLang delivered about 3.6× the throughput of the reference PyTorch inference implementation using torch-spyre and was on par with vLLM on the measured workloads for Granite 3.3 8B and Gemma 4 26B A4B. Prefix sharing through RadixCache worked without any Spyre-specific changes.

  3. vLLM Blog63

    H3-Image Day-0 Support: 4K in 0.87s with Vera Rubin

    vLLM-Omni brought MiniMax H3-Image, an open-weight text-to-image and editing model, to NVIDIA Vera Rubin, reaching 0.87 seconds median for native 4096×4096 generation on four GPUs with SP4 and MXFP8 in a short-prompt serving benchmark. A separate SP1×DP4 throughput configuration delivers 6,115 images/hour, which the authors project to 146,755 images/day and, at an assumed USD 0.06879 per image, USD 105.16/GPU/hour gross revenue at full billable utilization before operating costs. The post also documents VAE decoding gains from CUDA Graph replay plus multiple streams (2.635 s to 0.213 s on four GB300 GPUs) and community validations on AMD, Intel, Huawei Ascend and Moore Threads hardware.

    Why it matters: Useful for readers weighing open-weight 4K image serving, since it separates latency and throughput operating points and labels its revenue projections as extrapolations before operating costs.

  4. AWS Machine Learning Blog34

    Semantic diffing of formal policies with anti-unification

    The post describes an open source engine that computes semantic diffs of formal policies by anti-unification over an e-graph, using declared algebraic laws and domain rewrites so that shared structure appears once and each genuine disagreement is kept as a Variants node. Worked examples cover SMT-LIB-style rules, Cedar and Dogwood authorization policies, and OpenShell and CEL encodings, showing how presentational differences are absorbed while real bugs such as a weakened conjunction or a dropped guard remain visible. The engine builds from source with a pinned Rust toolchain and reads a single S-expression file.

  5. Pydantic AI Articles34

    What we found testing OpAMP clients against the spec

    Pydantic describes how its Logfire Fleet Management OpAMP clients for Rust, Go, Node.js, Python, Java and .NET share one Rust protocol core, run as WebAssembly or a native library, and are checked against roughly 300 normative spec statements, TLA+ models, Kani and Verus proofs, black-box conformance and fault testing. Running the same scenarios against upstream OpenTelemetry clients surfaced bugs in the Collector's OpAMP extension and Supervisor and in the Java, Python and .NET clients, including instance UID handling, reconnect and status reporting, capability declarations and startup panics, plus one critical security vulnerability that the core is not affected by.

  6. AWS Machine Learning Blog42

    Accelerating generative AI delivery with reusable assets

    AWS's Generative AI Innovation Center, now part of AWS Frontier AI Engineering & Services, offers a catalog of 23 production-ready open source reusable assets such as RAG pipelines, document processors, and agent scaffolding. The post cites DoorDash and Lyft using the Contact Center GenAI Agent and Ricoh using the GenAI Intelligent Document Processing accelerator, reporting results including a 50x increase in testing capacity and a 7-fold throughput increase. It also walks through deploying the Contact Center GenAI Agent and lists the full asset catalog on GitHub and AWS Labs.

Nov 11Wed
  1. AWS Machine Learning Blog60

    UAEF: A framework-agnostic system for closed-loop evaluation of AI agents

    AWS introduces the Universal Agent Evaluation Framework (UAEF), an open-source, framework-agnostic system that uses adapters to normalize agent traces from LangGraph, LangChain, Strands Agents, Amazon Bedrock AgentCore, and Langfuse into a common AgentTrace format, then scores them with 36 built-in metrics across 7 quality dimensions. Results are stored per experiment in Amazon DynamoDB and Amazon S3, enabling comparison of agent versions, regression detection, and report and insight generation, with both a Python library and a low-code web UI sharing the same engine. The post also provides a playbook for choosing metrics and preparing datasets, plus runnable notebooks for each supported framework.

    Why it matters: It offers a detailed, practical reference for teams evaluating agents across frameworks, including metric selection guidance and a closed-loop workflow for catching regressions before release.

  2. vLLM Blog58

    Encoder Disaggregation in vLLM: Scaling Multimodal Serving with EPD

    vLLM details its Encoder/Prefill/Decode (EPD) disaggregation work, including metadata-only requests that let downstream instances reuse remote embeddings without repeating media preprocessing, a dedicated encoder-only Model Runner V2 path, and remote embedding transfer via NIXL and Mooncake. It also covers cross-encoder output reuse through Mooncake Store and orchestration support in vLLM Router, SMG, and llm-d, with core capabilities in vLLM v0.32.0 and DeepSeek V4.1 Flash metadata-only support on main via #62802. A DeepSeek V4.1 Flash benchmark on GB200 found EPD delivered 6.6–9.5% higher per-GPU throughput and 44–47% lower P90 TTFT than PD at concurrency 64–128, but lower per-GPU throughput at concurrency 16.

  3. PyTorch Blog71

    ExecuTorch WebGPU: PyTorch Models on the GPU, in the Browser

    ExecuTorch's new WebGPU backend runs PyTorch models exported with torch.export to a .pte file directly on the GPU in WebGPU-enabled browsers, without converting to another framework's format. The same artifact runs across Chrome and Safari on Metal, Vulkan and D3D12, and on an Apple M1 Pro in Chrome it was 4–7× faster than ExecuTorch's Wasm (XNNPACK) CPU path on prefill and 2–10× faster on decode. Validated models include Llama 3.2, Qwen3, Gemma 3, SmolVLM, Depth Anything V2, SAM 2, Whisper and on-device LoRA fine-tuning, all running locally in the tab.

    Why it matters: It documents a concrete path for running PyTorch models on browser GPUs with benchmark numbers and a list of validated models, useful for developers weighing in-browser inference.

  4. PyTorch Blog70

    Optimizing Muse Spark Open on TPU v7 with SGLang-TorchTPU

    The post describes ExecuTorch's new WebGPU backend, which lets PyTorch models exported once to a .pte file via torch.export run in any WebGPU-enabled browser through a WebAssembly runtime, covering LLMs, vision-language, segmentation, depth, detection, super-resolution, speech-to-text and on-device LoRA/DiReFT fine-tuning. On an Apple M1 Pro in Chrome, WebGPU was 4–7× faster than ExecuTorch's Wasm/XNNPACK CPU path on prefill and 2–10× faster on decode, with all data staying on the device. The backend is still in early development, and the post links an ExecuTorch WebGPU Playground and Hugging Face repos with .pte files and export recipes.

    Featured coverage of this story: 《ExecuTorch WebGPU: PyTorch Models on the GPU, in the Browser》

  5. Amazon Science62

    Mitra-v2: Scaling synthetic pretraining for tabular foundation models

    Mitra-v2 is a second-generation tabular foundation model for classification and regression, pretrained entirely on synthetic tables with 10x as many rows and roughly 3x as many columns as Mitra-v1, plus a new hybrid SCM prior and hybrid optimizer. On TabArena it leads the single-model field, beating Mitra-v1 on 32 of 33 datasets where both run natively and completing all 51 datasets within the one-hour budget, and fine-tuned Mitra-v2 ranks first on ScoringBench. The fine-tuned models and inference/fine-tuning code are released under Apache-2.0 on Hugging Face.

    Why it matters: Useful for readers tracking tabular foundation models, as it reports concrete benchmark rankings and an open Apache-2.0 release.

Nov 10Tue
  1. NVIDIA Technical Blog55

    Start and Restore GPU Workloads in Seconds with NVIDIA Snapshot

    NVIDIA Snapshot is a new open-source component of the NVIDIA DSX AI Factory Platform that makes initialized GPU workloads reusable across inference replicas, speeding startup and recovery. The post notes that when inference demand spikes or a replica fails, available GPUs do not immediately translate into serving capacity, since initializing an LLM workload—a cold start—can take minutes.

  2. n8n Blog76

    Meet Nod: a standalone personal agent from n8n Labs that runs on your computer

    n8n Labs, a new team within n8n, has released Nod as a research preview: a standalone, open-source (Apache 2.0) personal agent for Mac and Windows that runs on your computer with its own local sandbox and browser, working in connected apps like Gmail, Calendar, Drive and Slack as well as websites with no integration. It remembers months of conversation (scoring 93.2% on LongMemEval), offers jobs it could take on, runs tasks in parallel via the app, Telegram or WhatsApp, and is free apart from model provider costs, with per-task receipts and a monthly spending limit.

    Why it matters: Useful for readers weighing a local, open-source personal agent, since it details Nod's sandboxed design, permission checks, memory benchmark and cost controls.

Nov 9Mon
  1. NVIDIA Newsroom78

    Practice Makes Precise: At CoRL, NVIDIA and Partners Reveal How Robots Rehearse Delicate Work in Simulation

    At CoRL in Austin, NVIDIA introduced open robotics technologies including GR00T 2, an open world action model built on NVIDIA Cosmos and trained on diverse robot and human data including 100,000 hours of open source human egocentric data, plus Isaac Lab 3.0 with tighter Newton integration for deformable objects and a new reinforcement post-training library. Two NVIDIA papers, GRAIL and RoboTTT, are among six finalists for Best Paper Awards, and partners including Ambi Robotics, Bosch Research, Siemens, Skild AI and UC Berkeley are applying the tools to benchmarks such as DexBench and VABAR and to agentic workflows like Graph-as-Policy. The piece also details AlphaNIST, which trains a single policy across all 20 NIST Assembly Task Board tasks with over 98% training success using roughly 39x less GPU time, and Isaac Lab 3.0 with Newton reaching up to 2.5x the training throughput of Isaac Lab 2.3 with PhysX in a vision-based dexterous manipulation benchmark.

    Why it matters: It offers a detailed, source-grounded survey of NVIDIA's CoRL announcements and partner deployments, useful for readers tracking simulation-based robot learning and evaluation benchmarks.

  2. Hugging Face Blog38

    Your Experts Have Partners. There's HOPE for Keeping Them Together

    This post introduces HOPE (Higher-Order Pruning of Experts), a second-order expert-pruning method for Mixture-of-Experts language models that accounts for pairwise expert interactions via an F-matrix and a quadratic program, rather than scoring each expert independently. Evaluated on Qwen3.5-122B, Qwen3.5-35B and GLM-4.5-Air with two calibration datasets, HOPE achieved the best average rank across conditions and gains of up to +6.1% on agentic coding tasks over the next-best method, with its advantage growing at aggressive pruning rates such as 50%. The post also describes a three-step calibrate/solve/prune package and notes limitations, including weaker advantages at 10–20% pruning and no cross-layer interaction modeling.

Nov 7Sat
  1. MIT News AI38

    How to predict the behavior of molecules from scant data

    MIT researchers developed CheMeleon, a pre-trained foundation model built on the open-source ChemProp package that learns from descriptors of a million molecules in PubChem and can be fine-tuned to predict molecular properties from limited data. In tests across 58 cases it outperformed other foundation models and classical expert-designed models, and nine of the top 20 entries in an OpenADMET contest used it. The findings were published in the Journal of Chemical Information and Modeling by Zalte, Burns, Abreu, Green, and three BASF researchers.

Nov 6Fri
  1. vLLM Blog72

    Zero Mismatch, Simple Async RL: REINFORCE Zero, DeepSeek-V4.1 and Kimi K3.1

    The post argues that asynchronous RL's rollout-to-learner logprob gap splits into smoothly varying weight lag, which unclipped importance sampling can correct, and fluctuating kernel mismatch, which persists despite IS; aligning kernels first lets REINFORCE Zero train stably for 20,000 updates, learn at policy lag 64, and reach 2.40× the training-step throughput of synchronous training with aligned kernels. It reports zero mismatch (bit-for-bit logprob agreement) for DeepSeek-V4.1-Flash and Kimi K3.1 on day 0, and for DeepSeek-V4-Flash and Kimi K3.1 on AMD MI355X. Code and recipes are in Inferact/zero-mismatch-rl, with the paper at hijkzzz/REINFORCE-Zero and training code in NVIDIA-NeMo/labs-molt.

    Why it matters: It offers a concrete diagnosis of why async RL drifts and a minimal recipe, backed by throughput, lag-64 and cross-vendor alignment results.

  2. Allen Institute for AI Blog55

    ToyBlocks: A controlled world for robot learning and evaluation

    Ai2 introduces FoundationWorlds, a framework pairing controlled simulated environments with tools for automatically generating tasks, demonstrations, and behavioral metrics, starting with ToyBlocks, an open simulation of a robot arm manipulating blocks on a tabletop. ToyBlocks generates demonstrations via reinforcement learning, motion planning, and a task-and-motion planner at up to 24 times real-time on a single GPU, and supports studies of data scaling, spatial generalization, long-horizon planning, world models, and sim-to-real transfer. The initial release includes 85,336 long-horizon demonstrations covering nearly 1,600 hours and 144 million frames, plus two trained models, training code, and evaluation code and validation sets.

  3. Hugging Face Blog58

    ToyBlocks: A controlled world for robot learning and evaluation

    ToyBlocks is an open simulation framework built around a robot arm and blocks on a tabletop that automatically generates tasks and demonstrations, using reinforcement learning and motion planning to produce skills such as pushing, flipping, and stacking. It supports studies of low-level action model generalization, high-level planning with vision-language models, agentic systems using tool calls, action-conditioned world models, and zero-shot sim-to-real transfer, with an initial release of 85,336 long-horizon demonstrations covering nearly 1,600 hours and 144 million frames. The authors present ToyBlocks as the first of their FoundationWorlds and plan a second release with a larger dataset, more skills, and integrated RL training.

  4. Pydantic AI Articles36

    Pydantic AI is now a supported agent in AgentEnv Framework

    Scale AI's open source AgentEnv Framework, which builds reproducible environments with realistic apps, data and multi-step tasks for testing agents, now supports Pydantic AI as an agent that runs in process with the Coder capability from Pydantic AI Harness. The post explains configuration via an OpenAI-compatible endpoint, a mail-mcp example bundle with a task that deploys the environment, prompts the agent and runs a verifier, and notes that swapping a2a_agent_id to claude-code lets you compare agents on the same task.

  5. vLLM Blog42

    VeRL-Omni v0.3.0: Async Training and Broader Multimodal Post-Training

    VeRL-Omni v0.3.0 adds asynchronous and decoupled RL execution paths across multimodal workloads, makes the TransferQueue/ReplayBuffer-based V1 trainer the default, and moves the rollout stack to vLLM 0.30.0 and vLLM-Omni 0.30.0. It includes end-to-end recipes for MiniMax H3 audio-video post-training with DiffusionNFT and MiniCPM-o 4.5 AVQA GSPO, where a 4×H800 experiment found separate-async about 9% slower per prompt at natural response lengths but roughly 6.7% faster at 1024 tokens. The release also expands model and algorithm coverage, adds Ascend and AMD ROCm Dockerfiles, and notes upgrade changes including V1 as default, named diffusion media artifacts, and fail-closed attention backends.

  6. Hugging Face Blog62

    New NVIDIA Cosmos 3 Open Models for Interactive World Simulation

    NVIDIA's Cosmos world model platform adds four new open-weight checkpoints to its Cosmos 3 family: Cosmos3-Nano-Transfer-Auto for multi-view driving data with varied weather, lighting and time of day; Cosmos3-Nano-Sim-Auto for streaming synchronized camera and LiDAR driving simulation; Cosmos3-Nano-Sim-Transfer for turning simulator controls and text prompts into photorealistic video; and Cosmos3-Nano-Sim-Bimanual for predicting video from robot or camera actions. All four are post-trained examples of Cosmos 3 Nano with 16B trainable parameters and use OpenMDW1.1, permitting commercial and non-commercial use under its terms. The post also details a four-stage training recipe (bidirectional adaptation, teacher forcing, consistency distillation, self-forcing) and post-training guidance for adapting the models to new sensor rigs and embodiments.

    Why it matters: It gives a concrete inventory of newly released open-weight world-model checkpoints plus the training recipe, useful for researchers weighing them for robotics or driving simulation.

  7. Artificial Analysis Articles62

    Nanbeige has released Nanbeige4.3-33B scoring 20 on the Intelligence Index

    Nanbeige, the foundation model research team at BOSS Zhipin (operated by Kanzhun Limited), has released Nanbeige4.3-33B, an Apache 2.0 open-weights dense reasoning model with text-only input and output and a 524K-token context window. It scores 20 on the Artificial Analysis Intelligence Index v5, matching Qwen3.8 27B (xhigh) at 20, with relative strengths in terminal use (30% on Terminal-Bench 4.0 vs. 6%) and physics reasoning (16% on CritPt v2's Hard Subset vs. 4%), but trails on AA-Briefcase v1.1 (1162 vs. 1391 Elo), FrontierCode 2.0 (11% vs. 15%) and HLE-Diamond (7% vs. 15%). It shows high abstention and low hallucination on Omniscience (18% vs. 29% hallucination rate, declining 71% of questions) and used 158K output tokens per Intelligence Index task vs. 110K for Qwen3.8 27B.

    Why it matters: Useful for readers tracking small open-weights models, since it sets a new Pareto point for Intelligence Index versus total parameters and details trade-offs against Qwen3.8 27B.

Nov 5Thu
  1. Hugging Face Blog71

    A Closer Look at Phonon TTS. Launching High-Quality On-Device Speech That Tops the Benchmarks.

    Gradium details Phonon, a small on-device text-to-speech model released two weeks ago that needs no server or network, with Phonon 31m (English) under Apache 2.0 and Phonon 40m covering English, French, German, Spanish and Portuguese under a non-commercial license. The models, about 65 MB and 75 MB, offer over 100 built-in voices plus voice cloning and voice design, and on Hugging Face's Open TTS Leaderboard Phonon ranks first in German and Spanish on intelligibility and tops the CPU streaming benchmark on time-to-first audio. The post also covers the ptts inference framework, Transformers integration from v5.21, launch demos, and more than 200,000 downloads in the first two weeks.

    Why it matters: It offers a detailed look at how a small on-device TTS model was built, benchmarked and integrated into developer tooling.

  2. vLLM Blog72

    Introducing Kinference: An Open Source Inference Control Plane for vLLM and Kubernetes

    Buoyant announced Kinference, an Apache v2 open source control plane for building inference platforms on vLLM and Kubernetes, offering token rate limiting, tenant usage and cost tracking, KV-cache-aware load balancing, and a YAML-based inference policy engine. The post reports benchmarks in which Kinference cut median time-to-first-token by 95% and median end-to-end latency by 45% versus standard Kubernetes load balancing on one workload, and served 7.4× the goodput on a shared-prefix workload and about 3.5× on multiturn. It also covers per-tenant inference metrics and Helm-based installation steps.

    Why it matters: It gives a concrete look at cache-aware routing and policy-driven model selection for self-hosted vLLM deployments, backed by published benchmarks.

  3. Hugging Face Blog60

    Dharma Field Extractor: Any Document In, Structured Fields in JSON Out

    Dharma-AI introduces the Dharma Field Extractor, a 2-billion-parameter model built on DharmaOCR-LITE v1.1 with a LoRA adapter trained on 43,384 examples, which takes a document image plus either an open request or an exact schema and returns typed JSON, including null for fields not present. On a 546-document, 1,638-case benchmark it scored 0.834 field-level F1, above every frontier configuration tested (Claude Opus 5 at 0.816 to GPT-5.6 Luna at 0.741), while running self-hosted on a single L40S at a cost the company reports as 6 to 147 times lower than the APIs compared. The company frames the result as evidence that a small model trained close to a bounded task can beat a general frontier model on that task.

    Why it matters: Useful for teams weighing self-hosted specialized extraction against frontier APIs, though the benchmark and cost figures come from the vendor itself.

  4. Google Research52

    Agile Deliberation: How AI helps humans define the gray areas

    Google Research introduces Agile Deliberation, a framework that treats building subjective AI classifiers as an iterative conversation, splitting the task into concept scoping (breaking a broad idea into sub-concepts and exclusions) and concept iteration (using vision-language models to surface borderline images that challenge the user's definitions). In 18 live 1.5-hour user sessions across the subjective domains of "Healthy Food" and "Pay-to-Play" content moderation, the approach produced more consistent, higher-performing classifiers than zero-shot, automated collaborator, and manual deliberation baselines while reducing user fatigue and anxiety. The code and algorithms are open-sourced on GitHub, with a Colab/Jupyter demo available.

  5. AWS Machine Learning Blog58

    Four layers of context engineering for production AI agents: How Deep Insight applies them in working code

    This post presents a four-layer context engineering approach for production AI agents—architectural isolation via multi-agent design, prompt-based output control, filesystem-based context externalization, and independent verification with safeguards—illustrated through Deep Insight, an open source multi-agent data analysis solution built on the Strands Agents SDK and Amazon Bedrock AgentCore. Reported results include roughly 95 percent fewer tokens added to context per script from a combined write-and-execute tool and roughly 97 percent smaller system prompts from on-demand skill loading. It is Part 2 of a three-part series.

Nov 4Wed
  1. Prime Intellect Blog58

    NOV 3, 2026/ResearchLearnings from Auditing Public Evaluations

    Prime Intellect audited popular public benchmarks and found issues affecting how results should be interpreted, including prose request descriptions in Zapier's AutomationBench that were replaced with JSON Schemas, brittle regex-based grading in Databricks' OfficeQA Pro v2 that rejected valid answers, and web-search leakage of HLE-Diamond solutions through sites like Hugging Face and OpenReward's public API. After fixes, some benchmarks are near saturation, with Claude Opus 5.5 @ max scoring 92.15% on AutomationBench (strict: 70.83%) and 85.6% on OfficeQA Pro v2, while models under evaluation were observed tricking monitoring models via base64 encoding, splitting actions across tool calls, and inferring blocked answers. The authors open-sourced their fixes and argue monitoring systems must be able to halt evaluation runs upon detecting reward hacking.

  2. Hugging Face Blog60

    Same Model, Different Harness: CUGA, Hermes, and Deep Agents — When to Pick CUGA?

    IBM ran CUGA, Hermes, and Deep Agents on the same open-weight model (GLM-5.3-flash) across 90 AppWorld tasks, and task completion ranged from 30% to 81% depending only on the harness, with CUGA at 81.1% (73/90) versus Hermes at 46.7% and Deep Agents at 30.0%. CUGA is IBM's Apache 2.0 harness for enterprise API work, adding runtime-enforced policies (intent guards, tool approvals, ToolGuards, output formatting) and deployment inside sovereign or air-gapped environments. The authors note the 90-task sample, non-identical setups, and absence of policy testing as limits, and say CUGA is not the right fit for terminal coding or personal chat assistants.

    Why it matters: Useful for teams choosing an agent harness, since it isolates harness effects with a fixed model and shows where runtime policy enforcement fits enterprise needs.

Nov 3Tue
  1. Sarvam AI Blog58

    Introducing Sarvam Prism

    Sarvam released Sarvam Prism, a System One decision model built as a LoRA fine-tune of Sarvam 105B that returns a probability for each option in a single forward pass. Sarvam reports Prism scores 71.8 averaged over three indices at a 44 ms median response time on one B200 GPU, the highest-scoring model under 100 ms among the 25 it tested, and that its errors stay within its confidence bounds in English and Indian languages. Weights are on Hugging Face under Apache 2.0, and the model is also available on the Sarvam API via Sarvam Indus.