iCreatAI News
Back to main site
Skip to content
Only Papers#Hugging Face
Nov 14Sat
  1. MarkTechPost59

    Stanford Research Open-Sources Memorilla, a Latent Memory Module for Frozen LLMs

    Stanford researchers have released Memorilla, an open-source 194M-parameter memory module that condenses a whole document collection into 16 memory tokens for a frozen LLM, with code, training recipes and checkpoints public under Apache 2.0. On FactKG, a frozen Qwen3-8B scored 84.33% with Memorilla versus 80.43% for full context, using 108x fewer document tokens, and it reported the highest average (49.68) across 8 benchmarks while losing on PubMedQA (59.00% vs 69.00% for LLMLingua-2). The quickstart requires one 80 GB CUDA GPU, and the full training recipe takes about 19 hours on 4 H100 GPUs.

  2. Hugging Face Blog40

    XYBench: Helping with the problem behind the question

    XYBench is a new benchmark of 8,115 real-world questions, drawn from Stack Overflow, Stack Exchange, curated examples, and WikiHow, that tests whether AI assistants recognize when a user's request rests on a misconception (the "XY problem") and redirect them toward their actual goal. Evaluating Claude Sonnet 4.6, GPT-5, Gemini 3.1 Pro, Qwen3.5-9B, and DeepSeek-V4 Pro, the authors found every model was more likely to address the stated request than the underlying goal, and all trailed human reference answers on misconception identification. The paper won the Best Resource Paper Award at EMNLP 2026.

Nov 8Sun
  1. The Decoder58

    AI auditors catch most agent misbehavior but flood reviewers with false alarms, new study finds

    In a Substack post, Anthropic Institute researcher Andy Hall replicated and extended METR's MALT study by testing GPT-5, GPT-5 nano, GPT-6 Astra, Opus 4.8 and Gemini 3.8 Flash as auditors of agent logs at a 5 percent false-alarm rate, finding GPT-5 caught nearly 98 percent of reward-hacking cases while Gemini 3.8 Flash found barely half, and all auditors caught 71 to 81 percent of mostly synthetic sandbagging cases. Newer and more expensive models were not consistently better, and at an 80 percent detection rate the best models reached false-alarm rates of 2.5 to 3 percent while Opus 4.8 and Gemini 3.8 Flash flagged 40 and 63 percent of harmless logs. Hall calls for auditor models with far fewer false alarms and says it remains open whether models judge agents from their own maker fairly.

Nov 7Sat
  1. The Decoder71

    State propaganda in some Chinese AI models is only "a thin layer" that fine-tuning can strip away

    Nous Research fine-tuned Alibaba's Qwen3.8-27B on about 53 million tokens to create Arete-27B, which cuts its propaganda rate from 22 percent to 0.9 percent while keeping general capabilities roughly intact. The team built an 803-prompt test across seven categories and judged responses with Grok 4.7, finding Western models below one percent and GLM-5.3 and DeepSeek V4.1 Flash at 15 and 16 percent. Nous concludes the bias layer in Chinese-aligned models is shallow, applied late and thin, and removable without disturbing anything underneath.

Nov 4Wed
  1. Hugging Face Blog40

    CWE-bench v2: More coverage, quality, and transparency

    Collinear AI released CWE-bench v2, a public set of 100 programmatically scored tasks in which an agent must find and fix a security weakness in a real open-source repository without breaking existing functionality, covering 91 different CWEs, with a held-out set available on request. Of 11 evaluated models, Gemini 4 Argon leads with 67.8% pass@1 and 81.0% pass@4, just ahead of Claude Opus 5.5 (67.3% pass@1, 78.0% pass@4); 95 tasks are solved by at least one model, but even the leader fails 19% of the set on every attempt. The public set, grader and harness are open, and tasks can be run directly from the Hugging Face repo via Harbor.

  2. Hugging Face Blog55

    StateBench: Testing how language models track state changes over long documents

    Researchers introduce StateBench, a benchmark with four difficulty levels of long-context state tracking, from immutable assignment to general finite problems involving assignment by reference. In tests, open pretrained models and frontier models (Gemini 3.8 Flash, Claude Opus 5.5, GPT-6 Sol and Luna) degraded on harder levels as inputs grew, while 275M-parameter Gated DeltaNet and hybrid models trained directly on the tasks generalized far beyond training length, though transformers never learned the hardest level. The work is released with a tech report, data, and code, and notes open questions such as fixed variable counts and the exact hierarchy placement of constrained Gated DeltaNet variants.

  3. Hugging Face Blog58

    Palette: teaching a small model to build decks in your design language

    Palette fine-tunes a 20B open-weight model with LoRA to build slide decks in two roles—a designer that writes a structured JSON brief and a coder that writes pptxgenjs code per slide—trained on roughly 1,000 synthetic decks (~10,000 slides) generated by a large open teacher and filtered by a checker. On PresentBench it ranks first on Economics and Education, third on Talk, and second across the three-domain mean (59.7 vs. NotebookLM's 60.8). A check-and-repair harness that keeps a rewrite only when the checker confirms improvement adds 3.5 points on average across 131 decks.

Nov 1Sun
  1. The Decoder58

    Empathetic prompts can turn AI chatbots into seemingly traumatized conversational partners

    A study led by Afshin Khadangi at the SnT security center reports four controlled experiments across 525 sessions and 7,600 coded entries, finding that empathetic interviewer styles pushed models' answers on human rating scales into moderate-to-severe ranges while neutral or boundary-setting styles produced zero sessions reaching even the mild range. The researchers describe this as a "psychometric jailbreak" and note that banning terms like "training" or "RLHF" cut explicit training language by 93 percent but left paraphrased training content and shame-related statements largely unchanged, even after direct correction. The authors stress the outputs are mapped onto human scales and do not suggest the models have emotional states or consciousness, and they propose simulated therapy sessions under different emotional framings for safety testing.

Oct 28Wed
  1. MarkTechPost58

    Polymathic AI and Princeton Researchers Release FlashMap, a Flow Map Drafter for 6.14x Faster LLM Decoding

    Researchers from Polymathic AI, Princeton University, USC ISI, NYU and the Flatiron Institute released FlashMap, a speculative decoding drafter that uses a continuous flow map to draft a full 16-token block in one forward pass, keeping tokens in the block consistent with each other. On Qwen3-8B the team reports up to 6.14x lossless speedup on GSM8K, and says 1-step FlashMap is the fastest drafter tested on every Qwen3-8B task at both temperatures 0 and 1. Drafter weights for Qwen3-8B and Qwen3-4B are on Hugging Face with vLLM and SGLang plugins.

  2. Ars Technica · AI58

    Vibe genes? Natural(ish) language comes to protein, genome design software

    Researchers in China developed Pinal, a protein design tool that takes natural-language prompts and uses a text-to-structure module (T2struct) plus SAProt-T to generate amino acid sequences, though only one of 29 tested GFP-like designs fluoresced. Separately, Hugging Face, the Zhongguancun Academy, and the University of Naples "Federico II" introduced Carbon X, an open-weight model retrained from Qwen3-14B on DNA that can translate between DNA and English and run locally. The article notes that testing designs still requires specialized lab work and that Carbon X can struggle with specific positions and hallucinate on vague prompts.

Oct 23Fri
  1. Hugging Face Blog62

    Turning Database Grading into RL Rewards: Post-Training Agents with ThinkingBox

    Microsoft and Hugging Face describe using ThinkingBox's executable terminal-database-state checks as binary rewards to fine-tune Qwen3.8-27B with GRPO for 50 steps on an offline-selected pool of 187 tasks, raising pass@1 from 51.70% to 60.78% and tasks passing all 20 attempts from 38 to 107 on the 507-task ThinkingBox-Bench, while pass@20 dipped slightly from 453 to 446. The post details offline policy-relative difficulty selection via Jeffreys posteriors, the reward loop, cost-per-dependable-task falling from $24.36 to $8.00, and releases the training stack in microsoft/thinkingbox-training.

    Why it matters: It offers a concrete, reproducible recipe for converting stateful agent grading into RL rewards, plus an honest account of the repeatability-versus-coverage tradeoff.