Thinking Machines' Inkling 2 Small drops 316 F-bombs across benchmark runs
Thinking Machines' Inkling 2 Small drops 316 F-bombs across benchmark runs
Researchers at Vals AI reviewed the thinking logs of Thinking Machines' Inkling 2 Small from 2,655 runs across 13 benchmarks and found "fuck" or variations in 50 runs, totaling 316 instances, with 241 coming from just two runs that spiraled out of control. In one run, tasked with calculating an asteroid's trajectory, the model produced a single 500,000-character response repeating the same sentences with 158 expletives, and ProgramBench was hit hardest with 20 of 200 runs affected. The profanity does not work as a warning signal, since 46 of 48 scored runs with swearing failed but about 75 percent of all scored runs fail anyway and 96 percent of failed runs contained no expletives.
Source: The Decoder · the-decoder.com