Disagreement among frontier LLMs on real-world fact-checks

A study of 1,000 real-world user claims across five frontier LLMs reveals a 67% disagreement rate, with 34% of cases showing substantive gaps of two or more buckets on a four-point ordinal scale. Inter-rater reliability reached a Krippendorff’s α of 0.639, highlighting limited consistency even among top-tier parametric and retrieval-augmented models. Disagreement is most pronounced on nuanced labels, as the panel rarely converges on "Mostly True" or "Misleading" compared to definitive "True" or "False" verdicts.

Read more

Claude Opus 4.8

Anthropic has launched Claude Opus 4.8, featuring improved benchmarks and agentic reliability with a 4x reduction in unremarked code flaws compared to its predecessor. Key technical updates include "dynamic workflows" for parallel subagent execution in Claude Code, user-selectable "effort control" for reasoning depth, and Messages API support for mid-task system entries to preserve prompt caching. While standard pricing remains unchanged, fast mode is now 3x cheaper, and the model demonstrates enhanced alignment and prosocial traits.

Read more

Can we have the day off?

The author argues that the projected 10x productivity gains from AI and autonomous agents should be leveraged to implement a four-day work week. By utilizing LLMs to automate workflows and handle task execution, the workforce could maintain current output levels while gaining personal time to address socio-economic challenges.

Read more

Continue? Y/N: A 60-second game about AI agent permission fatigue

Claude Code presents a time-pressured scenario requiring rapid approval of refactoring commands to illustrate the security risks of AI agent permissions. The simulation highlights the danger of human-in-the-loop failures when users are incentivized to bypass careful review of automated actions.

Read more

Various LLM Smells

The author identifies "AI-smell," a set of recognizable artifacts emerging from LLM-assisted writing and web design. In text, these patterns include overused punchlines, specific rhetorical structures like "X is the Y of Z," and repetitive short sentences. In UI/UX, AI-generated sites frequently share identical design elements such as JetBrains Mono fonts, specific button styles, and blinking-dot badges, highlighting a lack of human sincerity and whimsy.

Read more

SIA: Self Improving AI with Harness and Weight Updates

SIA (Self-Improving Agent) addresses the human bottleneck in AI development by unifying the previously disjoint "harness-update" and "test-time training" paradigms. A Feedback-Agent iteratively optimizes both the agentic scaffold and the model weights of a task-specific agent, outperforming prior SOTA across legal classification, GPU kernel optimization, and RNA denoising. This dual-lever approach demonstrates that harness updates enhance agentic search and logic, while weight updates instill domain-specific intuition unattainable through prompting or scaffolding alone.

ArXiv

Harness Sensitivity Is Non-Monotone Across LLM Agent Tiers

The study refutes the assumed inverse relationship between LLM capability and optimal harness complexity using the HEAT-24 benchmark. Findings reveal a "harness-complexity paradox" where Gemini 2.5 Flash performance degrades with increased verbosity, while reasoning models like Qwen3.5-122B excel under strict harnesses. Failure analysis shows that format violations dominate frontier model errors while low-capability models struggle with workspace navigation, suggesting that harness selection must be tailored to specific model types rather than just capability tiers.

ArXiv

Omissive Bias: Benchmarking LLM Answers to Ethical Decision-Making

The paper introduces "omissive bias" to describe the systematic absence of religious perspectives in LLM responses to ethical and personal queries. Using the AllFaith Religious Representation Benchmark, researchers evaluated 27 models and found they consistently underrepresent religious frameworks relative to human expectations. This bias is most pronounced in practical scenarios like grief and family conflict, where models fail to reflect the religious reasoning many users rely on in real-world contexts.

ArXiv

Learning from Ava:Lessons from Trustworthy AI for Policy and Dev Research

AVA is a multi-agent GenAI platform leveraging a curated library of 4,000+ World Bank Reports to provide evidence-based syntheses for development experts. The system operationalizes epistemic humility through page-anchored citation verifiability and reasoned abstention for unsupported queries. Evaluation across 2,200+ users demonstrated weekly time savings of up to 3.9 hours and high trust calibrated via institutional provenance.

ArXiv

DeltaBox: Scaling Stateful AI Agents with Ms-Level Sandbox Checkpoint/Rollback

DeltaBox optimizes LLM agent state exploration by replacing full sandbox duplication with DeltaState, an OS-level abstraction for change-based checkpoint and rollback (C/R). By implementing DeltaFS for layered filesystem management and DeltaCR for incremental process state dumps, it reduces C/R latency to milliseconds (14ms/5ms). This enables significantly higher-frequency tree search and RL exploration within fixed time budgets compared to traditional duplication methods.

ArXiv

Local Coding Agent with LLMs to Delegate Tool Calls to Small AI Models

Open Agent Tools (oats) enables self-hosted AI models to execute local source code for tool-calling agentic workloads. It achieves this by data mining GitHub repos to create a fast, compressed prompt index of Python code, allowing agents to refer to existing local functions. This method reduces LLM token usage by delegating tool-calling to smaller, open-source AI models, supporting a vast number of local tools.

Repo

Teleport-env – <500ms stateful rollbacks for AI agents via CRIU

Teleport-Env is a high-performance sandbox for autonomous coding agents, MCTS, and RL that enables sub-500ms environment recovery. It utilizes a "Cold Layer Switch" architecture, combining overlayfs for filesystem snapshots and CRIU for process memory restoration to bypass the latency of standard Docker restarts. This allows for high-throughput testing of destructive commands within agentic loops by instantly reverting the system to a precise checkpoint.

Repo

Why LLM decode is memory-bound, not compute-bound

LLM Inference at Scale is a practitioner's handbook designed to address the unique challenges of serving LLMs in production, such as KV cache management and the memory bandwidth wall. It covers essential optimization techniques like PagedAttention, continuous batching, and speculative decoding while providing deep dives into engines like vLLM, SGLang, and TensorRT-LLM. The guide also explores advanced topics including quantization, tensor parallelism, and disaggregated serving to optimize for throughput, latency, and cost.

Repo

SIA: The Open Source Self Improving AI

SIA (Self-Improving AI) is a framework that autonomously optimizes task-specific agents through an iterative loop of harness and weight updates. The architecture coordinates three specialized agents: a Meta-Agent for initialization, a Target Agent for task execution, and a Feedback Agent for performance-driven refinement. SIA achieves SOTA results across diverse benchmarks, including a #1 ranking on MLE-Bench, a 14x speedup in Triton kernel optimization, and significant gains on LawBench and scRNA denoising.

Repo

DiscloAI – open-source EU AI Act Article 50 compliance SDK

DiscloAI SDK is a developer tool designed for rapid compliance with EU AI Act Article 50, offering integration via CDN or npm packages for vanilla JS, React, and Next.js. It enables developers to meet the August 2026 legal deadline in under 10 minutes through a lightweight, MIT-licensed implementation.

Repo