Moondream 3 Preview: Frontier-level reasoning at a blazing speed

Moondream 3 is a new architecture of a 9B MoE model with 2B active parameters, designed to achieve frontier-level visual reasoning while maintaining fast and efficient inference. The model focuses on four key areas: visual reasoning, trainability, speed, and inexpensiveness, and has shown impressive results in object detection, pointing, structured output, and OCR, with the ability to understand and produce more complex queries and answers due to its increased context length of 32k.

Read more

Modular Manifolds

When training large neural networks, it's essential to keep the tensors healthy by normalizing them to prevent issues with numerical underflow and overflow, and to make the optimization process more predictable. Normalizing weight matrices, in particular, can be beneficial as it provides certainty about their size, removes the problem of exploding weight norms, and allows for more predictable behavior, but it is less commonly applied compared to normalizing activation vectors and gradient updates.

Read more

Suno Studio, a Generative AI DAW

Suno Studio is a comprehensive creative workspace where users can generate stems, edit audio, and create music with endless possibilities, from spark to song. The platform allows users to upload any audio, generate infinite stem variations, edit in a multitrack timeline, and export stems as audio and MIDI to continue editing in their digital audio workstation (DAW).

Read more

How to stop AI's "lethal trifecta"

Large language models, a type of artificial intelligence, have a security problem as they cannot separate code from data, making them vulnerable to attacks called prompt injections. To address this issue, coders need to start thinking like mechanical engineers, suggesting a need for a more robust and secure approach to building AI systems.

Read more

The great sameness: a comic on how AI makes us more alike

The provided text does not describe a comic about AI making us more alike, but rather appears to be a lengthy description of the cookies used on the website It's Nice That, including their purposes, maximum storage durations, and types. The website utilizes various cookies from different providers, such as Cloudflare, Google, and LinkedIn, to enable basic functions, personalize content, and measure advertising effectiveness.

Read more

Bit is all we need: binary normalized neural networks

Researchers have developed a new type of neural network layer, called binary normalized layers, which use only single-bit parameters, allowing models to use 32 times less memory while maintaining equivalent performance. These binary layers can be easily implemented on current computers and enable the deployment of large neural network models on simple and cheap hardware, such as mobile devices or CPUs, without requiring dedicated electronic hardware.

ArXiv

The Memory Paradox: Why Our Brains Need Knowledge in an Age of AI

As humans increasingly rely on AI systems and digital tools, their internal memory systems may weaken, impairing expertise, critical thinking, and long-term retention. The paper discusses how over-reliance on AI can hinder the development of robust neural encoding and intuitive mastery, emphasizing the need for strong internal models to effectively interact with and evaluate AI output.

ArXiv

LLM probabilities cannot distinguish between possible and impossible language

Researchers tested four Large Language Models to see if they could distinguish between grammatical and ungrammatical sentences, but the results showed that the models' probability assignments did not reliably indicate a difference between ungrammatical sentences and those that were semantically or pragmatically odd. This suggests that the models' probabilities are not a reliable indicator of their syntactic knowledge, and therefore, claims about their ability to distinguish possible from impossible language need to be verified through alternative methods.

ArXiv

Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Driving

Researchers have been applying large language models (LLMs) to autonomous driving motion generation due to similarities between the two domains, but a systematic understanding of which LLM components can be transferred is lacking. This paper evaluates the transferability of five key LLM modules to autonomous driving motion generation, demonstrating their potential to improve performance when adapted, and identifying which techniques are most effective and why others may fail.

ArXiv

Understanding RL for model training, and future directions with GRAPE

This paper provides a detailed, step-by-step explanation of key algorithms for instruction tuning of models, aiming to eliminate ambiguity and provide a clear understanding of the concepts. The paper covers several algorithms, including SFT, REINFORCE, and PPO, and also presents new ideas for research, including a literature review and the introduction of GRAPE, a new approach for policy evolution.

ArXiv

Automated Repair of Ambiguous Problem Descriptions for LLM-Based Code Generation

SpecFix is a novel approach that automatically repairs ambiguity in programming problem descriptions, minimizing modifications to reduce code generation uncertainty and better align natural language with input-output examples. An evaluation of SpecFix with four state-of-the-art language models on three popular code generation benchmarks shows that it significantly increases the accuracy of generated code, with its repairs generalizing across models.

Repo

Show HN: I ditched restful API conventions and built this

Nile is a TypeScript-first, service-oriented backend framework designed for building modern, AI-ready backends with a simple developer experience and speed. It offers a unique service-and-action model, multi-protocol support, and built-in agentic/AI workflows, making it different from traditional MVC or REST-based frameworks.

Repo

Verifiers: Environments for LLM Reinforcement Learning

Verifiers is a library of modular components for creating RL environments and training LLM agents, allowing users to build and train environments with a variety of tools and frameworks. The library includes features such as an async GRPO implementation, support for large-scale FSDP training, and easy integration with any RL framework that exposes an OpenAI-compatible inference client.

Repo

A proxy to use HTTP/SSE MCPs from STDIO clients

The mcp-proxy is a standalone binary that connects STDIO-based MCP clients to HTTP (SSE) based MCP servers, supporting both SSE and streamable HTTP specifications. It can be installed on macOS, Linux, and Windows by downloading the latest release or building from scratch using Rust, and its usage involves configuring the client with the proxy's command and arguments or environment variable.

Repo

SimpleFold: Folding proteins is simpler than you think

Researchers have introduced SimpleFold, a protein folding model that uses general purpose transformer layers and achieves competitive performance compared to state-of-the-art baselines, challenging the reliance on complex domain-specific architectures in protein structure prediction. SimpleFold is trained on a large dataset of over 8.6 million protein structures and can be used for protein structure prediction, ensemble prediction, and other tasks, with code and models made available for use and further development.

Repo