IBM at NeurIPS 2025

Analog in-memory computing (AIMC) is a promising compute paradigm to improve speed and power efficiency of neural network inference beyond the limits of conventional von Neumann-based architectures. However, AIMC introduces fundamental challenges such as noisy computations and strict constraints on input and output quantization. Because of these constraints and imprecisions, off-the-shelf LLMs are not able to achieve 4-bit-level performance when deployed on AIMC-based hardware. While researchers previously investigated recovering this accuracy gap on small, mostly vision-based models, a generic method applicable to LLMs pre-trained on trillions of tokens does not yet exist. In this work, we introduce a general and scalable method to robustly adapt LLMs for execution on noisy, low-precision analog hardware. Our approach enables state-of-the-art models — including Phi-3-mini-4k-instruct and Llama-3.2-1B-Instruct — to retain performance comparable to 4-bit weight, 8-bit activation baselines, despite the presence of analog noise and quantization constraints. Additionally, we show that as a byproduct of our training methodology, analog foundation models can be quantized for inference on low-precision digital hardware. Finally, we show that our models also benefit from test-time compute scaling, showing better scaling behavior than models trained with 4-bit weight and 8-bit static input quantization. Our work bridges the gap between high-capacity LLMs and efficient analog hardware, offering a path toward energy-efficient foundation models. Code is available at github.com/IBM/analog-foundation-models.

Visit us at the IBM booth in the exhibit hall to talk to our researchers and recruiters. We'll also be doing demos of our work. <Booth demo & staff schedule coming soon>

Mellea is a generative AI library for writing robust application software. LLM outputs are intrinsically unpredictable and often wrong. Agentic frameworks and prompt optimization libraries address this unpredictability by putting the LLM in control leading to systems that are often difficult to debug, maintain, evolve, and port. Mellea puts developers back in control by defining a programming model that encourages task decomposition, information hiding, and compositional contracts between isolated modules. Mellea’s programming model results in systems with manageable fault models, better portability, and lower inference-time costs.

https://neurips.cc/virtual/2025/loc/san-diego/talk/127761

Visit us at the IBM booth in the exhibit hall to talk to our researchers and recruiters. We'll also be doing demos of our work. <Booth demo & staff schedule coming soon>

Visit us at the IBM booth in the exhibit hall to talk to our researchers and recruiters. We'll also be doing demos of our work. <Booth demo & staff schedule coming soon>

Visit us at the IBM booth in the exhibit hall to talk to our researchers and recruiters. We'll also be doing demos of our work. <Booth demo & staff schedule coming soon>