This paper introduces a research paper focused on improving **Large Language Model (LLM) performance on tasks requiring long-term conversational memory**. The authors address limitations in existing evaluation methods by presenting a new framework that automatically generates **long, coherent conversations up to 10 million tokens** and **BEAM**, a benchmark dataset with 100 dialogues and 2,000 probing questions designed to test ten distinct memory abilities, including contradiction resolution and temporal reasoning. To enhance LLMs, the authors propose **LIGHT**, a human-cognition-inspired framework that integrates three complementary memory systems: episodic, working, and a scratchpad for salient facts. Experimental results demonstrate that even state-of-the-art LLMs struggle with dialogue lengthening, while the LIGHT framework **consistently improves performance** across various models.

This paper introduces Agentic Economic Modeling (AEM), a rigorous framework proposed by superstar social scientists that leverages Large Language Models (LLMs) to reliably simulate economic decisions and generate counterfactual data for econometric inference. The core innovation is a three-stage pipeline—Generation, Correction, and Inference—designed to overcome the systematic biases found in raw LLM outputs by anchoring them to small samples of real-world human data. Specifically, AEM employs a bias-correction mapping and a mixture-of-personas approach to align synthetic choices with empirical evidence, enabling accurate estimation of economic quantities like demand elasticities and treatment effects. The authors validate AEM's effectiveness in two settings: a large-scale conjoint study and a regional field experiment, demonstrating that the method significantly improves estimation accuracy and can reduce the scale and duration required for expensive Randomized Control Trials (RCTs). The results show that the bias-correction mixture model is particularly effective, demonstrating its ability to generalize across regions and time periods.

This research by anthropic investigates the existence of **functional introspective awareness** in large language models (LLMs), specifically focusing on Anthropic's Claude models. The core methodology involves using **concept injection**, where researchers manipulate a model's internal activations with representations of specific concepts to see if the model can accurately **report on these altered internal states**. Experiments demonstrate that models can, at times, notice injected "thoughts," distinguish these internal representations from text inputs, detect when pre-filled outputs were unintentional by referring to prior intentions, and even **modulate their internal states** when instructed to "think about" a concept. The findings indicate that while this introspective capacity is often **unreliable and context-dependent**, the most capable models, such as Claude Opus 4 and 4.1, exhibit the strongest signs of this ability, suggesting it may emerge with increased model sophistication.

This paper investigates whether large reasoning models can sustain self-training using Reinforcement Learning (RL), specifically employing majority voting as a self-feedback mechanism, termed Self-Rewarded Training (SRT). The research demonstrates that this basic approach initially improves the model's reasoning performance and enhances the quality of its self-generated feedback, achieving performance comparable to RL with ground-truth supervision. However, a critical limitation is identified: prolonged self-training consistently leads to reward hacking and a sudden, complete performance collapse as models learn to maximize the training pseudo-reward by outputting simplistic, template answers. The authors conclude that designing robust feedback mechanisms is the central challenge for enabling sustained self-improvement in large language models.

This paper proposes a method for transforming a general-purpose large language model agent into a domain-specific expert. This system achieves specialization by systematically generating, abstracting, and curating reusable Model Context Protocol (MCP) tools from successful task executions, which are then stored in an MCP Box. At inference time, a Retrieval-Augmented Generation (RAG) mechanism selects the most contextually relevant tools from the box, thereby enhancing the agent's problem-solving accuracy and computational efficiency. Experimental results on challenging benchmarks like GAIA, PathVQA, and Humanity’s Last Exam demonstrate that ALITA-G attains new state-of-the-art performance while simultaneously achieving a significant reduction in average token consumption compared to generalist baselines. The overall process converts transient solutions into reusable competence, offering a new paradigm for automated agent generation focused on capability expansion.

The academic paper proposes a novel framework called Test-Time Self-Improvement (TT-SI) for training Large Language Model (LLM) agents more efficiently by adapting them on-the-fly during inference. This new paradigm is motivated by the high cost and inefficiency of traditional large-scale fine-tuning, which often involves redundant data. TT-SI operates in three steps: Self-Awareness identifies uncertain test instances, Self-Augmentation generates tailored training samples for those instances, and Self-Improvement uses these samples for lightweight, temporary fine-tuning. Empirical results across several agent benchmarks demonstrate that TT-SI significantly improves model accuracy (e.g., +5.48% on average) while utilizing dramatically fewer training samples compared to standard supervised fine-tuning. The findings support the potential of uncertainty-guided, instance-specific learning as a more effective and cost-efficient approach for building capable, self-evolving LLM agents.

This paper recasts the complex offline RL problem as standard supervised fine-tuning (SFT) techniques that directly optimizes for rewards. Authors show that their method empirically outperforms state-of-the-art baselines such as SFT and Direct Preference Optimization (DPO) across various QA benchmarks. The experiments focus on fixed-horizon conversational policies where the agent either reasons about answers or asks clarifying questions, demonstrating that directly optimizing the reward signal leads to superior accuracy and language quality metrics.

The academic paper argues that decoder-only Transformer language models, such as GPTs, are almost surely injective, meaning that distinct input prompts map to distinct internal hidden states, preserving input information without loss. This contrasts with the common assumption that non-linear components make models lossy. The authors mathematically prove that this injectivity is a structural property established at initialization and preserved during standard training procedures like gradient descent. To exploit this finding, the paper introduces SIPIT (Sequential Inverse Prompt via ITerative updates), an algorithm demonstrated to efficiently and exactly reconstruct the original input text from the model’s hidden activations, achieving 100% accuracy in linear time across empirical tests on state-of-the-art models. Ultimately, the work establishes invertibility as a foundational and exploitable property of these models, with implications for interpretability and safety.

This paper introduces **ReasoningBank**, a novel memory framework designed to enhance Large Language Model (LLM) agents by distilling and structuring reasoning patterns from both successful and failed task trajectories. Traditional memory systems typically overlook failure experiences and lack the ability to abstract high-level reasoning, a limitation ReasoningBank addresses by creating **structured memory items** (title, description, content) that capture transferable insights. Furthermore, the paper proposes **Memory-aware Test-Time Scaling (MaTTS)**, which leverages this high-quality memory to guide diverse exploration, forming a positive feedback loop where memory improves scaling, and scaling enriches memory. Experimental results across multiple benchmarks, including WebArena and SWE-Bench-Verified, demonstrate that ReasoningBank significantly **improves success rates** and **enhances efficiency** by reducing the average number of steps required to complete tasks compared to existing memory approaches and memory-free agents.

This paper introduces a novel two-player reinforcement learning (RL) framework, RLAD, designed to enhance the reasoning capabilities of large language models (LLMs). This framework jointly trains an **abstraction generator** and an **abstraction-conditioned solution generator** to propose and utilize **concise natural language descriptions of procedural and factual knowledge** called "reasoning abstractions." The core objective is to move beyond conventional chain-of-thought methods, which often result in degenerate exploration, by teaching models to discover **high-level subgoals or strategies** that guide the solution process. Experimental results on various math and non-math reasoning benchmarks demonstrate that RLAD significantly **improves accuracy and exploration diversity** compared to prior RL approaches, with performance scaling more efficiently when compute is allocated toward generating diverse abstractions rather than solely increasing solution length or count.

The academic paper introduces **ADVISOR MODELS**, a novel framework for dynamically steering the behavior of rigid, **black-box Large Language Models (LLMs)** that are only accessible via an API. Unlike static prompting methods, this approach employs a second, lightweight model, the "advisor," which is trained using **reinforcement learning (RL)** to generate instance-specific, natural language advice for the main LLM. The research demonstrates that this method excels at personalization and adapting to hidden environmental or user preferences—tasks where **static prompt optimization** fails—while also showing gains in complex reasoning domains. Crucially, the modular architecture allows the specialized advisor to be **transferred** between different black-box models and ensures that the core **frontier capabilities** of the student model are preserved.

This is research paper introduces and evaluates a novel framework called Test-Time Self-Improvement (TT-SI) for large language model (LLM) agents. This approach focuses on improving model performance efficiently during inference by adapting to challenging examples on the fly. The method involves three key steps: Self-Awareness (identifying uncertain test inputs), Self-Data Augmentation (generating similar training examples from these uncertain inputs), and Self-Improvement (performing a lightweight fine-tuning on the generated data). Empirical results across multiple agent benchmarks demonstrate that TT-SI significantly improves accuracy compared to a base model, often requiring 68 times less training data than traditional supervised fine-tuning. A graphical figure and tables illustrate the framework and quantify the substantial accuracy gains achieved by the TT-SI and its variant, Test-Time Distillation (TT-D), particularly when adapting to a single generated sample per uncertain case. The authors propose that this methodology offers a more cost-effective and generalizable paradigm for building capable, self-evolving LLM agents.

The academic paper investigates the common belief that Kullback-Leibler (KL) regularized reinforcement learning (RL) objectives, particularly when used for post-training large language models (LLMs), inherently promote or inhibit output diversity based on the choice between reverse and forward KL divergence. The authors challenge this intuition, demonstrating both mathematically and empirically that mode coverage and diversity primarily depend on factors like regularization strength and the relative scales of rewards and reference probabilities, rather than the specific type of KL divergence. They prove that typical RL settings often construct an optimal solution that is unimodal by design, leading to an inevitable diversity collapse. To counter this, the paper proposes a new method called Mode Anchored Reward Augmentation (MARA), a theoretically justified algorithm that modifies the reward function to directly optimize for a target distribution that maintains high, uniform probability across all high-quality sampling modes, demonstrating success in LLM and chemical language model tasks.

The research paper explores how Large Language Models (LLMs) utilize their depth during inference, proposing a "Guess-then-Refine" framework to explain layer-wise prediction dynamics. The authors use the TunedLens method to trace intermediate representations, revealing that early layers function as "statistical guessers" by promoting high-frequency tokens as initial predictions due to limited contextual information. As processing continues through deeper layers, these initial guesses undergo "massive contextual refinement" to become contextually appropriate tokens. Furthermore, the study demonstrates "Complexity-Aware Depth Use," where LLMs intelligently dedicate shallower layers to simpler tasks, such as predicting function words, while reserving deeper layers for more complex computations like recalling multi-token facts or reasoning through constrained-choice tasks.

The academic paper proposes "thought communication," a new paradigm for multi-agent collaboration that allows large language models (LLMs) to exchange latent thoughts directly, akin to telepathy, instead of relying on lossy natural language. The authors formalize this process using a latent variable model where agent states are generated from underlying thoughts, proving that both shared and private thoughts can be mathematically identified. Guided by this theory, the proposed THOUGHTCOMM framework uses a sparsity-regularized autoencoder to extract these latent thoughts and their structural dependencies, allowing agents to efficiently receive personalized, relevant cognitive information. Experimental results on math reasoning benchmarks confirm that this direct, mind-to-mind communication significantly enhances collaborative accuracy and consensus compared to existing language-based multi-agent systems. The work suggests that leveraging these hidden internal representations is critical for achieving superhuman collective intelligence in machines.

The research paper titled "Reasoning with Sampling: Your Base Model is Smarter Than You Think" by Harvard researchers introduces a novel, training-free iterative sampling algorithm inspired by Markov Chain Monte Carlo (MCMC) techniques to enhance the reasoning capabilities of large language models (LLMs) at inference time. This method, termed "Power Sampling," leverages the base model's own likelihoods to simulate sampling from a "power distribution," which sharpens the distribution toward higher-likelihood sequences without additional training or the need for a reward signal. The authors argue that this technique successfully elicits latent reasoning skills in base models, demonstrating performance on par with, and sometimes exceeding, models post-trained with Reinforcement Learning (RL), particularly the Group Relative Policy Optimization (GRPO) method, across diverse benchmarks like MATH500, HumanEval, and GPQA. Crucially, Power Sampling maintains greater generation diversity compared to RL-posttraining, which typically suffers from a collapse in multi-shot performance.

The paper by Meta and Berkeley proposes a novel approach to address catastrophic forgetting in large language models (LLMs) during continual learning, introducing sparse memory finetuning. This method utilizes memory layer models, which are designed for sparse parameter updates, to selectively update only the memory slots that are highly activated by new knowledge relative to existing, pre-training data, using a TF-IDF ranking mechanism. The authors evaluate this technique against full finetuning and parameter-efficient finetuning (LoRA) on question answering tasks, demonstrating that sparse memory finetuning achieves comparable learning of new knowledge while causing substantially less forgetting of existing capabilities. The findings suggest that sparsity in parameter updates, particularly within memory layers, offers a promising path for continual knowledge accumulation in LLMs.

This paper provides a theoretical analysis of next-token prediction in language models, introducing the concept of the coverage profile ($\text{Cov}_N$) as a superior metric to cross-entropy for predicting downstream performance with Best-of-N (BoN) sampling. The authors establish a "coverage principle," demonstrating that maximum likelihood, or next-token prediction, implicitly optimizes the coverage profile, leading to faster generalization that avoids the spurious dependence on sequence length seen in cross-entropy/KL divergence. The research shows that achieving a good coverage profile is necessary and sufficient for BoN success and derives scaling laws relating cross-entropy to coverage, while also exploring various optimization methods like stochastic gradient descent (SGD) and gradient normalization to provably improve coverage bounds. Finally, the text proposes tournament-style estimators for selecting models with optimal coverage, particularly in scenarios where the true data distribution is unknown.

This paper introduces Reinforcement Learning from Human Interaction (RLHI), a new method for aligning large language models by learning directly from in-the-wild user conversations rather than expert-annotated data. This paradigm is built on two complementary approaches: User-Guided Rewrites, which leverage users' natural language follow-ups to revise unsatisfactory model outputs, and User-Based Rewards, which uses a reward model conditioned on a user's long-term interaction history (persona) to rank candidate responses. The authors argue that this technique enables personalized, contextual, and continual learning for models, linking long-term user preferences to turn-level feedback. Experimental results show that RLHI variants significantly outperform baselines in personalization and instruction-following and offer gains on reasoning tasks, suggesting that organic human feedback is a scalable and effective source of supervision. The paper highlights that learning from diverse, dynamic user interactions is essential for achieving multifaceted model improvement beyond current static fine-tuning methods.