This research paper examines the impact of an artificial intelligence tool for materials discovery on the productivity and performance of scientists working in a large U.S. firm's R&D lab. The study exploits a randomized rollout of the AI tool across teams of scientists, allowing the researchers to draw causal inferences about the effects of the technology. The paper demonstrates that the AI tool significantly increases the rate of materials discovery, patent filings, and product innovation, but these benefits are unequally distributed among scientists. The researchers find that the AI tool is most beneficial to scientists with strong judgment skills, which involve the ability to evaluate and prioritize AI-generated candidate compounds. The study also reveals that the AI tool automates a significant portion of idea generation tasks, resulting in a reallocation of scientist labor towards judgment tasks. This reallocation, along with the increased demand for judgment skills, explains the heterogeneous impact of the AI tool on scientific performance.

This research paper investigates how large language models (LLMs) can improve their ability to reason over long contexts. The authors propose a self-improvement method called SEALONG that involves sampling multiple reasoning outputs from an LLM, scoring these outputs using Minimum Bayes Risk (MBR), and then fine-tuning the model using the highest-scoring outputs or by contrasting high-scoring and low-scoring outputs for preference optimization. Extensive experiments on several leading LLMs demonstrate that SEALONG effectively improves the long-context reasoning capabilities of LLMs without relying on human annotations or advanced models. The paper further analyzes the impact of various prompting strategies, scoring methods, and training parameters on SEALONG's performance.

This paper is a survey of personalized large language models (LLMs), outlining different ways to adapt these models for user-specific needs. It analyzes how to personalize LLMs based on various user-specific data such as static attributes, interaction history, and pair-wise human preferences. The authors propose taxonomies for personalization granularity (user-level, persona-level, and global preference), techniques (RAG, prompting, representation learning, and RLHF), evaluation metrics (intrinsic and extrinsic), and datasets (with and without ground-truth text). The paper concludes by highlighting key challenges for the future of personalized LLMs, including the cold-start problem, stereotype and bias issues, privacy concerns, and the complexities of multimodality.

This research paper investigates the numerical understanding and processing abilities (NUPA) of large language models (LLMs). The authors introduce a benchmark, covering various numerical representations and tasks, to systematically evaluate LLMs' capabilities in handling numbers. The paper finds that while LLMs perform well on simpler tasks, their performance deteriorates significantly as task complexity and input length increase. The authors also explore various techniques to improve NUPA, including specialized tokenizers, positional encodings, and data formats. Despite some successes in improving NUPA during pre-training, these techniques are found to be ineffective when applied to already trained models. The paper concludes that further research is necessary to address the challenges of NUPA in LLMs and enable them to confidently handle numerical tasks in real-world applications.

This paper describes a novel method called Multi-expert Prompting that aims to improve the reliability, safety, and usefulness of large language models (LLMs). The method simulates multiple experts with different areas of expertise and aggregates their responses to a query, ultimately selecting the best answer based on criteria like truthfulness, factuality, and informativeness. This process is inspired by the Nominal Group Technique, a human-designed decision-making framework. The authors demonstrate that Multi-expert Prompting significantly outperforms existing prompting methods, especially in scenarios where diverse perspectives are valuable, and surpasses prior methods on various benchmarks. The paper also discusses ethical considerations related to the potential for bias amplification and explores ways to mitigate these risks.

The provided text describes a novel approach to in-context learning (ICL) called Mixtures of In-Context Learners (MOICL) that addresses key limitations of traditional ICL, such as context length constraints and sensitivity to noisy or out-of-distribution demonstrations. MOICL partitions a set of demonstrations into subsets, trains each subset as an "expert," and learns a weighting function to combine their predictions. The authors demonstrate that MOICL outperforms traditional ICL and other baselines in classification tasks across various datasets, achieving higher accuracy while being more robust to noisy data and label imbalance. They also show that MOICL is more data and computationally efficient, making it a promising approach for improving the effectiveness of ICL.

This technical report describes "Project Sid," an experiment that aims to create and study AI civilizations within a Minecraft environment. The researchers introduce a new cognitive architecture called PIANO, designed to enable agents to interact with each other and their environment in real-time while maintaining coherence across multiple output streams. They show that agents using PIANO can make significant individual progress by acquiring Minecraft items and that they can form meaningful relationships in large groups, demonstrating social understanding. Additionally, they explore the concept of civilizational progress through benchmarks that measure agent specialization into distinct professions, adherence to collective rules, and cultural transmission through memes and religion. The report concludes by discussing limitations of the current system and outlining areas for future research.

This research paper investigates the structure of the concept universe represented by large language models (LLMs), specifically focusing on how sparse autoencoders (SAEs) can be used to discover and analyze concepts within these models. The authors explore this structure at three distinct scales: the “atomic” scale, where they look for geometric patterns representing semantic relationships between concepts; the “brain” scale, where they identify clusters of features that tend to fire together within a document and are spatially localized; and the "galaxy" scale, where they examine the overall shape and clustering of the feature space. The authors find that the concept universe exhibits a surprising degree of structure, suggesting that SAEs can be a powerful tool for understanding the inner workings of LLMs.

This research paper investigates the phenomenon of hallucinations in large language models (LLMs), focusing on distinguishing between two types: hallucinations caused by a lack of knowledge (HK-) and hallucinations that occur despite the LLM having the necessary knowledge (HK+). The authors introduce a novel methodology called WACK (Wrong Answers despite having Correct Knowledge), which constructs model-specific datasets to identify these different types of hallucinations. The paper demonstrates that LLMs’ internal states can be used to distinguish between these two types of hallucinations, and that model-specific datasets are more effective for detecting HK+ hallucinations compared to generic datasets. The study highlights the importance of understanding and mitigating these different types of hallucinations to improve the reliability and accuracy of LLMs.

This paper provides a comprehensive survey of small language models (SLMs) in the context of large language models (LLMs). The authors discuss the benefits of SLMs over LLMs, including their low inference latency, cost-effectiveness, and ease of customization. They also explore the various techniques used to develop and enhance SLMs, including architecture design, training methods, and model compression. The paper goes on to analyze the applications of SLMs in various NLP tasks, such as question answering, coding, and web search. Finally, the authors address the trustworthiness of SLMs and identify several promising future research directions.

This research investigates how large language models (LLMs) perform arithmetic tasks. The authors find that LLMs do not rely on robust algorithms or memorization but instead use a "bag of heuristics," a collection of simple, memorized rules, to solve arithmetic problems. They identify a specific set of neurons in the LLMs that implement these heuristics and analyze how they develop over the course of training. Their findings suggest that improving LLMs' mathematical abilities may require fundamental changes to training and architecture rather than relying on post-hoc techniques.

This research paper investigates the vulnerabilities of vision-language models (VLMs) used to power computer agents. The authors demonstrate that these agents can be easily manipulated by carefully crafted adversarial pop-ups, causing them to click on these malicious elements instead of performing their intended tasks. This attack successfully diverts the agents from their intended actions in over 80% of cases, significantly reducing their task success rate. The authors explore various attack design elements and find that basic defense strategies, such as asking the agent to ignore pop-ups or including an advertisement notice, are ineffective. They conclude that more robust agent systems are needed to ensure safe agent workflow in real-world computer environments.

This research paper proposes a new method for training large language models (LLMs) to solve complex scientific problems. The authors argue that current LLMs struggle with complex scientific questions, often hallucinating answers instead of providing accurate solutions. To address this, they suggest integrating LLMs with specialized tools. However, traditional methods for integrating tools often result in the model over-relying on these tools, even when simpler reasoning would suffice. To overcome these limitations, the paper presents a two-component fine-tuning method: World Knowledge Distillation (WKD), which allows the LLM to learn from the solutions generated by tools, and Tool Usage Adaptation (TUA), which trains the model to intelligently choose between direct reasoning and using external tools based on the difficulty of the question. The authors demonstrate the effectiveness of their approach on various datasets across different scientific domains, including mathematics, climate science, and epidemiology, finding significant improvements in both answer accuracy and tool usage precision.

This report details the research process of a team of researchers at Shanghai Jiao Tong University and other institutions who are attempting to replicate OpenAI's O1 model, a groundbreaking language model capable of complex reasoning. The report advocates for open science by detailing the entire research journey, including successes and failures, to help other researchers accelerate progress in the field. The researchers have introduced a new paradigm called "journey learning," where models learn not just shortcuts to solutions, but the entire exploration process, including trial and error and reflection, which they believe is key to O1's capabilities. They share valuable resources including technical hypotheses, cognitive exploration maps, and custom-developed tools. The report also highlights the challenges of traditional AI research and proposes a new framework for scientific communication and collaboration.

This paper provides a comprehensive review of document parsing, a field focused on converting unstructured and semi-structured documents into structured, machine-readable data. It explores two main approaches: modular pipeline systems and end-to-end models based on large vision-language models. The paper examines the core components of document parsing, including layout detection, content extraction (text, tables, mathematical expressions), and relation integration, as well as the challenges each approach faces. The authors provide an overview of key methodologies, datasets, evaluation metrics, and open-source tools, ultimately emphasizing the need for further research and development to advance the field.

This study investigates the gradient patterns of different layers in large language models (LLMs) during instruction tuning. The researchers compare the gradients of LLMs trained using "fast thinking" (without chain-of-thought reasoning) versus "slow thinking" (with detailed chain-of-thought reasoning). The study examines how these training methods affect gradient stability, response correctness, and the ability to distinguish between correct and irrelevant responses. They further analyze the impact of different initial models (pre-trained vs. instruction-tuned) on gradient behavior. The results show that "slow thinking" leads to more stable and efficient training, while "fast thinking" results in larger gradients and greater fluctuation across layers. The researchers also find that "slow thinking" helps distinguish correct responses from irrelevant responses, but this ability is not as pronounced in "fast thinking" training. Finally, the study explores the effects of response length and popularity on gradient patterns in knowledge-learning tasks, demonstrating that increasing response length alone does not necessarily mimic the effects of "slow thinking."

This research paper introduces REPOCOD, a new benchmark for evaluating large language models (LLMs) in code generation tasks that require knowledge of entire software projects. The benchmark consists of 980 code generation problems collected from 11 popular open-source Python repositories. REPOCOD is designed to be more challenging than existing benchmarks by requiring LLMs to generate complex functions, often involving interactions across multiple files in a project. The authors evaluate the performance of various LLMs on REPOCOD, finding that even the most advanced models struggle to achieve high accuracy, particularly when dealing with functions requiring knowledge of the entire repository. This research highlights the need for further development of LLMs that can effectively handle the complexity of real-world code generation within the context of software projects.

This research paper introduces SocialGPT, a new modular framework that leverages the perception capabilities of vision foundation models (VFMs) and the reasoning capabilities of large language models (LLMs) to identify social relationships between people in images. Unlike previous methods that train a dedicated network end-to-end, SocialGPT translates image content into a textual social story using VFMs, which is then used for text-based reasoning with LLMs. The paper also proposes a novel prompt optimization method called Greedy Segment Prompt Optimization (GSPO), which helps improve the performance of LLMs by performing a greedy search on the segment level with gradient guidance. SocialGPT achieves highly competitive results on two datasets without additional model training and provides interpretable answers, offering language-based explanations for the decisions.

This paper describes the development of ROCKET-1, a novel hierarchical agent architecture that leverages visual-temporal context prompting to enable agents to master open-world interaction in Minecraft. This system utilizes a low-level policy (ROCKET-1) that predicts actions based on concatenated visual observations and segmentation masks, guided by a high-level reasoner. The key innovation lies in the visual-temporal context prompting protocol, which uses object segmentation from both past and present observations to effectively communicate spatial information. This approach allows for the successful completion of complex tasks, such as crafting and mining, in Minecraft, which were previously unattainable using traditional language-based prompting methods. The paper highlights the potential of visual-temporal context prompting to overcome the limitations of existing approaches and unlock the full potential of vision-language models for embodied decision-making.

This research investigates the effectiveness of knowledge editing techniques in correcting hallucinations in large language models (LLMs). The authors present HalluEditBench, a comprehensive benchmark that evaluates the performance of different knowledge editing methods across five dimensions: efficacy, generalization, portability, locality, and robustness. They discovered that while some knowledge editing methods show promising results on existing benchmarks, their effectiveness in correcting real-world hallucinations may be significantly lower, highlighting the need for more robust evaluation methods. Additionally, the study provides insights into the limitations of different editing methods, suggesting that no single method excels across all five dimensions. The authors conclude by emphasizing the importance of understanding the potential and limitations of knowledge editing techniques for achieving more accurate and reliable LLMs.