Research→Official→arXiv Computation and Language
Researchers introduce TAKE, a framework for text dataset distillation that reduces large text corpora to as little as 0.1% of their original size while maintaining performance on downstream tasks. TAKE leverages influence functions and trajectory-aware knowledge estimation to identify and select the most informative samples using an optimal transport objective. Evaluations on text classification and natural language inference tasks demonstrate that TAKE achieves extreme compression without significant loss in accuracy.
Why it matters: This work offers a principled approach to drastically reducing the size of text datasets, potentially lowering storage and computational costs for NLP training and fine-tuning.
Jul 15, 2026
Research→Official→arXiv Computation and Language
A new preprint introduces QIMG-7, a benchmark designed to evaluate multimodal retrieval-augmented generation (RAG) systems under conditions of retrieval pollution, including unreliable or manipulated text and images. The authors propose source-aware trust resolution (SATR), a training-free method that selects among parametric, text-only, and multimodal answers based on the reliability of retrieved sources. SATR achieves a balanced score of 0.816 on QIMG-7, outperforming naive multimodal fusion by 11.7 points and demonstrating improved robustness to polluted retrieval.
Why it matters: This work provides a practical approach to improving the reliability of multimodal RAG systems in the presence of unreliable or adversarially manipulated retrieved content.
Jul 14, 2026
Research→Official→arXiv Computation and Language
ResearchQA is a new benchmark comprising 6,211 question-answer pairs from 494 open-access scientific papers across eight domains, designed to evaluate citation-grounded question answering. It covers four question types and rewards grounded refusal when answers are unsupported by the source. Evaluation of eight leading models shows that citation-based metrics distinguish system performance more clearly than LLM evaluator scores, and open-weight models approach the citation accuracy of top closed models while operating at lower latency.
Why it matters: This benchmark enables more rigorous assessment of whether language models can answer scientific questions with verifiable citations, addressing a key limitation in current evaluation methods.
Jul 14, 2026
Research→Official→arXiv Computation and Language
Researchers used GPT-4.1 to annotate approximately 9,000 customer support conversations, breaking down satisfaction into five axes: overall, agent, outcome, product, and effort. Four axes (overall, agent, outcome, effort) closely tracked self-reported customer ratings, while product satisfaction showed weak alignment. The study found that satisfaction scores are significantly lower when measured across all contacts (2.91) compared to only those who responded to surveys (3.62) on a five-point scale, highlighting survey bias.
Why it matters: This work demonstrates that LLM-based decomposition can uncover hidden drivers of customer experience and reveal biases in traditional survey-based satisfaction metrics.
Jul 14, 2026
Research→Official→arXiv Computation and Language
A new preprint systematically examines how various NL2SQL pipeline extensions—such as the NatSQL intermediate representation, preprocessing, synthetic data fine-tuning, and a novel reranker—interact when integrated into two backbone models, SmBoP and RASAT. Through ablation and Shapley analyses, the study finds that combining all components does not necessarily yield the best performance; instead, the effectiveness of each extension depends on its interactions with the baseline and other components. These findings challenge the assumption that more features always improve NL2SQL systems.
Why it matters: The work offers practical guidance for building more efficient NL2SQL models by clarifying how different pipeline optimizations interact.
Jul 14, 2026
Research→Official→arXiv Computation and Language
A new preprint introduces Progressive Tree Drafting (PTD), a speculative decoding method that accelerates large language model (LLM) inference by up to 2x. PTD is both training-free and model-agnostic, using a progressive tree structure and stepwise pruning to explore multiple semantic paths in a single forward pass, which improves draft diversity and coherence.
Why it matters: PTD offers a practical way to significantly speed up LLM inference without the need for additional training or auxiliary models.
Jul 14, 2026
Models→Official→arXiv Computation and Language
A new arXiv preprint evaluates Claude Fable 5 on eight biomedical benchmarks, finding it refuses to answer between 8.0% and 99.4% of questions depending on the benchmark—a pattern not seen in its predecessors or GPT-5. When refusals are excluded, Fable 5's accuracy meets or exceeds all other models on every benchmark. The study also identifies distinct refusal patterns related to basic-science content and rare disease domains.
Why it matters: This suggests that the main limitation for deploying advanced language models like Fable 5 in biomedical settings may be their willingness to answer, rather than their underlying capability.
Jul 14, 2026
Research→Official→arXiv Computation and Language
Researchers introduce three metrics—Reference Abstraction (RA), Summary Abstraction (SA), and Abstraction Ratio (AR)—to quantify how much a summary diverges from extractive copying of source text. Evaluated on 100 XSUM documents across four summarization models, these metrics effectively distinguish between extractive and abstractive approaches, with AR highlighting summaries that may require manual hallucination checks.
Why it matters: These metrics offer a principled alternative to traditional evaluation methods like ROUGE, enabling more nuanced assessment of summarization models and aiding in the detection of potential hallucinations.
Jul 14, 2026
Research→Official→arXiv Computation and Language
Researchers have identified 'Thinking Collapse,' a failure mode in on-policy self-distillation (OPSD) for large language models, where intermediate reasoning abilities degrade during training. They introduce Adaptive Dual-Perspective OPSD (AD-OPSD), a method that dynamically moderates the self-distillation objective to preserve reasoning capacity. Experiments show that AD-OPSD improves accuracy by up to 4.1% on mathematical benchmarks compared to standard OPSD.
Why it matters: This work addresses a critical limitation in a widely used LLM training technique, potentially leading to more reliable reasoning in complex tasks.
Jul 14, 2026
Research→Official→arXiv Computation and Language
A new preprint investigates the pragmatic reasoning abilities of large language models (LLMs) in multi-party collaborative tasks where participants have only partial information. The study finds that while LLMs can display some pragmatic skills, they often struggle to communicate effectively under epistemic asymmetry and frequently fail to recognize violations of Grice's cooperative principles.
Why it matters: This work reveals key limitations in LLMs' ability to engage in effective, cooperative communication in real-world scenarios where information is not fully shared.
Jul 14, 2026
Research→Official→arXiv Computation and Language
A new preprint finds that language models have difficulty reliably accumulating facts in their weights during continual learning. After twenty sequential writes, facts trained with diverse data retain 46% accuracy, while those trained with bare statements retain only 1%. The study suggests that context, rather than model weights, is the more reliable channel for composing or preserving facts through multiple updates.
Why it matters: This challenges the feasibility of using language model weights for continual knowledge accumulation, with implications for how models are updated and retain information over time.
Jul 14, 2026
Research→Official→arXiv Computation and Language
MafiaScope is an open testbed that adapts the social deduction game Mafia to probe the beliefs and Theory of Mind of LLM agents. After each public utterance, agents privately answer structured probe questions, with responses scored against ground truth and visualized as belief trajectories. In a 32-game case study using DeepSeek, the system revealed poor calibration (expected calibration error 0.17) and a tendency for agents to over-predict being suspected. The platform, including engine, viewer, and a large corpus of games, is released under an open license.
Why it matters: This introduces a novel, systematic approach to measuring and visualizing the internal beliefs and reasoning processes of LLM agents during social interactions.
Jul 14, 2026
Research→Official→arXiv Computation and Language
The first ChineseBabyLM challenge will be held at the 2026 NLPCC conference, inviting researchers to train language models from scratch using 100 million Chinese tokens. The competition will evaluate models on three tracks: natural language understanding (NLU), cognitive alignment, and Hanzi knowledge, with no restrictions on tokenizer, model architecture, or training epochs.
Why it matters: This challenge aims to advance research on data-efficient and cognitively plausible language models for Chinese, encouraging approaches that better reflect human-like language acquisition in a non-English context.
Jul 14, 2026
Policy Safety→Official→arXiv Computation and Language
A new preprint introduces DC-GRPO, a turn-level credit assignment framework for multi-turn jailbreaking of large language models (LLMs). The method assigns learning signals to individual dialogue turns, enabling more effective automated red teaming. Experiments show DC-GRPO achieves over 97% attack success rate across multiple benchmarks, substantially outperforming previous state-of-the-art methods.
Why it matters: This work demonstrates a highly effective automated jailbreaking technique for multi-turn LLM interactions, highlighting a significant vulnerability in current conversational AI safety measures.
Jul 14, 2026
Research→Official→arXiv Computation and Language
Anamnesis is an open-source platform that enables large-scale survey simulation using large language models (LLMs), designed to be accessible for non-technical users. The system leverages structured narrative backstories to condition responses and supports multimodal surveys, including image and audio. Case studies demonstrate that Anamnesis produces opinion distributions that more closely align with real-world survey data compared to standard persona-prompting methods.
Why it matters: This platform offers a transparent and reproducible alternative to proprietary survey simulation tools, allowing researchers to prototype and stress-test survey instruments without relying on human subjects.
Jul 14, 2026
Research→Official→arXiv Computation and Language
A new framework combines multidimensional classification and stratified sampling to select representative subsets of opinions before summarization by large language models (LLMs). This approach reduces token usage and computational cost, while experiments on product reviews, hotel feedback, and social posts show improved coverage, balance, and semantic preservation compared to traditional and standard LLM summarization baselines.
Why it matters: Efficiently summarizing large-scale opinionated text without sacrificing viewpoint diversity is crucial for applications such as product analysis and social listening.
Jul 14, 2026
Research→Official→arXiv Computation and Language
Researchers present FATE, an 8B-parameter language model specifically designed to evaluate AI tutors across four pedagogical tracks: Mistake Identification, Mistake Location, Guidance, and Actionability. By leveraging knowledge distillation from a frontier LLM, FATE achieves up to 22.63 percentage point absolute performance gains. The model is used to benchmark instructional responses from several commercial LLMs, with Gemini 2.5 Flash achieving the highest average score.
Why it matters: This work provides a specialized tool for automated, reliable evaluation of AI tutors, addressing a key challenge as LLMs become more prevalent in education.
Jul 14, 2026
Research→Official→arXiv Computation and Language
Researchers have introduced EYT-Bench, a benchmark designed to evaluate large language models (LLMs) as multi-turn conversational partners. EYT-Bench uses a decoupled, three-party design involving a persona-grounded user simulator, a target model, and an independent judge, with personas sampled from human-curated corpora. In a 17-model evaluation, the benchmark reveals that while models are statistically similar on subjective measures like empathy and persona consistency, they differ by up to 9x on objective intent tracking. Additionally, enabling reasoning in models improves objective tracking without affecting subjective scores.
Why it matters: EYT-Bench exposes significant gaps in objective intent tracking among conversational AI models that are not captured by single-turn benchmarks, highlighting the need for more comprehensive evaluation methods.
Jul 14, 2026
Research→Official→arXiv Computation and Language
Researchers present Structured Thoughts, a framework that organizes large language model (LLM) reasoning into alternating scratch and summary blocks. Fine-tuning LLMs on this structured data leads to up to 8.08% performance improvements on reasoning benchmarks. The framework also enables context pruning, which can save 85% of memory with only an 8.67% drop in performance on mathematical tasks.
Why it matters: This approach offers a practical method to enhance both the reasoning quality and computational efficiency of LLMs by addressing the memory inefficiency of long reasoning chains.
Jul 14, 2026
Models→Official→arXiv Computation and Language
A new arXiv preprint examines how fine-tuned GPT-2 models reconstruct natural English from 'impossible' languages with disrupted word order. The study finds that these models tend to recover sentence structures with shorter dependency lengths, indicating a strong preference for information locality. Recovery becomes more difficult as word order is increasingly disrupted, and sentence length affects reconstruction differently depending on the type of perturbation applied.
Why it matters: This work provides quantitative evidence of architectural biases in language models, offering new insights into how they process and reconstruct language beyond what learnability experiments reveal.
Jul 14, 2026