Research→Official→arXiv Computation and Language
Researchers introduce PCC-SQL, a method that enforces column-use policies during text-to-SQL decoding by applying per-token logits masking, which deterministically prevents policy violations. The approach achieves a 0% leakage rate and up to 88.7% coverage on the Spider-CU benchmark, with minimal increase in token usage compared to direct prompting. The method is evaluated across three benchmarks and three open-source models, and also assesses semantic alignment with execution accuracy.
Why it matters: This work provides a deterministic solution to enforcing column-level access control in text-to-SQL systems, addressing a key security concern in real-world deployments.
Jul 15, 2026
Research→Official→arXiv Computation and Language
Researchers introduce Knowledge-Less Language Models (KLLMs), which are pretrained on corpora with anonymized named entities to reduce the model's reliance on memorized factual knowledge. KLLMs demonstrate reduced closed-book factual recall but outperform standard models on contextual question answering, fact verification, and hallucination detection, especially in retrieval-grounded settings with imperfect evidence, achieving up to 20–25% relative gains. The models also show improved calibration and more reliable abstention behavior, indicating a shift toward evidence-grounded reasoning.
Why it matters: This work shows that controlling knowledge acquisition during pretraining can produce language models that are more robust and reliable in evidence-grounded tasks, potentially improving real-world AI applications.
Jul 15, 2026
Research→Official→arXiv Computation and Language
A new preprint evaluates whether translating non-English data into English and fine-tuning English BERT models can match or outperform native-language BERT models across six NLP tasks. The study finds that this translation-based approach is comparable or superior in 53.3% of cases, especially for syntactic tasks and languages typologically close to English, but is less effective for token-level or culturally nuanced tasks.
Why it matters: This work demonstrates a practical, resource-efficient strategy for extending NLP capabilities to low-resource languages without the need for large native-language models.
Jul 15, 2026
Research→Official→arXiv Computation and Language
Researchers have identified a general 'countdown subcircuit' in Llama-3.1-70B-Instruct that compares the current position to a goal length, allowing the model to estimate remaining tokens. This subcircuit uses an identical motif previously found in a frontier LLM on a different task, indicating shared mechanisms across models. Unsupervised probing revealed that the subcircuit is employed in various natural language tasks, including those where the goal length must be inferred from context.
Why it matters: This finding advances mechanistic interpretability by revealing a shared neural mechanism for token counting across tasks and models, potentially improving our understanding and control of LLM behavior.
Jul 15, 2026
Research→Official→arXiv Computation and Language
A preprint analyzing 2,848 Claude Code runs across 103 tasks finds that reducing tool-output tokens does not reliably lower billed costs for API-based coding agents, as prompt-cache traffic accounts for about 87% of costs. The study also shows that compression can harm task completion, with patch application rates dropping from 27/40 to 15/40 on SWE-bench-derived Go tasks due to corrupted edit anchors.
Why it matters: This challenges the assumption that token reduction directly reduces costs for coding agents, emphasizing the need for billing metrics that account for both cost and task success.
Jul 15, 2026
Research→Official→arXiv Computation and Language
A new method formulates evidence selection in retrieval-augmented question answering (RAG) as a Quadratic Unconstrained Binary Optimization (QUBO) problem. This approach balances relevance, coverage, and redundancy, and achieves competitive performance with LLM-based selectors on the HotpotQA benchmark, while enabling the use of Ising/QUBO solvers for context selection.
Why it matters: The work demonstrates that multi-hop evidence selection can be effectively handled by discrete optimization solvers, potentially reducing reliance on LLMs for intermediate steps and improving scalability and cost-efficiency in RAG pipelines.
Jul 15, 2026
Research→Official→arXiv Computation and Language
Researchers present CARE-PPO, a reinforcement learning framework that adapts the actor-critic PPO method to enable large language models (LLMs) to jointly produce accurate numerical predictions and well-calibrated confidence estimates. By repurposing the critic network as a confidence estimator during inference, CARE-PPO demonstrates improved uncertainty calibration compared to logit-based and verbalized baselines on real-world healthcare and finance tasks. The approach also shows robustness to out-of-distribution data and reduces overfitting compared to supervised methods.
Why it matters: This work advances the reliability of LLMs in high-stakes domains by improving both prediction accuracy and confidence calibration, addressing a key challenge for trustworthy AI deployment.
Jul 15, 2026
Research→Official→arXiv Computation and Language
Researchers evaluated agentic large language model (LLM) systems for generating breast cancer treatment recommendations using 72 real clinical cases and 1,147 case-specific rubrics. The best-performing system, Claude Opus 4.8 with a D&C+SA pipeline, achieved a global score of 0.594, but oncologist-led error analysis found persistent clinically relevant failures, including incorrect or missing recommendations, flawed justifications, citation errors, outdated claims, and overconfidence.
Why it matters: The study demonstrates that while agentic LLMs can generate clinically relevant recommendations, they are not yet reliable enough for unsupervised clinical use in oncology.
Jul 15, 2026
Research→Official→arXiv Computation and Language
A new preprint reports that state-of-the-art large language models (LLMs) perform poorly on Korean-Braille translation tasks, producing unstable outputs that often disagree with human judgments. The study finds that these failures stem from missing Braille-aware tokenization and weak alignment between Korean and Braille patterns. In contrast, a small T5-small model, when fine-tuned on the task, achieves large and stable improvements over LLM baselines. The results highlight a systematic limitation of current LLMs in handling accessibility-critical modalities.
Why it matters: The study exposes a significant gap in LLM capabilities for accessibility, indicating that current models are not reliable for Braille translation without specialized adaptation.
Jul 15, 2026
Research→Official→arXiv Computation and Language
Researchers demonstrate that scaling point-in-time language models—trained only on data available up to each calendar date—can substantially narrow the performance gap with models trained on unrestricted internet data. Using up to 4 billion parameters and 1 trillion temporally filtered tokens, their monthly checkpoints from 2013-2024 approach the performance of leading open-weight models like Gemma-3-4B and LLaMA-7B on reasoning and language understanding benchmarks, though some gaps remain. The team also provides a reproducible pipeline for dataset construction, training, and evaluation.
Why it matters: This work enables more valid backtesting and causal inference in finance and social sciences by eliminating future data leakage in language models.
Jul 15, 2026
Research→Official→arXiv Computation and Language
Researchers have introduced CANDI-QA, a new dataset designed to evaluate large language models (LLMs) on context-sensitive and user-aligned question answering in specialized fields such as medical diagnostics and financial advisory. The dataset includes both factual and multi-hop reasoning questions, and initial tests show that current LLMs face significant challenges without improved contextual or symbolic reasoning capabilities. CANDI-QA also provides a neuro-symbolic baseline model, MTSS-Net, to benchmark progress.
Why it matters: CANDI-QA exposes critical limitations in current LLMs for high-stakes, specialized domains, guiding future research toward more reliable and context-aware AI systems.
Jul 15, 2026
Research→Official→arXiv Computation and Language
Researchers introduce G-SHARE, a framework that translates the CNNP nine-step human-factor event diagnosis guideline into a multi-stage pipeline for analyzing events in nuclear power plants. The system integrates evidence extraction, stepwise diagnostic reasoning, and post-hoc consistency repair, enabling explicit use of report evidence and logical validation of outputs. In evaluations on real-world event reports, G-SHARE substantially outperforms one-shot prompting and traditional machine learning baselines, with structured reasoning and consistency enforcement shown to be critical for robust diagnosis.
Why it matters: This work demonstrates a practical method for converting expert diagnostic guidelines into auditable, structured reasoning workflows, potentially improving reliability in safety-critical industries.
Jul 15, 2026
Research→Official→arXiv Computation and Language
A new preprint identifies two distinct mechanisms in language models that enable them to separate a character's beliefs from reality: a generic value slot that binds attributed values, and a router at the query position that determines which frame—belief or reality—is accessed. The value slot itself does not encode whether a value is a belief or reality; instead, the separation is achieved through dissociated routing subspaces that can switch queries between frames without transferring the donor's value. These findings are consistent across multiple model architectures and sizes, and are robust to confounds from theory-of-mind benchmarks.
Why it matters: This mechanistic insight advances our understanding of how language models represent and reason about mental states and non-actual contexts, which could inform future improvements in AI reasoning and interpretability.
Jul 15, 2026
Research→Official→arXiv Computation and Language
Researchers fine-tuned LLaMA 3 (8B) as a reranker for retrieval-augmented generation (RAG) pipelines using supervised fine-tuning and 4-bit quantization. The adapted model achieved 14-21% improvements over traditional cross-encoders on key RAGAS benchmark metrics, while also reducing inference overhead.
Why it matters: This work demonstrates that instruction-tuned LLMs can serve as efficient and accurate rerankers in RAG systems, potentially enabling real-time deployment without the computational costs of traditional cross-encoders.
Jul 15, 2026
Research→Official→arXiv Computation and Language
A new preprint introduces MAGE, a framework for controlled analysis of how different components in iterative prompt optimization interact. The study identifies the Prompt Optimization Coupling Effect (POCE), where combining multiple stochastic signals in prompt optimization can simultaneously improve performance and increase variance. The research also finds that failure-grounded reflection is essential for prompt improvement, and that in low-data regimes, well-designed fixed prompts can outperform reflective optimizers.
Why it matters: This work reveals that prompt optimization systems act as coupled stochastic processes, emphasizing the need to evaluate both performance and stability rather than focusing solely on peak accuracy.
Jul 15, 2026
Research→Official→arXiv Computation and Language
Researchers introduce TAKE, a framework for text dataset distillation that reduces large text corpora to as little as 0.1% of their original size while maintaining performance on downstream tasks. TAKE leverages influence functions and trajectory-aware knowledge estimation to identify and select the most informative samples using an optimal transport objective. Evaluations on text classification and natural language inference tasks demonstrate that TAKE achieves extreme compression without significant loss in accuracy.
Why it matters: This work offers a principled approach to drastically reducing the size of text datasets, potentially lowering storage and computational costs for NLP training and fine-tuning.
Jul 15, 2026
Research→Official→arXiv Computation and Language
A new preprint introduces QIMG-7, a benchmark designed to evaluate multimodal retrieval-augmented generation (RAG) systems under conditions of retrieval pollution, including unreliable or manipulated text and images. The authors propose source-aware trust resolution (SATR), a training-free method that selects among parametric, text-only, and multimodal answers based on the reliability of retrieved sources. SATR achieves a balanced score of 0.816 on QIMG-7, outperforming naive multimodal fusion by 11.7 points and demonstrating improved robustness to polluted retrieval.
Why it matters: This work provides a practical approach to improving the reliability of multimodal RAG systems in the presence of unreliable or adversarially manipulated retrieved content.
Jul 14, 2026
Research→Official→arXiv Computation and Language
ResearchQA is a new benchmark comprising 6,211 question-answer pairs from 494 open-access scientific papers across eight domains, designed to evaluate citation-grounded question answering. It covers four question types and rewards grounded refusal when answers are unsupported by the source. Evaluation of eight leading models shows that citation-based metrics distinguish system performance more clearly than LLM evaluator scores, and open-weight models approach the citation accuracy of top closed models while operating at lower latency.
Why it matters: This benchmark enables more rigorous assessment of whether language models can answer scientific questions with verifiable citations, addressing a key limitation in current evaluation methods.
Jul 14, 2026
Research→Official→arXiv Computation and Language
Researchers used GPT-4.1 to annotate approximately 9,000 customer support conversations, breaking down satisfaction into five axes: overall, agent, outcome, product, and effort. Four axes (overall, agent, outcome, effort) closely tracked self-reported customer ratings, while product satisfaction showed weak alignment. The study found that satisfaction scores are significantly lower when measured across all contacts (2.91) compared to only those who responded to surveys (3.62) on a five-point scale, highlighting survey bias.
Why it matters: This work demonstrates that LLM-based decomposition can uncover hidden drivers of customer experience and reveal biases in traditional survey-based satisfaction metrics.
Jul 14, 2026
Research→Official→arXiv Computation and Language
A new preprint systematically examines how various NL2SQL pipeline extensions—such as the NatSQL intermediate representation, preprocessing, synthetic data fine-tuning, and a novel reranker—interact when integrated into two backbone models, SmBoP and RASAT. Through ablation and Shapley analyses, the study finds that combining all components does not necessarily yield the best performance; instead, the effectiveness of each extension depends on its interactions with the baseline and other components. These findings challenge the assumption that more features always improve NL2SQL systems.
Why it matters: The work offers practical guidance for building more efficient NL2SQL models by clarifying how different pipeline optimizations interact.
Jul 14, 2026