Soft-Token Fusion Framework Enables LLM-Based Recommender Systems to Incorporate Numerical and Embedding Features

Jul 14, 2026

A new soft-token fusion framework allows large language model (LLM)-based recommender systems to integrate continuous numerical and embedding features by mapping them into the LLM embedding space. The framework, implemented in a two-tower retrieval model with an interaction-based fusion module, demonstrates improved retrieval performance over LLM-based baselines on three Amazon recommendation benchmarks. The results also show that interaction-based fusion outperforms simple concatenation of heterogeneous soft tokens.

Why it matters: This work addresses a key limitation of LLM-based recommenders by enabling them to utilize non-textual signals common in real-world recommendation systems, potentially enhancing their practical effectiveness.

Full story at: arXiv Information Retrieval ↗