Back to Articles a]:hidden"> ALTK-Evolve: How learned memory tops the τ²-bench leaderboard on customer service tasks without a frontier model Enterprise Article Published October 20, 2026 Upvote 24 +18 Ashwath Vaithinathan Aravindan ashwath-vaithina Follow ibm-research Vatche Isahagian Vatche Follow ibm-research Vinod Muthusamy vinodmut Follow ibm-research Jayaram Radhakrishnan jayaramkr Follow ibm-research Evelyn Duesterwald evduester Follow ibm-research Gaodan Fang gaodan-fang Follow ibm-research Punleuk Oum illeatmyhat Follow ibm-research G Thomas gsthomasx Follow ibm-research Michał Ulewicz Michal Follow ibm-research Merve Unuvar mrvnvr Follow ibm-research Ayhan Sebin ayhansebin Follow ibm-research :last-child]:mb-0"> Learned memory takes a basic ReAct agent with GPT-4.1 to the top τ²-bench tests policy-bound customer service Guidelines improve capability and reliability The best amount of guidance depends on the domain Better performance costs more tokens, not more turns Learned guidelines also improve agents using frontier models ALTK-Evolve tops the leaderboard comparison Agents can improve without weight updates Links: ALTK-Evolve learns and applies a playbook using GPT-4.1, outperforming the retail entries on τ²-bench, a customer service agent benchmark—without updating model weights
Learned memory takes a basic ReAct agent with GPT-4.1 to the top Using GPT-4.1, an older, non-reasoning model, a basic customer-service agent reaches 67.5% Pass^4 on τ²-bench retail — meaning it solves the task on all four independent attempts for 67.5% of tasks. It scores above every retail entry in our October 12, 2026 leaderboard snapshot. It runs on the benchmark's stock ReAct scaffold, with learned guidelines appended to its system prompt
ALTK-Evolve builds that playbook using GPT-4.1. It analyzes the agent's training-task attempts and uses the model to distill successes and failures into short behavioral rules: how to authenticate a customer, confirm a transaction, or handle a return. Those guidelines are then added to the agent's prompt for subsequent conversations. The same model powers both guideline extraction and the agent; no weight updates or human annotation are needed
