The World Won't Wait: Evaluating and Teaching Agents Under a Live Clock

arXiv:2610.20665v1 Announce Type: new Abstract: Large language model (LLM) agents can solve complex interactive tasks, yet reasoning and tool calls take time. In continuously changing environments, an otherwise correct action may arrive too late. We introduce RealtimeBench, with 180 tasks across six game types, to evaluate agents while games continue running during inference. Paired evaluations show that GPT-6.1-sol outperforms the faster Jev when games are paused, but falls behind when they keep running. We propose Two-Timescale Offline Recursive Self-Improvement (TORSI): a slow teacher studies failures and tests corrections in paused games between attempts, then writes reusable instructions for a fast student. The student acts without teacher calls during gameplay, and validation scores determine which updates are adopted. Two rounds raise Jev's average test score from 13.51 to 24.19 on a 0--100 progress scale, improving all six game types. The instructions also improve average scores for three other models without retuning. These findings point toward real-time intelligence: enabling powerful LLM agents to remain effective in a continuously changing world by organizing deliberation, learning, and action across timescales.

Improves AI performance in dynamic environments, but implications for human flourishing are uncertain.

This is an incremental capability benchmark paper on agent performance under time pressure, not a claim about societal impact. Absolute scores remain low (24/100), indicating early-stage research. Mild negative lean reflects generic capability-advance risk, but evidence of societal implication is thin.

The paper introduces a method for LLM agents to operate more effectively in real-time environments by using a two-timescale learning approach. This could make AI agents more functional and adaptable, which is a mild positive development for AI capabilities. There are no direct societal implications discussed.

The paper presents a method for improving LLM agents in real-time tasks. The results show significant improvement in performance. However, the implications for human flourishing are uncertain.

SOURCE     arXiv cs.AI (tier 1)
   ↓
DOCUMENT   444318ce-5cac-4af8-b96b-2b32fbc76a48
           https://arxiv.org/abs/2610.20665
   ↓
EVIDENCE   2 extracted excerpts
   ↓
MODEL RUN  4 runs, methodology 1.0.1
   ↓
SCORE      -14  (Mixed or uncertain)
   ↓
CONFIDENCE 75%