Beyond Plausible Prediction: Safety Sufficiency in Web World Models

arXiv:2610.40798v1 Announce Type: new Abstract: Web world models increasingly predict the consequences of candidate actions before agents act, yet a plausible predicted future does not necessarily support the correct safety decision. In a matched human evaluation of 3,000 predictions from ten frozen open models, 86.1% are judged semantically plausible. However, several models exhibit highly asymmetric authorisation behaviour, combining low false-safe execution with poor recall of clarifiable ASK cases and a strong preference for BLOCK. This motivates safety sufficiency: whether a predictive interface preserves and exposes the policy-decisive information required to distinguish actions that may be EXECUTED, require clarification through ASK, or should be BLOCKED. We introduce SafeWebWorld, a benchmark for evaluating safety sufficiency across five web-safety domains through action-conditioned futures, policy-decisive evidence, and policy-conditioned decisions. Under a controlled Security-Aligned Supervised Fine-Tuning protocol, evidence preservation can improve substantially while authorisation-sensitive decisions deteriorate. To determine whether this deterioration reflects loss of authorisation-sensitive score structure or a failure of the decision readout, we introduce Latent Authorisation Recovery (LAR). Frozen-score recalibration reveals substantial but backbone-dependent recovery, while paired recovery provides only a modest additional gain. Yet within-family and matched-authorisation evaluations remain poor, showing that aggregate recoverability does not imply robust f

The study reveals safety concerns in web world models, showing that plausible predictions do not guarantee correct safety decisions.

The research highlights challenges in AI web safety models, showing that plausibly predicted futures do not ensure correct safety decisions. The finding that improved evidence preservation can coincide with worse decision-making suggests a risk of AI systems making unsafe authorisation choices. However, these are controlled experimental results and don't imply catastrophic outcomes at present.

This is a technical safety benchmarking paper showing current web world models have inconsistent authorisation behavior and that naive fine-tuning can worsen safety decisions. It is a methodological contribution highlighting gaps rather than a deployed harm, so implications are modestly concerning but constructive in intent.

The paper identifies that AI models predicting action consequences, even when plausible, are insufficient for making correct safety decisions in web environments. It demonstrates models exhibit problematic authorization behavior, favoring 'BLOCK' and failing to identify cases needing 'ASK', even after attempts to improve evidence preservation. This highlights a significant safety challenge for AI systems in critical web applications.

The text discusses significant issues with web world models' safety decisions. The models often block actions unnecessarily, and fine-tuning can degrade their decision-making. This suggests potential over-blocking of human actions online, hindering user experience and possibly restricting legitimate activities.

SOURCE     arXiv cs.AI (tier 1)
   ↓
DOCUMENT   b878e7da-3a06-46a2-8f0e-fc4266cd0ad0
           https://arxiv.org/abs/2610.40798
   ↓
EVIDENCE   2 extracted excerpts
   ↓
MODEL RUN  5 runs, methodology 1.0.1
   ↓
SCORE      -42  (Adverse)
   ↓
CONFIDENCE 78%