Artificiality's First Exam: An LLM-Referenced C-test for Supraintelligence

arXiv:2610.14005v1 Announce Type: new Abstract: Under the accelerated pace of AI development, any benchmark whose difficulty is calibrated with humans or current AI systems becomes circular, or saturates soon. How can we then evaluate machines that may become more capable than the current machines and the people who write the tests? One answer to the question hinges on inverse problems, inductive inference questions for which difficulty depends on the complexity of the underlying mechanism, given the observations and the previous knowledge. Levin's Kt, a computable variant of Kolmogorov complexity, puts a theoretical upper bound to that difficulty, balancing the simplest representation encoding for the mechanism and the logarithm of the time steps to run that mechanism. Previous tests derived from Levin's Kt, such as the original C-test, are based on minimalistic Universal Turing Machines (UTM), capable of very limited representations and no previous knowledge. Reasoning large language models (LLM) are UTMs that capture a compendium of human culture as knowledge, using prompts as natural language programs. This paper presents the first test based on Levin's Kt with LLMs as representational machines. Fresh test items can be produced at high difficulties, estimated in a ratio scale measured in bits, before any human or frontier model can solve them. We formalise the measure, devise a lightweight generation and certification pipeline, explore the effect of the power of the reference machine and the gap between examiner and examinee. We produce a first battery of certified item

The paper presents a new test for evaluating AI systems, but does not make a substantive claim about AI's effect on society.

This is a methodological contribution to AI evaluation theory, not a claim about societal deployment or impact. It has indirect relevance to tracking AI capability trajectories but makes no direct claim about human flourishing, so it sits near neutral with modest upside for better measurement tools.

This paper proposes a new method for testing AI capabilities, particularly in the context of advanced future AI. It focuses on developing a benchmark that can generate tests of increasing difficulty, rather than making specific claims about AI's current or future societal impact. The content is technical and methodological.

The paper presents a method to create benchmarks that outpace current AI and human abilities. This could drive rapid AI advancement. The test's existence does not imply immediate risks, but its success could accelerate AI capabilities.

SOURCE     arXiv cs.AI (tier 1)
   ↓
DOCUMENT   0ab448dc-ef2b-439c-b029-5a2aa7c446e2
           https://arxiv.org/abs/2610.14005
   ↓
EVIDENCE   1 extracted excerpt
   ↓
MODEL RUN  4 runs, methodology 1.0.1
   ↓
SCORE      -31  (Adverse)
   ↓
CONFIDENCE 81%