What AI gets wrong and what failure teaches us

Jennifer Neville is a partner research manager at Microsoft who’s built a career around understanding and advancing AI for real-world use, and much like the human-AI interactions she’s been studying, her early-career path was multiturn: math, then physics; cognitive science, then work; and finally computer science—despite her best efforts to avoid the field. In this conversation with Principal Applied Scientist Chad Atalla , she explores the role evaluation plays in pushing the performance boundaries of today’s AI systems to meet user needs and the “surprising failures” that emerge when models are tested beyond traditional benchmarks. Neville also shares practical guidance for working with current AI systems and discusses why looking closely at data matters when results defy expectations, and what decades of AI progress have taught her about predicting what comes next. From an unexpected career trajectory to the frontier of AI interaction and learning, this episode asks a larger question: what can we learn when the path—whether human or artificial—doesn’t unfold the way we expect? Subscribe to the Microsoft Research Podcast : Apple Podcasts Email Android Spotify RSS Feed Transcript [MUSIC] JENNIFER NEVILLE: There were times that I felt like I was spinning my wheels. Things weren’t working. I would get, kind of, dejected. And then we’d get to a point where we did learn something, and it was just the, the emotional thrill of that was, that was really what hooked me. The being able to, kind of, understand something that no one else understood yet … CHAD ATALLA: Sure … NEVILLE:

Item discusses AI improvement through evaluation and learning from failures.

This is a podcast introduction and career profile with no concrete findings, statistics, or specific claims about AI's societal impact. It references unspecified 'surprising failures' without detail, making it essentially a promotional/biographical piece rather than substantive evidence.

This item is an interview abstract discussing AI evaluation and unexpected failures. It describes ongoing research aimed at understanding AI limitations and improving performance. There are no direct claims about AI's societal implications, positive or negative, only a focus on research methodology.

The text discusses failures in AI and the importance of evaluating and understanding data. It does not provide specific evidence of harm or benefit to humanity. The implications are uncertain.

SOURCE     Microsoft Research (tier 1)
   ↓
DOCUMENT   b861faab-589e-43b7-8378-6e01b691159d
           https://microsoft.com/en-us/research/podcast/what-ai-gets-wrong-and-what-failure-teaches-us
   ↓
EVIDENCE   1 extracted excerpt
   ↓
MODEL RUN  4 runs, methodology 1.0.1
   ↓
SCORE      -24  (Adverse)
   ↓
CONFIDENCE 50%