The Irony of AI SREs: Automation May Erase Engineering Intuition

The rise of AI-assisted incident response tools—often termed “AI SREs”—is transforming how companies manage infrastructure, but it may be creating a dangerous skill gap for the humans left in charge.

As Sylvain Kalache first reported, these tools now possess the capability to inspect alerts, query telemetry, correlate deployments, and implement fixes autonomously. While this automation reduces the average mean time to resolution (MTTR) for routine issues, Kalache, AI Labs lead and DevRel at Rootly, warns it is stripping engineers of the very experiences they need to handle high-severity, ambiguous incidents.

Kalache argues that routine incidents serve as the primary way responders safely develop an intuition for how systems behave and fail. When AI handles these common tasks, engineers lose the practice necessary to intervene when automation fails. This phenomenon mirrors a concept described by human-factors researcher Lisanne Bainbridge in her 1983 paper, *The Ironies of Automation*. Bainbridge posited that automation reduces opportunities for operators to practice routine work, yet still leaves them responsible for abnormal situations, meaning those operators actually require more training than they did prior to automation.

Kalache points to the aviation industry as a model for mitigating this risk. While planes are highly automated, pilots must still manage rare, critical failures such as engine shutdowns or stalls. Under US FAA rules, captains undergo recurrent training or proficiency checks every six months to rehearse these emergencies in simulators. Kalache cites TransAsia Airways Flight 235 as a cautionary example: after a propeller autofeathered, the crew misidentified the problem, leading to a crash just 117 seconds after the initial warning.

In the software world, Kalache suggests this lack of hands-on experience leads to "comprehension debt," a widening gap between system complexity and responder understanding. He predicts that while routine fixes will speed up, resolution times for complex incidents will likely increase because responders have lost touch with their systems.

To combat this, Rootly has partnered with Uptime Labs to create realistic incident simulations. These simulations place engineers in the role of incident commander during a simulated e-commerce outage, requiring them to use observability tools and coordinate with LLM-powered stakeholders in Slack. Kalache emphasizes that while AI can explain its diagnostic steps, passive observation is no substitute for active practice, noting that hands-on education is superior for troubleshooting skills.