TL;DR
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
AI SRE tools are gaining attention as companies report more deployments and on-call pages, with vendors and some engineering teams building agents to investigate incidents. Current systems can help triage and gather evidence, but fully autonomous production fixes remain more a conference-demo vision than an established practice.
AI-assisted incident response is moving from conference demos into vendor products and some in-house engineering projects, according to The Pragmatic Engineer’s report on tools from Rootly, incident.io and ResolveAI. The systems are primarily being used to speed up incident investigation and alert triage; the report says autonomous agents that diagnose an outage and safely ship a production fix without human oversight are not yet a routine reality.
The report links growing interest in AI SRE tools to rising software change rates. Its author says many technology companies have seen pull-request frequency at least double over the past year as AI coding agents contribute more code. The report describes a basic reliability relationship: more changes can mean more opportunities for a change to cause a failure, even when teams also improve testing and verification.
Rootly data cited in the report shows on-call pages are up compared with a year earlier, with pages outside business hours nearly doubled over that period. Those figures are attributed to Rootly; the supplied material does not give the underlying sample, exact counts or a broader industry-wide measurement. The report also names OpenAI’s Sevbot and an in-house project at ING as examples of organizations working on AI incident response.
In practice, the tools described use agents to collect information and investigate likely causes, often within Slack or Microsoft Teams. Some systems split work among multiple agents, draw on company-specific documentation and incident history, or let engineers steer the investigation. Whether an agent should go further and make code changes is contested. The report says teams use methods such as backtesting, evaluations, multiple models and adversarial checks, while stressing that accuracy is difficult to measure and depends on the company and engineering team.
More Pages, Different On-Call Work
If software changes and pages continue to rise, incident response tools could help teams handle more alerts without adding the same amount of manual investigation. Faster evidence gathering and alert triage may reduce time spent locating logs, previous incidents and likely causes. Some systems are also being used to suppress or triage less important pages, which could eventually mean fewer interruptions for engineers.
That does not mean the difficult work disappears. The report expects incidents that reach human on-call engineers to remain challenging, and it flags the possibility of deskilling if engineers depend on agents without practising incident response themselves. It also points to a new risk: AI systems and AI-generated changes can themselves contribute to outages. For engineering leaders, the practical question is not simply whether an agent can produce a plausible diagnosis, but whether it can do so reliably in their own systems and under real incident conditions.
Products Worth Considering
Samsung 27-Inch Odyssey QD-OLED G8 4K 240Hz Gaming Monitor
LG 34GX90SA-W 34-inch Ultragear WQHD OLED Curved Gaming Monitor
Samsung 32-Inch Odyssey OLED G8 (G80SH) 4K Gaming Monitor
As an affiliate, we earn on qualifying purchases.
From Build Demo to Incident Tools
At Microsoft’s BUILD conference in May 2025, a demonstration of Azure SRE Agent showed an alert triggering an investigation that identified a root cause and shipped a fix without waking a human on-call engineer. The report’s author said the scenario seemed artificial and questioned whether engineering teams would allow an AI to make production decisions without oversight. The report says there was little visible discussion of SRE agents after that event until more recent activity among vendors and technology companies.
The term “AI SRE” is being used for AI-augmented on-call tools, although the report says vendors themselves consider it an imperfect label: site reliability engineers do much more than respond to alerts. Its reporting is based partly on conversations with Rootly, incident.io and ResolveAI. The author disclosed an investment in incident.io and said they were not paid to mention any company. That relationship is relevant when weighing the report’s vendor-related descriptions.
““Incident rate = [Change rate] × [Probability].””
— The Pragmatic Engineer report
Evidence Behind the Page Counts
The figures cited for increased pull requests and on-call pages are not enough to establish a single industry-wide trend: the report provides no full methodology, sample size or exact page counts for Rootly’s data in the supplied material. It also does not provide comparative, independently verified results showing how much AI agents reduce investigation time, prevent incidents or improve outcomes.
Performance depends on a company’s systems, data and definition of a successful investigation. The report says measuring accuracy is difficult and team-specific. It remains unclear how often these tools make incorrect suggestions, how companies govern access to production systems, or what safeguards are used before an agent proposes or executes a change. Vendor descriptions and individual company projects should not be treated as proof that unattended remediation is broadly safe or effective.
Testing Agents in Real Incidents
The next test for AI SRE products is whether teams can show reliable results through evaluations and backtesting based on their own incident histories, rather than relying on polished demonstrations. The report describes approaches including multiple agents, model checks and company-specific context, but does not identify a shared industry benchmark or a timeline for broader autonomous remediation.
For now, the described direction is toward agents that help investigate and triage while engineers retain responsibility for consequential decisions. Teams will also need to monitor whether fewer low-priority pages are offset by harder incidents, and whether on-call skills remain strong. The report suggests incident-response simulations as one way to keep engineers practised as AI tools take on more of the initial investigation.
Key Questions
What does “AI SRE” mean?
It is a loose label for AI tools that support reliability work, especially on-call alert triage and incident investigation. The report notes that the term is imperfect because SRE work extends well beyond responding to alerts.
Can AI SRE agents fix production incidents on their own?
The report says fully autonomous systems that diagnose incidents and safely ship production fixes without human oversight are not routine today. Current use described is more focused on investigation and triage, with human involvement remaining a key part of the process.
Why are companies looking at these tools now?
The report points to increased software changes, including higher pull-request frequency at many companies, alongside more on-call pages in Rootly’s data. More changes can create added reliability demands, making faster incident investigation attractive.
Do AI SRE tools reduce outages or on-call pages?
The supplied report does not provide independent evidence that the tools reduce outages overall. It describes page triage and investigation as current uses, but says accuracy is difficult to measure and varies by company and team.
What risks should engineering teams watch?
Teams need to consider inaccurate diagnoses, unsafe changes, dependence on vendor or company-specific data, and the possibility that engineers lose practice responding to incidents. The report also points to AI-related outages as an emerging category.
Source: rss
