Evaluating Self-Reflective LLM Agents for Autonomous Kubernetes Troubleshooting
IEEE Access, cilt.14, ss.161397-161415, 2026 (SCI-Expanded, Scopus)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 14
- Basım Tarihi: 2026
- Doi Numarası: 10.1109/access.2026.3743626
- Dergi Adı: IEEE Access
- Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Compendex, INSPEC, Directory of Open Access Journals
- Sayfa Sayıları: ss.161397-161415
- Anahtar Kelimeler: AIOps, autonomous operations, DevOps, incident remediation, Kubernetes, large language model agents, Reflexion, self-reflection
- TED Üniversitesi Adresli: Evet
Özet
Resolving Kubernetes incidents demands considerable operational knowledge, and the dependability of large language model (LLM) agents for autonomous remediation is unclear. This paper examines whether Reflexion, a verbal reinforcement learning method driven by self-produced failure analysis, improves LLM agents on Kubernetes troubleshooting tasks. We built a Reflexion-based remediation system with a deterministic programmatic evaluator and assessed it in 1,500 controlled runs over five Claude models, ten fault scenarios, and three configurations (one-shot baseline, two-try without reflection, and full Reflexion with at most one reflection per run). Permitting a second attempt raised overall success from 38.6% to 68.4%, whereas adding Reflexion increased it only to 69.2%. Reflexion slightly reduced single-fault success (-2.0 percentage points). Its 12.0-point gain in the multi-fault cases rests on one double-fault and one triple-fault scenario and is therefore a preliminary observation, not a main finding. Post-hoc controls on the three currently available models qualify these results. Forwarding the evaluator’s verbatim failure reason without reflection outperformed full Reflexion, whose reflection module had not received that reason, on identically instantiated multi-fault scenarios (76.4% versus 26.0%). Given the same reason, reflection matched the feedback-only control (79.1%) but did not exceed it, so the gap was informational. Allowing the actor to verify its own fix within one attempt recovered about half of the retry effect on single-fault scenarios (67.5% to 80.8%), while the externally verified second attempt remained superior (92.5%). Mid-capability models benefited most, weaker models did not use reflection effectively, and the strongest model showed a ceiling effect. The results do not support routine self-reflection after every failure; a verified retry and the evaluator’s raw feedback should come first, and any benefit for complex incidents remains preliminary.