DevOps metrics and KPIs: A multivocal literature review

Abstract

Context: Information Technology organizations are aiming to implement DevOps capabilities to fulfill market, customer, and internal needs. While many are successful with DevOps implementation, others still have difficulty measuring DevOps success in their organization. As a result, the effectiveness of assessing DevOps remains erratic. This emphasizes the need to withstand management in measuring the implementation process with suitable DevOps Metrics. But what are these metrics?

Objective: This research seeks to provide relevant DevOps Metrics to facilitate the efficiency of DevOps adoption and better analyze DevOps performance in enterprises.

Method: A Multivocal Literature Review (MLR) is conducted, with 139 documents gathered and thoroughly examined from throughout the community, including books, scientific articles, white papers, conferences, among others.

Results: This article conducts an extensive and rigorous MLR, contributing with a definition of DevOps Metrics, 22 main metrics, their definitions, importance, and categorization in sets of Key Performance Indicators, as well as exposing clear indicators on how to improve them. It is also discussed how metrics could be put into practice and what constitutes a change in the context of DevOps Metrics. The study’s outcomes will assist researchers and practitioners understand DevOps Metrics and how to better implement them.