DevOps-RAF: An assessment framework to measure DevOps readiness in software organizations

DevOps is a software process that embraces agility practices on software development and delivery, requires that organization meet strict culture and tooling requirements, and has become widely used by leading software development companies, as well as Latin-American Small and Medium Enterprises (SME). Several DevOps maturity models have been proposed around CAMS (Culture, Automation, Measurement, Sharing) or variations, but they do not allow to identify capability gaps in specific organizations; and altough some have been evaluated at European and USA companies, none are LatAm SME-focused. This article describes DevOps-RAF, an assessment framework to (1) evaluate SME's readiness to adapt and adopt DevOps practices, culture and tooling, (2) identify capability gaps, and (3) propose improvement roadmaps. DevOps-RAF exploits the notion of DevOps as Dev + CI + Ops, and recognizes four domains for them (Agile Development, Agile Operations, Continuous Integration) and the cross-cutting Culture; 17 key performance indicators (KPI); and four maturity levels (Initial, Intermediate, Advanced, and Optimized). This domain structure allows to assess both development organizations (doing Dev and Integration) and their customers (doing Integration and Ops). An initial validation of DevOps-RAF formulation and structure was carried with a survey to several DevOps Chilean industry experts; respondents suggested to retain 71% of proposed KPI's, change 29%, and drop none. Once pilot-tested and stabilized, DevOps-RAF will allow to systematically assess and mitigate the capacity gaps that bedevil LatAm software SME's in adopting DevOps.