I have been working as an Agile coach for one of the top-tier companies in the world. I want to share my experience working with leaders and teams in regard to Agile metrics. To give a high-level background, one of my clients has adopted the Spotify organisational model — Squads/tribe/chapters and guilds.
FORMING AND TRAINING TEAMS
In the 1st quarter of 2017, I was working an FTSE 100 company, whose objective was Agile adoption and transformation across CIO (more than 2,500 people). We had different domains and subdomains under CIO. A coach was assigned to work with each domain. We started with forming the right teams, doing the right work, and measuring what mattered. The entire 2016 was marked as Wave One — to form the right teams; enable the team members, including the product owner; and train them on Agile principles and values and the basics of Agile (mainly Scrum and Kanban). We also worked on-demand versus delivery management in parallel with forming the teams.
After our teams and workflow system were in place, we worked on capturing the Agile metrics. We agreed to capture the velocity, story points, burn-down/burn-up charts, defect trends, throughput, and cycle time as a few of the key metrics. The Agile Wave One was completed in January 2017, when we (55 coaches across the globe) had finished training 2,500 team members on Agile adoption and implementation.
During the reflection/relearning session from Wave One, we were clear on the metrics we were capturing for the end of each sprint. Leaders were keener on getting the metrics that rolled up into their respective tribes, subdomains, and domains but had no clue on how to interpret the numbers. And leaders were also trying to compare the velocity across squads and tribes; it was just a numbers game at the end of each sprint.
CAPTURING AND COMMUNICATING METRICS
I was interested in discussing with the squads what we should measure. Was measuring velocity and story points helping the squads get better? And did they receive feedback from management on the metrics the squads submitted at the end of each sprint?
The outcome of the metrics discussions was that the squads felt that it was not about measuring the velocity for each sprint that mattered; it was more important to study the trend of the velocity for each squad and analyze the variations/deviations, which would help the team get better and course correct. They concluded that the trend analysis should be done at the individual squad level and not compare the metrics with other squads. A few team members also felt that velocity was more of a predictable number, and the real measure was the value delivered to the customer at the end of the sprint. There should also be a benefits realization of the work done by the team (i.e., Product is responsible for cascading the value and benefit realization to the squad).
KEY METRICS
Metrics are used to measure four key aspects: people, customer, financial, and process. Then balance your metrics among outcome, process, and behaviour.
People: Team morale or team satisfaction; attrition rate or skills gap
Customer: End-user satisfaction or number of defects
Financial: Unit cost per transaction or total cost of whole operations
Process: Throughput or cycle time; the size of the backlog
Takeaway: Don’t focus solely on the numbers. Observe trends. Plot the data over time to determine whether there’s an improvement or fallback.
It is not always about metrics, metrics that matter to the team. Rest all is nonsense!
It is vital that metrics be maintained daily or weekly, and that you are consistent in the way you measure them.
Keep the metric simple: Measure only what matters, not more.
Make big visual charts of the key metrics, and put them up on the walls for viewing.
Leadership Tribe has been helping organisations in their transformation journey and we are here for you. Contact us directly and we can have a quick discussion over a cuppa to support your Agile Transformation.
