As an Agile coach for one of the top globally renowned Fortune 100 companies, I had the privilege to work with various clients and assist their Agile transformation. In the journey, I have often been approached by leaders with the question: how we are doing? In this article, I’d like to share a case study and my thoughts on defining and adopting Agile metrics which is a critical part of the Agile transformation. The Agile metrics help to measure not only the velocity, throughput, product quality, team health, etc. but also reveal the value delivered to customers.
Case Study
Background
In 2016, I had the privilege to work as part of a team of 55 Agile Coaches to roll out Agile adoption and transformation across a client’s CIO division which has over 2,500 people. The client’s CIO division is comprised of several domains and subdomains. It has adopted the Spotify organizational model of Squads, Tribes, Chapters and Guilds. Just for your background, the specific structure can be described in the broad sense as follows:
- A Squad is a basic unit of a development team;
- A Tribe is a collection of Squads working in related areas;
- A Chapter is a small family of people with similar skills and competencies in the same Tribe;
- A Guild is a community of people with the same interest across the organisation.
A more detailed explanation of the Spotify way of working can be found following the link, should you need more information.
Capturing Agile Metrics
We adopted the Wave Planning model which is a phased planning technique for large and complex projects like this one. The entire 2016 was marked as Wave One. We worked collaboratively with the client, formed the right teams, trained them on Agile principles, values, tools and practices, mentored and coached the teams to adopt the new way of working.
After the teams and workflow have been established, we started working with specific Squads to capture the Agile metrics. We held a few discussions with the teams and collected their thoughts. We understood that the velocity, story points, burn-down/burn-up charts, defect trends, throughput, and cycle time are some of the key metrics that are crucial to them. During the reflection sessions, we reviewed the metrics with the teams and made sure they still serve the purpose of performance measurement for the specific team.
After Wave One has been successfully implemented in January 2017, the leaders were keen to devise the metrics to the other Squads, which I have advised them not to. I had separate discussions with the targeted Squads regarding if measuring velocity and story points help the Squads to get better and if they receive feedback from management on the metrics they have submitted at the end of each sprint. The Squads suggested that it is not the sprint velocity that mattered. Instead, the trend of the sprint velocity and analysis of the variations/deviations would help the team to course correct and get better. A few team members felt that velocity is more of a predictable number, and it is the value delivered to the customer at the end of each sprint that should be measured. Some also suggested that there should also be a measure of the benefits realized for the work done by the team. Therefore, it is advised that the Agile metrics be defined and owned at the individual Squad level and not to be compared with each other.
The Agile Metrics
To sum up, Agile metrics are an important tool to help teams understand how effective they are, and allow teams to continue learning and improving processes. They need to be understood and applied properly to deliver value. Below are some of my advice to help Agile teams establish powerful metrics:
- Don’t measure everything for the sake of it.
- Management team not to impose the metrics on teams.
- Teams to adopt the metrics that matter to them and add value.
- Consider using a few metrics together to provide a comprehensive view of the Agile activities.
- Make them easy to understand and update.
- Maintain metrics regularly (daily or weekly) and be consistent in the way you measure them.
- Different units in the organisation may use distinctive metrics, what works for one may not work for the other.
- Don’t merely focus on the numbers, interpret the factors that are impacting the team.
- Ground the measurement in the agile process, observe trends and understand whether there’s an improvement or fallback.
- Make visual charts of the key metrics, and make them visible to the team.
Agile metrics provide stakeholders with the baseline of where the teams are and transparency over their journey. You may consider the measurement from four key aspects: people, customer, financial, and process. Below are examples of some key metrics that have been used by teams we have worked with.
- People: Team Satisfaction, Attrition Rate, Skills Gap
- Customer: Release Net Promoter Score (NPS), End-User Satisfaction, Escaped Defects
- Financial: Unit Cost per Transaction, Total Operational Cost
- Process: Agile Velocity, Sprint Burndown, Lead Time, Cycle Time, Throughput, Backlog Size
Are you interested in learning more about Agile transformation best practices? Are you looking for help to understand what types of metrics would be relevant to your teams? Leadership Tribe has been helping organizations of all sizes with their transformation journey and we are here for you.
Also, don’t miss to check our other articles on Agile Transformation
Contact us directly at [email protected] to see how we can support your Agile Transformation.
