Team Performance Management Part II
Having set the stage in part 1 of this post, I will now describe specifically how we’ve set it up. I should note that template we have come up with applies to all team members, no matter what their specific roles (QA, BA, or developer).
Team Performance
This is the focus of our performance management document. We using a scoring system (built into the software we use) which can be weighted, so this portion represents 50% of the score. As must as possible, we’ve included quantifiable criteria for evaluation. Some examples of the things we measure:
- the consistence of sprint velocities
- the number of sprint commitments met
- code coverage
- code ‘quality index’ (a sum of several code quality measures)
There are also some more qualitative criteria (positive and negative) which we look at, such as:
- Stakeholders recognize consistent, feasible innovation in providing useful software products
- Software products are recognized by stakeholders as meeting the needs of the business
- Customers regularly raise concerns about team performance, product quality
Individual Contribution to the Team
This section represents 25% of the scoring. Here we found quantifiable criteria hard to come by. Instead we look for certain behaviours our actions (again, positive and negative), such as:
- Contribute to team knowledge by conducting at least one learning session (lunch and learn or otherwise) on a relevant topic in the last six months
- Seek to gain and demonstrate new skills and knowledge to make yourself more useful to the team
- Clearly support the team in achieving its goals; take personal responsibility for and understanding of the team’s goals
- Provide feedback to others in a constructive and insightful manner
- Demonstrate an unwillingness to listen to others’ ideas
- Fail to share ideas and information on your work
Individual Goals
The final 25% of the performance score is reserve for individual goals. Here is an opportunity to set out specific goals and individual may wish to accomplish, or otherwise perhaps goals related specifically to their roles on the team.
And that’s it. We set this up in the summer, and we’re still actually in discussions around what some of the numbers should be, but the structure is there. It’s a little early to tell how it will work, but the concept and approach appear to have been accepted by the team, so far.
Team Performance Management Part I
Like most organizations bigger than a small business, we have a performance management process in which every employee participates. The structure of the process is focused on individual goals and performance metrics. Unfortunately, managing performance at the individual level doesn’t work well in a team environment. In Scrum, peoples’ focus needs to be on how the team is performing and delivering to support the business, not the individual. In fact, using an individually-focused performance management process not only doesn’t work, but can actively work against progressing as a team. As a team member, consider how individual performance assessments would impact your motivation to help the team look good, at your own expense … it doesn’t.
There isn’t a great deal out there on performance management in Scrum or for a team. A lot of what I’ve read just dismisses formal performance management as a waste of time or counter to Scrum best practices. This is especially true when the Scrum Master, who should be in a supportive role for the team, is asked to conduct or participate in performance reviews of team members: it doesn’t work very well. But the reality is most of us have to adhere to corporate policies, and therefore have to follow the process our HR groups have laid out. My current situation in thinking about this problem is that I am not the Scrum Master, and some of the team members report directly to me, and some report to a manager who works under me. I am in a situation where I have to do performance reviews, though, and where I need to these to support team performance. So, what do to, what to do … .
There are a couple of critical issues to deal with: (1) the process itself, as well as (2) any incentive plans that are tied into it. Fortunately (well … maybe not so fortunately) my organization doesn’t have a high-valued bonus system tied into personal performance, so at least I’m not working against that. For this post, I’m going to focus on the second issue: building a performance management document which supports team work.
My perspective on the performance management process is that is gives a manager two tools: (1) a framework to have conversations (sure, that’s something you shouldn’t need a framework for …), and (2) a document which describes how or what a staff member needs to accomplish. This second point is the critical consideration for framing an effective performance management document. It needs to be effectively written to support team-focused work.
Written properly, the document can provide appropriate incentive to perform in different ways, although this depends on the stakes involved (e.g. is there significant financial incentives at stake?). My experience is that different people respond differently to this document — some argue every point and demand concessions, even if it isn’t a high stakes process, and other are more relaxed about it. Nonetheless, even in a low-stakes situation with employees who are laid back about the process, the document and how it’s structured can impact behaviour.
So now we arrive here: how does one craft a performance management document which effectively supports a team focus for individual members? I’ll describe my take on this in the next post.
I’ve just finished reading Rework. I follow the 37Signals blog pretty closely and agree with most of what they say, so I (like many, many other people) have been eagerly anticipating the publication of this book for several months. I read their last book, Getting Real, and loved it.
What did I like about Rework? At it’s heart it supports lean and agile concepts really well. The authors advocate working at a sustainable pace, a lack of detailed, long-term planning, going for quick wins, and making decisions as late as possible, for example.
It’s also an easy read. I read it mostly during my commute to and from work, so in 20 minute blocks, and I got it done in a few days. It’s easy to absorb and is written in a casual style (another lesson from the book: “Sound like you”). Each chapter is broken into discreet essays which I don’t think ever exceed three pages. It’s a great way to break up a book.
The most important thing: they’re right. They take apart so much of what goes on in a typical business environment and tell you it doesn’t have to be that way. They tell their readers that things can be done better and simpler. I think it would take a lot of courage to follow everything they suggest — and in a lot of companies it would be impossible without great leadership at the top, but I’d be surprised if there aren’t a few things anyone can take away and start using, even in the most structured corporate environments.
One thing I found frustrating was that it was a little too light on details for my taste. It felt thin (which I suspect is something they were going for). In rare instances they would back up their statements with examples from other companies, which helped to add some weight to what they were saying. I would have loved to have seen more of that. That ‘thin-ness’ presented many of the things they write about an almost off-handed nature, which undermined some valuable ideas.
I’ve read reviews describing Rework as a recipe book for running a business. I would disagree, it’s not a recipe book. I think it’s closer to a philosophy book. I also think there may more to the book than I got out of my first reading — I have no doubt I’ll pick it up again in a few months and have another look.
Overall, an excellent read — especially given the low time commitment. With so much packed into it, why wouldn’t you read it?
Agile PM Scorecard
… or the six things I review with my steering committee.
Every three weeks I meet with my steering committee to review the status of our system redesign project. Over the course of a year what we talk about and the information I provide has changed. We started talking in a more general sense on scope, budget, and quality, but in many cases we lacked real statistics to support our conversations. As well, given how much I was hammering the importance of quality, my committee wanted more quantifiable data to support what I was saying. I finally hit on the idea of a pseudo balanced scorecard to show project data. After a few iterations we’ve finally settled on the most valid and concise data which works for us:
- Budget. This is an obvious one. Each sprint we spend money on resources (mostly). I report on actual versus planned.
- Scope. This is a little more complex, but the use of Version One helps. The scope of the project is tracked in story points. Specifically, we look at the total remaining scope from the last report, then do some math: (1) story points (SP) added through new stories, SP removed (through re-estimation or backlog grooming), and SP burned down — leaving us at a new total. What we’re looking for here is a trend of falling scope, obviously.
- Remaining Scope per Release. The project is divided into somewhat-logical production releases. Our production releases are set at the quarter level, although the scope may or may not fit into that time frame. We use this data to manage the project / product at the release level, determining if we need to redefine the scope of the release, either adding or taking away functionality. As part of this, we estimate the total sprints remaining to complete the release; this includes a best guess at taking into account unknown and yet-to be added SP (our unknown unknowns).
- Un-estimated Stories per Release. This is so we know what’s out there, and represents one aspect of risk against the releases.
- Quality. Reporting on quality is probably the most difficult part of the scorecard. It is challenging to explain to executives the intricacies of software quality, and coming up with straight numbers to quantify the system’s quality isn’t easy, either. That said, for better or worse I show them: (1) code coverage, (2) defect count, and (3) a ‘quality index’, which is a formula I developed from statistics generated out of our continuous integration style checker.
- Velocity. This is reported in the first section, but we expand on it here, and include a rolling average. This is often the focus of our discussion. I’ll leave it at that
Overall this makes for a meeting where we can get into some good discussion as to why some data looks good or bad. I can use it as a focus to ask for additional resources or help, or we can discuss realistic target dates around functionality. The level of transparency this project has is beyond any other project in our organization, or any project I’ve every run. Sometimes is makes for some difficult conversations, no doubt, but those conversations can be had sooner rather than later.
Thinking about Scrum and Blogging
Clearly I haven’t posted in a while, which is of course breaking one of the cardinal rules for blogging: post often. I have enjoyed blogging over the past year and have found it helpful in thinking through some of the major challenges I experienced in learning about Scrum and trying to lead my department to greater agility. I found as my difficulties decreased so did my blogging. I am re–thinking how I use this blog and website. In the mean time, I’ve listed the top posts since starting the blog. I will take these to be topics people are interested in, and have already started rewriting some of them with some more detail and the added benefit of a little more experience.
Hiring …
For the first time since we adopted Scrum, I am looking for staff. Specifically I’m looking for a new development manager (the current one is moving out of country) as well as a programmer. My perspective on hiring has changed dramatically in the past year.
For starters, I believe it’s important to find people who’ve worked using Scrum and XP practices before. For my manager, I need someone who can coach the team, and for the programmer, I need to know that he or she has ‘drunk the agile cool-aid’ — I don’t want to have to do a sales job on working this way. I’m finding this has replaced technical skills as my primary evaluation criteria. I just hope I’m not being too unrealistic.
I’m also looking forward to a different type of interview. I’ll still conduct the ‘get-to-know-you’ interview, but I’ve become more interested in the growing trend of more realistic interviews — for example pairing programmer candidates so see how they approach a more realistic task and environment. I know there will be limitations to this, but it’s a step beyond hiring someone based on how they do answering your questions in a board room.
So far it has been an interesting experience. The job market in Vancouver is obviously slower than it has been, which means I’m seeing more resumes than I’ve seen in the past (probably 20-25 for the manager and 30-35 for the programmer). What I’m not seeing is quality — most people are wildly under-qualified or ‘wrongly-qualified’, making me wonder if they’ve even read the job description. Every time I hire I experience this problem, but I guess I’m just seeing it on a bigger scale with more candidates on the market.
Right now my biggest concern is that I’m being too unrealistic. Will I ever find a programmer who can write a cover letter addressing the qualifications I’ve asked for? Will anyone who actually has actually worked as programmer in the past apply for the manager role? Only time will tell … .
