Last year, I had the pleasure of helping deliver a new product for a local legal electronic discovery company with a great team using a strict SCRUM development model. We evaluated several Team Foundation System 2010 work item templates, and  selected the (then in Beta!) Scrum for Team System V3 template built by EMC Consulting. This posting reviews the main pros and cons we experienced with the template.

The V3 system represented a major leap forward in the complexity, flexibility, and sophistication of work item templates for TFS 2010. We made minor modifications  to add some metadata to the product backlog item template, but otherwise used the template pretty much as is.

For a basic overview of the template concepts, there's some great process guidance documentation that comes with the template. Key things to understand include:

• The idea that the work items are related to each other, and that some pieces of data roll up and/or down in the relationships. Product Backlog items, for example have a Work Remaining field which is derived from the sum of the sprint backlog task work remaining fields that are linked with a 'Implemented By' relationship. Entities are related, and the relationships are more than just labels! They have embedded rules inherent in their behavior.

• Certain work item entities have to actually exist in order to relate them. A sprint, for example, is not a metadata field. It’s a work item entity. So, while it may look like you are putting a product backlog item into a sprint because you've edited the metadata on the project to include that sprint in the tree, unless you have created an actual work item of the type sprint with that same name, you wont' be able to see aggregated sprint data. Make sure you understand the containing work item entity tree, and that it is done before you start managing the real meat of your product (product backlog items!).

Installation and Setup

Installation was pretty straight forward, but the update service installation didn't work initially, and we didn't even notice. Following the installation instructions was straightforward, given the right access level .  All in all, no complaints with the installation.

Configuration of the template was time intensive, however. The early versions of the Scrum Masters workbench was unstable, not particularly intuitive, and only seemed to edit project metadata, and not actually create the work item entities we needed. We aborted mission on the workbench tool, and were able to get everything configured through the Team Explorer interface.  I personally don't love tear sheets on the wall to manage a backlog, so I was motivated to translate all of our RIP session sticky notes and wall sheet notes into the template quickly. Editing the WIT through Visual Studio in the normal way worked well, and we were able to add appropriate role and  business value fields quickly. Crispin's Blog on getting started focuses on using the workbench tool, but if it doesn't work for you, just create the release, sprint, and team sprint entities manually so you can get to creating stories.

While this template really helped with making team interactions easier (at least once the template was setup), the toughest, most important exercise for the Product Owner, prioritizing the backlog, was not something that was made particularly easy. No template that I've found has a good solution for it, but this template follows the pack. The core issue is with the idea of 'business value'. It is a simple numerical field that divides business value by story points to give you an ROI measurement. In theory the Product Manager would simply sort the backlog by ROI, set a delivery order for like ROI's and be done with it. The reality is that determining business value is like mystery-magic-voodoo that ends up being entirely subjective, so the prioritization ends up being a simple reflection of the product owner's short term preferences for what is important. Unlike everything else in SCRUM/Agile, this prioritization system is, in my experience, not based on objective data, which is unfortunate.

Care and Feeding

Using the template day to day through the Team Explorer interface worked fine. Planning and prioritization exercises worked fine in the backlog listings, or in Excel, and developers were able to easily bite off and knock out work items without too much trouble. Probably the hardest thing we had trouble staying on top of was writing acceptance tests for product backlog items. The designed workflow assumed that you would only be looking at product backlog items that had acceptance tests associated with them (an excellent practice!) for prioritization, but we never really got ahead of the curve on that front. Acceptance tests were usually written during design and implementation.

Generally, the team felt that the amount of process template interaction required to test, log bugs, fix bugs, and resolve them back for verification was complicated. The bugs themselves don't have any effort calculations in them, so  the expectation is that when a bug is logged, a PBI would get flagged as 'broken', and that the team would estimate and create a new sprint backlog task with effort hours in it to fix that particular bug. That means a /lot/ of sprint backlog tasks, with effort data that was pretty inaccurate at a low level (the bug fix effort data averaged out okay). The nice part of this system is that you are having to plan and prioritize your bug fixing effort against the rest of your feature development backlog, but it also means that you are requiring a lot of developer 'documentation' for any work that gets done.  The team felt it was time consuming, but it also protected them from over-committing.

The update service, which is a great theoretical design, was flaky for the entire four months we were using the template. Calculated fields, and  relationship updates (failed acceptance tests 'breaking' PBI's, for example) were not reliable, and this meant a lot more manual updates and verification than we really wanted. The Scrum Master had to make sure that all bugs, acceptance tests, sprint backlog tasks, and PBI's were up to date at least 3-4 hours in advance of those report views so that the updates could wend their way into the data warehouse (which is on a two hour update cycle for some information).

The last thing I would note about the template, that isn't the fault of the template, but of common practices in SCRUM/Agile shops is that you should note give Epic stories (ones where you really have no idea what the effort is to build them) enormous story point values (like 1000). It is available to you as a value, but don't use it. It'll make your cumulative flow diagrams worthless for most projects. Since most epics don't decompose into 1,000 points of stories, as soon as you decompose that huge story into its constituent parts, your cumulative flow diagram (which will faithfully represent your backlog size over time!) will look like a roller coaster. If you don't know, make it a 100 point story and decompose it something 40 points or smaller as soon as you can.

Reporting

SCRUM/Agile is inherently a data driven development methodology. To make decisions based upon data, you need access to it, and this template delivers on that front in spades! There are about 19 reports out of the box, and all of them are fairly presentable. The MSF for Agile template has more attractive reports, and they are more data rich, but there aren't as many relevant ones (and their work item template isn't nearly as nice). 

There is a lovely Excel export that summarized the product backlog makeup with pivots across all the metadata. This was great for the product owner to evaluate the backlog composition and figure out where the gaps were!

The Cumulative Flow diagrams (for both the product and sprint backlogs), in particular, was very powerful. This report is what sold us on this template.

Product Cumulative Flow

Sprint Cumulative Flow

In Summary

Positives

• Great for big teams, geographically spread out teams, or multiple teams working on concurrent efforts.
• Great reporting!
• Easy installation.

Negatives

• Complicated process for testing/dev interactions.
• Initial configuration time consuming.
• Update service flaky.