Forecasting: Is it EVIL in Agile Portfolios?

*Part one of a two-part series

I’m often quite wordy in my blogs. I’ll pose an initial question in the title, throw out a thousand words or so, and then present a conclusion. I’m not going to do that here. Instead I’ll be much more succinct.

Is forecasting evil in agile portfolios?

YES!

The context for this conclusion and subsequent discussion is three-fold:

  1. Forecasting when you are just building your agile teams OR are in the early stages of an agile transformation
  2. When you’ve been doing agile for awhile, and you’ve become overconfident with your capacity awareness.
  3. Forecasting, in this sense, is anyone determining how large or how long something will take and NOT fully engaging their team in the estimation; planning and forecasting.

The Traditional Way

In my experience, the following IS the way traditional projects have been forecasted. Usually, a small group of product folks will get together with technical managers, a project manager, architects, and perhaps a technical team lead or two.

Often QA and other functional team representation are left out of the mix with the thinking being that the developers can estimate “for” them. The product folks present an “ask” to this small team and they estimate the level-of-effort associated with the functionality. If it’s a Waterfall project, then who (# of Dev, QA, etc.), how long (days, weeks, months, transitions or workflow, etc.), and output (lines of code or components, tests, documents, etc.) are the usual units of estimation.

In agile contexts, the same things occur. However, the estimation units usually change to be specific numbers of small agile teams, velocity or capacity, and number of iterations.

These folks are not destined to do the work, but they set an expectation for the work and then go back to their additional work. At some point the “ask”, if the cost and ROI are deemed worthwhile, is handed off to a set of teams to deliver. Often, the view is that the “hard work” has already been done, and there is simply “execution” left for the teams.

The Problem

My key problem with all of this is that someone else is estimating for, and let’s be real here – committing for, the teams. There are often many excuses. Here are some I often hear:

  1. The team is working on something important right now. We simply don’t have the time to interrupt them to estimate another project. It would COST too much to do that as well. What we’ll do is select a small set of high-skill team members to do the pre-work before we get the entire team engaged.
  • This project is WAY too big to get everyone together. We do Enterprise-level projects around here. We have large numbers of teams AND many of them are distributed around the world. There’s just no way that we can get EVERYONE together.
  • This project has new technologies and new approaches intended. The team doesn’t have a clue about this, but Bob, the architect does. We’ll get Bob and his team to “iron out” the hard-bits in advance of the team’s execution. Bob can even run some “Lunch & Learns” as a way of passing off technical knowledge in the beginning.
  • We’re trying to prioritize our entire PORTFOLIO right now and we need high-level level-of-effort estimates to do that. There’s no way we can ask the entire team to estimate 10-15 major initiatives.  Of course, we’ll give each team the opportunity to pull together a more realistic estimate before we start execution.

Now, there is validity to all of these points. I truly understand the balance in getting things done (current project work) versus planning & forecasting (determining future capacity). If our goal is to accurately forecast and best understand our projects, then I still feel the best way to do that is to engage as much of our team targeted towards the work, as early as possible, so that we get the most realistic estimates.  In the next part, I will explain my five reasons to engage in forecasting.