Criteria for Success

I was having a conversation with a client recently and the topic of "11th hour" came up. He seemed to think it wasn't possible to deliver a software project without some sort of "11th hour" panic. I disagreed

I disagree because I've done it on small and large projects. Don't get me wrong; it's very easy to lose sight of things and easily get a project into a state where "11th hour" panic is inevitable. But, it's not something that can't be avoided.

One of the problems with software projects, it seems, (and with other projects, I suppose) is losing sight of things and not having a "destination". This is what I refer to by "criteria for success".

Criteria for success is basically the criteria by which what is expected to be delivered will be evaluated.

This criteria no only gives you a focused destination, it also provides a focus for the team. Without some sort of criteria it's hard to gauge whether the team is side-tracked on something that doesn't add value to the project. Criteria for success not only gives you a destination to strive for; but also means your efforts can be focused on that destination and be able to get to that destination quicker. This means the project can be completed quicker and progress can be judged more accurately.

Without defining criteria for success, software projects that do succeed succeed by accident. I don't know about the rest of the industry; but I'm not really comfortable with completing my work "by accident".

Defining "Done" is a very similar. Defining done is another popular mantra; but, I think "criteria for success" is more clear in its goal in that it's almost explicit that acceptance criteria is what we're looking for. "Done" can be a little wishy-washy and open for interpretation.

I don't think it's enough to complain that people have a problem defining success or "done"; so, I think it's also important to help people get their own criteria for success. i.e. people have a problem coming up with criteria for success because no one has defined criteria for success for defining criteria for success—so to speak.

There's various tools to help with this sort of thing; the most obvious is Agile. Defining user stories and getting acceptance from stakeholders on these stories is an excellent first step. Many people simply view user stories as a check-list of items. This can be extremely useful; but explicitly defining criteria (acceptance criteria is the more general term) isn't an explicit task for most people.

Often we can't expect out stakeholders to know how to define the acceptance criteria. "Just do it" or "make it work" are often the definition of done—which doesn't leave us much in the way of unambiguous.

Agile kind of get's away from "requirements" because requirements are often associated with waterfall methodologies and people expect that "requirements" mean a full set of requirements for the project before the project commences and doesn't change for the life of the project. This really isn't what requirements are; but, we'll continue using terms like "criteria for success" and "defining 'done'" to avoid these associations. But, theories and practices for effectively defining requirements can give us some valuable tools for defining good criteria. Karl Wiegers has a couple of excellent books that has some great criteria that we can reuse for our criteria:

Correct

Is the criteria correct (easy to say)? Is how we're judging the deliverable right? If we're judging x against y, is y what will actually be used as the criteria? If it's really z, we can't deliver something that satisfies y on purpose.

Feasible

Is the criteria even attainable? If we simply can't reach that criteria, we're going to fail… This can be a bit tricky because we don't always know whether what we're actually working is actually possible. With Agile, we like to separate what is known to be possible from what is not known to be possible with "spikes". We purposely research things we're not sure are possible (or not entirely sure how long things may take) into spikes so we can gather the information we need to define our criteria for success correctly. This mitigates the risk and gives is the ability to separate and time-box these aspects of the project because if we're not really sure if it's feasible or how long it's going to take we need to prioritize it differently and explicitly.

Necessary

This is kind of obvious; but without this explicit criteria our criteria may lack focus and our work not be prioritized correctly and we end up working on tasks that are not important (i.e. not necessary). It's critical that the criteria actually define necessary qualities.

Prioritized

I've already alluded to this; but prioritized criteria is essential for getting a project done properly. "Prioritized" is a bit ambiguous; so, it's important to define what we mean by prioritization. The most essential part is that criteria is prioritized in relation to other criteria. There's a tendency for everything to be prioritized as "important" and this has the opposite effect because it also means everything is defined as "unimportant" and the order that tasks get performed is open to interpretation (ease of implementation, interest in the task, coolness of the task, etc. none of which are stakeholder criteria). If it's not important enough to prioritize a task correctly, it's not important enough to actually perform the task.

Unambiguous

It may seem obvious; be we need to make sure our criteria is not ambiguous. It's easy for criteria to be open for interpretation and end up being ambiguous. See the next point…

Verifiable

The criteria that Weigers' details is that the criteria needs to be verifiable. It's not enough to say "done" or "works", we need to be able to unambiguously and empirically verify our work against specific criteria. Criteria like "works" just doesn't cut it. This is stakeholder-driven. Often we'll need to help our stakeholders think through what they're ultimately going to use for verification—we want them to think about this before we begin the work, not after (which is very common).