[T]here are known knowns; there are things we know we know.
We also know there are known unknowns; that is to say we know there are some things we do not know.
But there are also unknown unknowns – the ones we don't know we don't know.
- Donald Rumsfeld
I'll dare to be very simplistic and state that most projects can be abstracted into 3 main stages – determination of a goal, advancement towards that goal, and completion of the goal. There's lots to debate in that tiny framework: the time-frame, the notion of “complete”, how pure the goal remains through the process, etc. but let's run with this for a bit.
Most projects start with the determination of a goal. This may be directed towards solving a problem (“People are copying music and we want to stop it”) or maybe to create/design something new (“What does the next form of online communication look like?”). Assuming it is agreed that this goal is worth pursuing, an effort is undertaken to reach that goal. In some cases the goal is not really achievable – your project is an ongoing effort to get as close to the goal as possible – but you have a direction that you're heading in. At the end of the project or at a useful point in time (say, the release of the next version) you look back and reflect on how well that goal was being worked towards. You may then continue your efforts, refactor the goal or just dump the project.
Newell and Simon (1972) described that middle stage (advancement towards the goal) as a space through which problem-solvers must traverse to achieve their goal. It is in this problem space that we often see the trouble begin.
Consider a project that you've been on recently – did you experience staff changes, political pressure, changing technologies, ideological differences, technical hitches or just anything that made the traversal go off in a different angle to what you'd expected? If you didn't then you probably ran the project in your own shed, with the door locked. If you did then the map of your traversal features a few detours, renegotiations and stop signs. Understanding (or at least generally comprehending) the space through which your project is traversing is the difference between watching your project fall from the sky like a flying piano and getting somewhere near your goal.
In their 2003 paper “The new dynamics of strategy: Sense-making in a complex and complicated world”, Kurtz & Snowden state that “we challenge the universality of three basic assumptions prevalent in organizational decision support and strategy: assumptions of order, of rational choice, and of intent”. Having thrown down (or taken up) this gauntlet, they present their Cynefin sense-making framework. This was modified slightly in an HBR piece in 2007 and makes for a useful reflection point when you just can't see the forest for the trees.
Cynefin describes four domains: Known (Simple), Knowable (Complicated), Complex and Chaotic. I find these to be useful reference points on projects as they beckon me to review the project environment and explore why things are going the way they are. I also like the fact that Cynefin isn't the same old 2x2 matrix that every second analyst seems to barrow out whenever they want to describe an optimal outcome. As Kurtz & Snowden indicate, there is no optimal state in Cynefin – you can't make everything stable (simple) and nor should you want to.
In the Known domain we see standards, well established procedures and even best practice. If we feed in input X, we get output Y and we always get output Y. In terms of technology projects, this domain invites ERP and off the shelf solutions. You probably use more traditional, structured approaches to project management, based on the SDLC.
The Knowable domain is less structured than the Known but, with the help of experts, you can find a pertinent pathway. We need to engage with the right people in the organisation to understand how and why things are. We can acquire systems that can be customised to achieve our goals and the project can be reasonably structured but we must be aware that eliciting expert knowledge is rarely a straight-forward brain dump.
Projects in the complex domain are probably my favourite. There aren't any really solid rules and you need to search for themes, commonalities and patterns as you seed ideas and check the response. In this domain there's probably not a lot of single, off the shelf titles that will meet the goal requirements. Instead you call on a mix of existing software, development, data exchange and training to work towards the goal. This, to me, is the domain in which agile practices fit well. Quick iterations, frequent demonstrations and releases, and close client contact allow the team to see how effectively they are traversing the problem space.
Chaos can be a difficult domain in which to run a project. Chaos can take form when you've suddenly dropped out of one domain due to an unplanned "happening". Maybe it's cutting edge work, a huge upheaval in the organisation, or it's a death march project taken over by a sociopath manager that has their own dream to fulfill. In operational systems, chaos occurs if your system suddenly fails (hacked, rushed/forced to production, solar flare). At this point you have to move quickly and definitively to fix the problem. Once fixed, you may be in a good position to explain (again) to senior management why you've been lobbying for that high availability setup.
To be honest I lied a bit – there's a fifth domain, disorder. You're most likely to have experienced this domain if you're a high-school teacher trying to teach VB6 to students that found a case of Red Bull in the fridge at little lunch. Being extremely agile here is the only way to weather the storm.
These domains aren't set on some dogmatic lore and you may even notice the your project spans across more than one domain. Take for example a payroll system. It's most likely in the Known/Knowable domain in terms of software but the organisational context may be Complex/Chaotic. Maybe Product X was selected but the accounting team used Product Y and look set to engage in passive/aggressive mode. Maybe you're facing an organisation with 6 payroll systems or maybe you got the budget for the software but none for the customisation.
It is important to bend your mind a bit and see if the domain really is what it appears. Perhaps what you're seeing fits the Known domain but it's really out of date and people are just clinging on to old ways – maybe it needs some disorder thrown at it. What's critical is that you get a sense of how decisions and understandings are reached within those domains. Armed with this you'll at least be able to tailor your navigational instruments as you motor towards your goal.
As someone who often works on projects that seek to introduce several new concepts to stakeholders I am often suprised that it's a struggle to convince the project office that agile methodologies are more effective than their highly structured project management framework. Everyone in the room knows that the timelines and feature-set will be difficult to set out up-front. Everyone in the room knows that we're doing more than development work - we're bringing about cultural change. But not everyone in the room likes to admit that things will get complex.
Kurtz, C.F. & Snowden, D.J., 2003. The new dynamics of strategy: sense-making in a complex and complicated world. Engineering Management Review, IEEE, 31(4), p.110.
Newell, A. & Simon, H.A., 1972. Human problem solving, Prentice-Hall.
Snowden, David J. & Boone, M.E., 2007. A Leader’s Framework for Decision Making. (cover story). Harvard Business Review, 85(11), p.68-76.