I should be writing my research methods chapter at the moment but to be honest I am finding that a combination of dull and demoralising so am just quickly putting this together instead. I am trying to resist the urge to write about the policy making cycle as part of my thesis because I really need to focus at this point and my focus is around that formal / informal transition that turns civic activity into democratic participation. However – there is clearly a huge need for reform around the policy making process – mainly with respect to transparency and speed. That post on transparency is still on the todo list so this is more a post about speed and process.
This is not a fully formed thought but the similarity between the evolution in software development methodologies and the needs of government keep occurring to me. After all – software developers have been adjusting to the pressures of the network society for some time now so they may well have something to teach us. But firstly – unusually – a picture:
Look familiar? This is the waterfall model of development (thanks to Conrad Huang for the picture) See how we have a nice tidy progress from the requirements to the delivery – with no change of context on the way? No new information is allowed to intrude on the sanitised process of development. This is how I think of our current policy process – though of course we don’t actually use it like that as we give it little political nudges enroute as per my last post. Look instead at this agile model of development:
Thanks to Neil Perkin for this one The thing that strikes me is that the idea of iteration is central to the Agile model (as is the idea of continuous testing and constant progress against a larger goal) and this is where it differs so much from the waterfall model. Instead of assuming that we can write the omnipotent specification document Agile (and associated methodologies such as RAD for example) assume that there is some kind of learning during the building process and that we can adjust to accommodate this without some kind of weighty change control process – we build the idea of change and learning into the process. There are many reasons why software developers have adopted this kind of approach but the main one is that the speed of web development means that its quicker to try things out and see what happens rather that fully describe them first. This may not be what we want from the people who are building roads and hospitals but as digital simulation technology improves it will become easier and easier to model policy before implementing it and the network society means that we can get more and more detailed reactions from the public before committing.
One of the byproducts of this kind of approach is also a shift in the attitude to failure as it becomes a learning enroute to our destination rather than an insurmountable problem. Isn’t this what we need to do with political decision making? Build a little more humility in and ask people as we go along? The risk here is that of never delivering anything as we constantly creep the mission – and its a mistake to think of Agile as a less disciplined approach than the command and control style of the waterfall approach.
If we were to translate this to government then we would need far better decision support tools and also a more transparent discussion as to our destination – our shared vision. There are some really strong parallels between the pressures that have moved software development from an engineering / waterfall type model to a RAD or Agile method that could be used to discuss the changes need to the policy forming process to both involve citizens more directly and also to speed up the process. One big barrier to this is the get popular acceptance for the idea of a non-perfect policy enroute to a good one – ie that mistakes can happen – but as people grow up with a digital footprint of youth indiscretions we will have to get more tolerant of ‘mistakes’ on public life generally.
Anyway – this is a rather wide ranging and undisciplined post – but its really a marker for a larger piece of work that needs to wait for the thesis to be a little bit more formed. Oddly it does also relate to the reading I am doing at the moment on action research methods so perhaps I will try and join this up as well.
Curiouscatherine's Blog 
November 12, 2010 at 8:19 am
One unhelpfully different thing about the policy process compared to software design is that there is disagreement about the ends as well as the means. Also even when you force a decision about the ends that is liable for revision every time you have an election and sometimes in between! Since your objectives determine your criteria for success or improvement, this rather complicates the situation as you listen out for feedback! I don’t think this undermines the interest of the comparison you make. I just think it underlines the difficulty of the policy-making. Some might even argue that the messiness of the current process is actually quite helpful in masking the issues. Take the welfare trade-off between providing a decent lifestyle for those who are unable to achieve this for themselves and discouraging people who could provide for themselves from living off benefits. There are no doubt many design things you can do to minimise this tradeoff, but at some level at the margins etc there is going to be a tradeoff and different people will assess the tradeoff differently. Imagine there was a very simple and clear calibration – the current system generates £X of pain to the genuinely vulnerable and allows Y% of not fully deserved claims. Then the policy argument could just be over the numbers – one party could say cutting benefits by an average of £5 a week is worth cutting not fully deserved claims by !%, the other party could say the opposite. Then we could just yo-yo around as the balance of public opinion shifted from “oh we are being a bit too hard on vulnerable groups” to “oh we need to toughen up on scroungers”. This is a national and obviously political issue, but even at the local government level there are similar tradeoffs, e.g. new housing vs preserving existing communities or greenbelt etc.
November 12, 2010 at 8:41 am
Completely agree – though I think in many ways this echoes the shift that software developers had to make when coding for the internet in that they had to accept the fact that they could no longer work in a closed system – which I think is a big part of the reason for the iterative nature of web based methodologies.
Ah…closed systems….so elegant…but so utopian!
Will be interested in how you handle this issue of trade offs making things more complex – and even unmanageable in the context of transparent decision making. Personally think its a matter of careful and clear scoping of context – but that may be oversimplifying things
November 14, 2010 at 9:45 am
Catherine, I think your instinct is dead right: agile software practices are a great model for this sort of change. As you say, you can’t iterate your way to building a hospital, but I feel that changes with a substantial behavioural component scream out for this approach.
In a simple and static world Waterfall works well, but where there is simultaneously complexity and dynamism, it really doesn’t. This archetype seems to be deeply ingrained everywhere in government management machinery and, because the world has changed, it is doing us no good (billions-wasted-every-year no good). I wrote (ranted really) about this recently and also about a different kind of leadership needed for complex/dynamic environments (pls forgive self publicity).
You mention forming larger thoughts. Have you read any of Peter Senge’s work? His Dance of Change, and other ‘Fifth Discipline’ books, might be useful (provided you have strong bookshelves).
And there’s another thing… you touched on failure. I hear lots of discussion now about how ‘risk is a bedfellow of adaption’ and how we need to ‘transform our attitude to failure’. But I am not sure I go with this.
Working in a genuinely Agile way, making incremental changes that properly integrate behavioural change, it’s pretty much impossible to make a big cock-up. People do say when things are going wrong and managers, provided they are not bound into great big juggernaughty programmes, can respond before things go horrible. Agile is inherently less risky, its just that it needs a different kind of leadership.
Glad I’ve got that of my chest. Time for a bacon sandwich. Happy Sunday.
(And, almost forgot… nice pictures!)
December 5, 2010 at 11:31 am
[…] wrote a piece the other week about the idea of agile policy making and this links in to this thought. Its not enough to get mass participation at the start of the […]
December 12, 2010 at 4:53 pm
This strikes a chord as I am about to embark on an attempt at applying agile approaches to policy implementation (though not, in this case, to the process of forming the policy to be implemented). But while the challenge of shifting the policy process from a waterfall to an agile approach is a good way of thinking about it, I think there are a couple of interesting differences (the first of which is related to but not quite the same as the one Paul describes).
An awful lot of political decision making isn’t about the thing you are ostensibly deciding on, it is about the place of that thing in the context of everything else. I wrote a blog post a couple of years ago which tried to capture that difference. I suspect that there is an intrinsic tension between what I called there the absolute and relative questions – it’s not even conceptually possible to answer the relative questions and then deal with the absolute questions, and it’s a million miles from being practically possible. That’s quite an important difference from a software project which is about answering an absolute question – and if it isn’t is almost certainly on its way to being a failed software project.
A second potentially important difference is modularity. Agile development works in part by breaking the problem down into smaller pieces with the structure of the project as a whole providing assurance that the pieces can fit back together again without everything getting bogged down by everything else, and at a scale where dropping one approach and switching to another in a single module is a positive change, not a disaster. It would be interesting to think (not that I have done any of that thinking) whether there are categories of political decision making which lend themselves to that approach and others which don’t.
But that’s not to detract from your central point – this is an analogy well worth pursuing, even if it’s not perfect (as no analogy ever is).
January 6, 2011 at 10:30 am
Firstly Stefan – so sorry not to reply to this before Christmas – really rude of me – especially when its such an interesting comment.
Picking up on your point about modularity – I think one of the reasons that a more agile approach should be considered is that we have to break things down more. Its just so arrogant to think that we can come up to big solutions to big problems – its a media trap born of short attention span. Incremental improvement makes fewer headlines but is far more sustainable in the long term. I think this is linked to your point about absolute / relative?
Anyway – glad this is going to get some airtime at GovCamp – hope I can see you there!
C.
December 31, 2010 at 2:12 pm
Really interesting post! I was following some of the
discussion & posts quietly on Twitter last night. I was a
little bemused as to why a way of doing things that seems obvious
to me is provoking such discussion. Reading your post, Catherine,
something clicked into place. I realise that my background in art,
design and history plus experience of research and development of
digital things leads me to expect that the most practicable
approach is what others call the “agile” method. It’s a ‘normal’
method to me. I think that it’s a good way of working, but it is
difficult when trying to use it in an environment in which the
audience/consumers/voters have different expectations (and I say
that from some bitter experience a few years ago); and in which
those who have the ultimate say over budgets do not trust their
professionals enough to let them get on with things. I can also see
that applied to government work, it could become a way of never
doing things, a different approach to red tape. Sometimes, doing
nothing is good, even the most effective and cost-efficient
approach, but most of the time, getting things done is so much
better. I can imagine risk-averse managers insisting on a
never-ending pattern of iterations. Of course, there is more to a
successful agile approach than is included in the diagrams usually.
Some people look but don’t see, hear but don’t listen – so the
feedback tells them only what they want to see or hear. Experience
and knowledge are other major factors that I think are
underestimated. We can start out with a myriad of possibilities of
how something could be done. The faster we can weed out the ones
that are least likely to work, the less iterations are required in
the process. For it to work in a government context, I suggest that
education about the methodology and possibly open discussion about
expectations between public sector and the public is probably
necessary.
January 6, 2011 at 10:31 am
Agree! Its also always interesting to see what someone from a different background makes of this stuff…
Open discussion is definitely needed – this would be such a culture change to most people….
C.
January 5, 2011 at 3:25 pm
One other quick problem to throw in – for political decisions (but not for the finer minutiae) an agile approach would doubtless attract cries of “u-turn!” and the like.
January 6, 2011 at 10:21 am
Sorry for delay in replying – got distracted by fact gathering in the afternoon!
I think this is important – at some point we are going to have to accept the legitimacy and even advisability of decision makers changing their minds when new facts emerge.
On your point about community meetings – do you think its the fact that the officer is more likely to know the people that makes the difference? I think that’s what your implying in which case its maybe a case of how do we create that introduction process online? Administrator seems like a good place to start.
C.
January 8, 2011 at 5:48 pm
I think the trouble is that the media won’t, they and politicians both love to slam u-turns and bewail mistakes.
On community meetings, it’s because the officer can at least see the audience, feel the temperature of the meeting; but most importantly she/he knows the chair reasonably well, and it’s the chair who ultimately takes the heat and intervenes when things get ugly. Of course that’s slightly idealised! So yes, I’d imagine if officers were told “it’s just like a community council” and were introduced to the admins, it would put them more at ease.
January 23, 2011 at 8:01 pm
[…] have written before about the need to move from Waterfall to Agile thinking for Policy delivery and Stefan has also […]