Virtual Town Hall workshop: Essex

Spent the day in Chelmsford today meeting loads of people re: Virtual Town Hall.  This is by way of some observations and notes for me to remember for research purposes:

Meeting the districts:

• We spent the morning with 3 districts who are considering getting involved and we had a very in-depth discussion about the concept of the VirtualTH as well as the specific project details.  Once again the basic proposition held up to scrutiny but there was a lot more concern about risk management than we have had in other meeting - this is perhaps because I have spent a lot of time talking to social web people who are more relaxed about the idea of content being unmoderated – but highlights the need to talk about risk when I start interviewing officers/members about their experiences of the project.
• It was also clear that there is a considerable amount of additional complexity added by working in two tier areas.  Questions of where decision making will sit, relationships between the two sets of members and concerns about what might happen if two participating organisations found themselves in conflict over an issue where raised.  It will be interesting to see which (if any) of the districts decide to get actively involved and which decide to wait on the sidelines until it is clear how things are going (have put predictions in an envelope and we will see how accurate I am!!).
• There were a couple of folks there from the LSP and they seemed very interested in the project.  LSP involvement could be really important in terms of connecting citizens to the right decision makers for specific issues so it was good to have their involvement
• Overall it was a very useful morning with lots of good discussion – hopefully it will lead to some active involvement from districts – if not immediately then early next year

Getting the internal infrastructure working

The afternoon was spent with the Essex CC team discussing more operational issues such as:

• Social web audit:  We have yet to really crack the social web audit as Essex and we spent some time discussing ideas of new places to look for activity.  Lots of new leads here which was very useful.  We also defined some more of the useful questions which we can apply to other sites.  it is increasingly clear that this is a process which will never be finished – we need to build it in as a regular maintenance task
• Social web policy:  We discussed the process of getting this into place and also went through the key points for inclusion – next up I have to deliver a draft.  Part of this will be defining the commitment which is made to participants in the site and as part of this I am going to suggest a list of democratic activities which could be offered here – I will post this list later in the week hopefully.
• Defining the democratic promise – we need to be very clear about what people can expect in terms of democratic outcomes from participation
• Recruiting Community Ambassadors:  We had a good discussion about where to find the right kind of people and came up with some existing internal mechanisms which can be connected to

Conclusions

We made a lot of practical progress today and it was also very useful to talk through the project with the districts.  As I was driving home however I was thinking about the scale of what we are trying to do here – in terms of fundamentally addressing the nature of the relationship between council and citizens – and though it is clear to me in every way that this is an essential process if we want to use the social web to do democratic things – it may be as well to try and break this down into more understandable stages (as per the ladder of engagement) so that we can focus people on more immediate goals.  Will need to think about this.

 

Donegal workshop – rather long action research note!!

I was in Ireland this week helping to run a Citizenscape workshop in Donegal (one of the pilot sites). We are looking to involve Youth Councillors and other young people as community moderators (Just as an aside – I really struggle with what to call these folks as a group – “young people” makes me feel like we are talking about them as an alien race but what else do you use? For now I will call them the folks at Donegal and you’ll have to remember that they are all under 25!). It was a really enjoyable session and I am looking forward to working with this lot as they campaign around getting government buildings using sustainable energy and getting more cycling lanes in Donegal.

Anyway – this post is really an action research note on the workshop to help improve the format etc for next time and then highlight points for future research so brace yourselves – its long.

The aim of the workshop was threefold:

• Identify a topic that they wanted to work with

• Make sure they were all comfortable using all the technology involved – including filming short pieces to camera

• Get to a common agreement around how the site would be moderated and agree some immediate actions to get things moving

Overall the workshop was run very loosely as its difficult to know in advance where the participants would like to focus it. Next time I do this I will try and spend more time on the actual topic – we got rather carried away with the technology stuff which was fine with this group as they were interested but I will try and bring the balance back on the content. I think it would also be good to have a stand campaign template that people could start to complete in the workshop as a takeaway.

The first section of the day however was a discussion of current web tools that the team already use – so that we could then relate them to a citizenscape context. We organised the data in these catagories:

• 1 to 1 tools where you know the person (or people) you are communicating with well. These tools include: SMS / MMS / Email / Skype

• 1 to Many tools where you are is a shared space of people that you may not know in person. These broke down further into two groups: Social and Themed

• The internet out in the wild with no really social aspect. General sites and services included:  Google,Music Download, Yahoo

I am doing a more detailed analysis on this as the catagorisation fits in with my wider theoretical framework that I am using for evaluation. However there are a few particular themes I wanted to pull out of this session which I will pick up on in my focus groups for the project evaluation:

• Privacy / Safety – the group were reassuring aware of online safety and were careful about what details they revealed online.

• Identity – they were also sophisticated about the need to have different personae online and were comfortable with the idea that you might have a specific persona for a specific purpose.

• Space and place – in discussing the way in which we were catagorising the sites/services they had all listed there was clear agreement about the different social spheres that these worked for. There was a sense of appropriate spaces for different activities and when we started talking about campaigning we were able to talk about how we can use these different social spheres to contact different people.

• This is stating the obvious perhaps but there was a huge difference in the level of online skills when compared to an older group. I would like to explore this more and look at doing more mixed age groups.

• Also stating the obvious there was a big skills gap between the participants and the youth workers which would need to be addressed in future iterations. Happily the officers for this group were also really enthusiastic and used the event as a real chance to learn – but this could be a barrier with other sites and needs to be looked at

• There were no gamers in this group – but they said this was not typical and we should keep an eye out on other groups.

The other thing to note is that the two of the main propositions around citizenscape – that you can use the Social Web to find people who are interested in stuff and that you need a specific place to talk about ‘civic’ issues both stood up to scrutiny here which is reassuring.

If anyone is interested I can share the workshop plan etc.

PS If anyone from the workshop is reading then I am very concious that I have not met the interesting blog criteria of having photos and I know this is too long – I promise I will try harder next time you you know know how I like to talk!!

 

Evaluation – You know you want to really….

However tempting it is just to dive in a set projects up in the social web I you need to consider stopping to think about how you will measure and evaluate success. It’s a big part of using these new tools to have a positive impact rather than just creating empty buzz.

I have been doing a few things this week that all tie together to make me think about evaluation. I’m right in the middle of writing my research proposal and so am having to focus on how to evaluate the impact of the CitizenScape approach in an academically rigorous way. I also took part in a MJ round table event talking about the way in which the social web is being used by Local Authorities and finally I was helping with the judging of the LGComms reputation awards. All of these things highlight the important of figuring out how to measure the impact and effectiveness of using web 2.0 sites and technologies and the need to bring some discipline to the process. In many ways this is a reflection of the fact that these technologies and sites are entering the mainstream – after all if the Prime Minister can make a t*t of himself of YouTube then the possibilities for Councils are endless!!!!

What makes a good evaluation?

This is probably stating the obvious but the key to good evaluation is knowing what you want to achieve in the first place. I think that experimentation is a perfectly good reason in its own right to try something. Its obvious that Local Authorities need to get involved in the online world and that the social web phenomenon is now too big to ignore and I have a huge amount of respect for the Council’s who are making foray’s into this world. However without systematic evaluation of the impacts of these trials we are just dabbling and not really learning. Its the difference between skimming the headlines and sitting down and reading a book on current affairs – you may be able to give the sound bites but you won’t have any particular depth of knowledge. Clearly I am a bit biased here as I take the importance of evaluation so seriously I am doing the PHD but still – evaluation matters.

What can you evaluate?

So – how can we evaluate social web projects? Many people seem to be looking at the traditional web metrics of counting things; numbers of people joining a facebook group, number of followers on twitter, number of views or comments on YouTube. This is one approach but if you go back to the question of what you are trying to achieve then the only question you can really answer with basic metrics with these is “did more people see my content” – its an advertising eyeballs evaluation. For many marketing campaigns then this might be enough but if what you are really trying to do is to reach ‘hard to reach groups’ or encourage some kind of participation then you are missing both demographic and impact assessment data. The absence of traceable / checkable demographic data is probably the biggest frustration here and one of the main reasons why I think it will remain impossible to carry out deliberative debate on these sites – or at least deliberative debate which can then be counted strongly as part of the decision making process. Its also one of the reasons that I think the Virtual Town Hall approach is a better bet. The issue of impacts is also an interesting one. You can probably judge whether or not the numbers of people – the metrics – have effected the decision but how can you measure whether you have effected the people? If you are trying to increase democratic participation then you probably need to know whether your interventions have meant they are more or less motivated to participate in the future.

Finding richer data – not just a head count

Richer data of course means more work. You probably need to do a survey and hound people to answer it and you should also run some actual focus groups (yes – face to face evaluation of an online project – oh the irony!!). My basic plan is to gain a baseline of participation, in both democracy and generally online from as large a group as possible as I can initally and then re-sample this group at the end of the project (and again in the middle if the elapsed time is more than a few months). I will use this survey as a recruitment tool to find out who is willing to either be interviewed or join a focus group. Simply put this approach breaks down like this:

• Web metrics will show you how many actions have been carried out

• Surveys will show you has done this and some basic motivations for their actions

• Interviews will allow you to get a sense of changes in attitudes

Hopefully this balances the need not to overburden the team with work and the need to actually find out more about the people and their reasons for being involved.  I am currently working on a baseline questionnaire and hope to have it out in the world fairly soon.

Analysis: Find a framework and stick to it

So – now we have a lovely lot of data what are we going to do with it? The chances are you will not be thinking of one large pilot – more about a series of smaller projects. In which case a standard evaluation framework (and consistency across your survey questions) is going to help make data collected across pilots comparable and also allow you to make draw some conclusions about whether you are having an effect on your population. In my research I am intending to translate the ladder of engagement idea into something which relates more closely to formal democracy and then to define online activities which have equivalence (where appropriate) with offline democratic actions. The underlying idea of this of one of progression – you plot where people are in terms of democratic engagement at the start of the project and then see whether or not they have moved through the course of your actions. Because you are gathering qualitative data as well as the easier quantitative stuff you can find out more about people’s motivations and their attitudes to the process.

There are also all kinds of interesting social network analyse tools you can use to look at measuring social capital – but these are probably a bit too much for everyday use.

Good value for money?

Just one final thought – though we would all like to do these projects for the love of democracy and the common good the reality is that at some point we will be asked about value for money. This is a huge post in its own right but the basics are:

• For communications projects: Equivalent ad spend figures can be a useful starting point

• For Community engagement projects: Cost of recruitment to a process comparisons or cost effectiveness of running better attended meetings with online supporting

• For democracy engagement projects: Democracy costs! But can you can make some comparisons between online and offline methods. If you look at the ‘cost ode democracy’ formula (yes – councils do have one) then online methods compare well to offline ones

Where you can make comparisons between offline methods then online always looks more cost effective. The issue is of course that no-one wants to stop doing offline – and nor should they. The trick then is to ensure that your pilots are not only creating online effects but also enhancing the existing offline process – for instance by reducing the cost of recruiting a citizens panel or by ensuring that more people attend a public meeting.

Any use?

Well – this has been helpful for me as I will now try and write something very similar but far more detailed for my research proposal!