Monday, May 5, 2008

Government Watch

In the Information Technologies (IT) market there is a domain called Application Monitoring (AM) and sometimes Business Process Monitoring (BPM). The company I work for (HP) has some products in that market (PDF here, lots of techinical language but some nice screenshots), and so do others. The paradigms of this practice deals with how to validate the quality and performance of IT systems over time, in production (which means – while the systems are being used). The paradigms has been shifting over the years from monitoring the technology to monitoring the process, and I wonder if we can nudge it just a bit further and make the same thing for monitoring human processes. And if so, can we use it to enhance our public, governmental systems?

Application Monitoring for Dummies (no offence meant)

Let me give a rough explanation on the AM or BPM paradigm (cool word, I’m gonna use it a lot!). The basic assumption of this domain is that IT systems, once deployed and used by real users, do not necessarily maintain their planned quality and performance. This does not require much explanation to anyone who has Windows. One approach to maintain good quality (which means that the process of using the system to achieve the user objective is executed as expected) and good performance (which means that the system is available at all times and any major action can be completed in the expected time frame) is to rigorously test the system before it is being used. Unfortunately for most systems it is unrealistic to test for all scenarios and all possible loads in advance.
The AM idea is to keep ‘testing’ the system even while it’s being used. It started out as a simple hardware monitoring – track the server and see if it works slowly. If it does, then you probably have an issue with your application and you should do something about it. After a while, monitoring added the software as another resource to look at for problems.
AM gradually shifted focus to the user experience and not the system resources. The idea was that what really interests the administrator of the system (in charge of its quality and performance) is not how a server was performing but how a user of the system felt using it. It may turn out that a server is doing fine, but the user still can’t complete the action in the 2-3 seconds he expects. The last stage of AM (usually called BPM) is to define a sequence of user actions on the same issue to be a ‘process’, and then to monitor the quality and performance of the process as a whole, and not to focus on a specific action. Note that these monitored processes may combine human actions and system actions. For example, buying a book at Amazon includes a human action (choosing and ordering the book) and a system action (issuing the book from storage to shipment).
In most cases, organizations which practice AM are doing all three approaches – monitoring the hardware and software resources, monitoring specific user actions and the user experience, and monitoring the business processes.
The end result of AM is usually statistical data shown as graphs and tables, indicating what can currently be done easily and what is difficult (or impossible) to complete. The more advanced products let the administrator look at the data overtime, allowing him to realize trends and come up with insights (for example, that most issues happen at 9am, when people get to work and start using the system).

From computers to people

So can we use similar techniques to monitor human systems? I have in mind the Israeli government offices, for example – can we monitor, visualize and report the time it takes to obtain an Israeli passport? Issue a construction permit? Submit tax returns? And if we could monitor these, what will we gain by doing so?
Most of these processes are not just partly human – they’re mostly (and even wholly) human. To realize how much time it takes to obtain a passport, you need to collect data on how much time it takes to stand in line at the Interior Ministry office nearest to you, how much time it takes for the clerk to go over your documents and sign an application, and how long will you wait until you get the passport in the mail. Unfortunately, only a small portion of this process is fully automated, and waiting until the government will automate it all may take some time (like a few decades). If we want to monitor this process, we probably need an external system, and not rely on the government to build one.
Let me first try and show that there is something to be gained by doing so, and then we can dive into the details of how it may work. My suggested implementation is probably only one way of doing it, but I will use it to help visualize what I’m aiming for.

This is how it feels when your word means something at all

One day you will go to a Web site, say http://www.govwatch.co.il/ (domain still vacant, if you’re the startup kind of guy) and have a look at an overview of the government offices performance. You may see that the Ministry of Interior offers 127 different procedures, completed on an average time of 2 months and 3 days, and completed successfully 63% of the times. The Ministry of Defense, on the other hand, has only 71 procedures, completed at an average time of a week and 4 days and at 84% success rate. This may interest you or may not – what you’re really interested to know is how much time it may take you to get your marriage certificate.
You search for ‘marriage certificate’ and soon you reach a graph which shows you the average time it takes to obtain one, with an activity breakdown – how much time it will take to schedule a meeting at the Rabanut, how much time it will take to wait there while the Rabbi is out for lunch, how long will you have to suffer the nonsense rumbling of the Rabbanit etc. Even better, you will have a location-based breakdown, showing you how much time it takes at Tel Aviv as opposed to Jerusalem or Sderot.The first obvious gain is that you can make better plans based on this information – decide when to apply for a marriage certificate, for example. The second gain is somewhat less obvious – you may have more trust in the government office, because your expectations will be met, or to be more precise, you will know what to expect and will not be disappointed because a process takes ‘too long’ or is likely to fail.
But user (a.k.a citizen) satisfaction is only the first step forward. Visibility into the governmental procedures, even offered by an external party, will likely drive the office in charge’s motivation to improve. After all, who wants to be publicly known as the slowest DMV office in Israel (backed up with fancy colored graphs)?! And the Web site even offers these offices information on how to better themselves – they can look at the activities and find out bottlenecks and potential improvements, they can see which offices are doing better and then call them up and get some tips. I’m sure that it is very naïve of me, but I think it may add another motivation to their work environment, more closely linked with their customer needs than the motivations the government currently provides.
To maximize on this vision, imagine that we have a multitude of such Web sites, one for each state. We then may have another way to measure states and indicate their efficiency and stability (similar to scales such as the Corruption Perceptions Index). One can use the information to decide where to live, but it may even be a diplomacy tool. Imagine the EU insisting that Ukraine will improve the ‘preliminary trial’ procedure time by 10% before joining the EU…

If you build it, they will come (I think)

From wuthering heights, let’s drop down to the possible mechanisms. Again, this is just one possible suggestion, to show that this is an achievable target, and not necessarily the best method.
Let’s assume that the technology to analyze the data and display the visualized results is not an issue. The main challenge of implementing such an approach would be collecting the data in the first place. For IT systems, data collection is done through agents reporting to a main server. These agents are usually small programs which run at a server or a user computer, and monitor the system resources or the user actions. The advanced tools offer automation – mostly of the user actions – so that you will not waste human resources on doing a monitor job.
Can we find analogues to these agents? Our Web site may employ people to server as agents. They will go through the procedures and then report the quality and performance. This is similar to food critics. The problem is that in order to reliably report a system we need a mass of repeatable results, which then can be measured statistically. It will take quite a work force to go through all the governmental procedures, even without repeating them every once in a while. Another disadvantage (which AM for IT systems is facing as well) is that processes in production needs real data and often result in real consequences. For example, you cannot use a fake ID to issue a passport, and when you use a real ID you cannot issue a new passport often enough to report the process.
A better approach would be to reply on customer feedback. Customers’ feedback is a proven tactic (with known disadvantages), for example in recommendations systems such as NetFlix and Amazon and in ‘customer watch’ sites such as Service Report. A person who is about to go through a procedure can register on the Web site, then issue a ticket (a unique id) for the process he’s about to undertake. We need to have a pre defined set of common activities, leaving room for remarks but enabling the users to focus on reporting the time and success of an activity (coupled with location and other meta-data). In a wiki-style, we can have some of the users define these activities and combine them into processes.
Now that the user has a ticket id matching his process, all he has to do is logon to the site every time he completes an activity and report it. The challenge here is to come up with a motivation for people to do so. I think there may be several approaches to achieve that:
A wiki-approach that offers ‘virtue as its own reward’ is a valid one. Its limitation is that we will only have a miniscule percent of the people do it, but that’s OK as we only need enough data to have meaningful statistics.
The personal value of using the Web site will be obvious and will encourage people to participate. One possible model would be to link participation with usage, although that has the risk of limiting the site to a ‘members club’ and not getting the greater benefits which I listed before.
The site can serve as an unofficial channel for people to communicate their governmental experiences. Dedicating time to resolve specific issues (as do ‘customer watch’ sites) and reward efficient offices and persons, the site may be able to gain credibility as an effective measure. The 2 main weaknesses of this approach is that the site will be biased towards the extreme cases and will have difficulty getting the needed masses of data; and in order to be effective, some governmental cooperation must be achieved.
Finally, the site may have money rewards for contributing. This can be financed by contributions, by global organizations (like the UN) budgeting and by charging for using the site (maybe for just some of the site services, e.g. location breakdown of procedures).
Of course, the more rewarding the contribution is, the greater the risk of abuse. That can be mitigated as well, for example validating that data is not submitted at once if it is likely to be gathered over time (to prevent people from just making up cases).
The customer feedback system can be further augmented by having the Web site employees gather objective data, such as the number of clerks at a given office, time limitations set by law, statistics of people waiting in line for a specific application etc. That is somewhat analogues to the hardware monitors that AM for IT systems use – it doesn’t necessarily indicate the quality of the procedure, but it can help to evaluate it.

I have a dream

Naturally there are a lot of details to discuss and questions to answer. Still it seems to me that the possible value of having such a tool at our disposal it so great, that it’s worth the effort, money and the shortcomings. I think this may harness the strengths of the Internet (masses, equity, anonymity, ease of use) to slightly advance our states to be more user oriented – meaning, being closer to what you and me want and expect.

So, do I hear anyone who wants to make the dream a reality?

* thans to Dubi, whose comments on incremental changes and giving feedback to the government encouraged me to post this. Unfortunately you'll have to work your way through an interesting but way too long a discussion to get to them.

7 comments:

iod said...

What's missing from your plan is how people will hear about this. Without government cooperation and/or funding for publicity, you will forever stay with a small (probably biased, especially geographically and socio-economically) non-random sample of the population.

Socio-economic biases, of course, are also inherent in using the internet, in generally using a written (Hebrew) medium (what about older/Russian/Arab citizens?).

Inbar Shani said...

'Hearing about it' is definitely a good point. I think it really depends on the approach you'll take with the conrtribution issue, as money-making opprtuniteis are so much easier to market. Alternatively, if you go for getting budget as a UN project, then surely some of it will be used for marketing. The wiki-approach will likely need Geek-style marketing, such as getting an article in WIRED and equivilant magazines, being published on major Internet venues as a 'sensational new project' which 'just may change the world' :-)

I don't see any issue with multi-lingual support.

Anonymous said...

I'm just glad you finally posted a new blog!
It has been a while...

-oor

Anonymous said...

Fascinating. I'm not quite certain what's my opinion about this whole "Monitoring processes" thing - it seems like a huge leap - but this is definitely some good reading and some food for thought. I won't be as quick as iod to offer criticism, mainly because my main problem is seeing it actually put to use.
Well done, anyway. Well done indeed.

Anonymous said...

Great thought provoking post!
My comments:
- I think the service you're envisioning here should not concentrate on collecting the "operational" data generated in the interactions of a citizen with its government. This should be demanded from the government to perform, and I think even now, even in Israel, most of the data you're looking for is there. Taking advantage of the Freedom of Information Act, for example, can help you get the data. International organizations (UN, World Bank, OECD) should help you "persuade" your government to measure and publish its performance.
- You should concentrate on (a) gathering data from all the governmental agencies, (b) integrate the data and (c) publish it in a user-friendly, end user-oriented form. There's a great power in integration of data from different sources. Take gapminder.org as an example. They don't create data, just collect data from all around and integrate it to show the things user might care about (in a cool way).
- In my short stay in the US so far I've experienced the following examples of performance reporting for public services: Immigration agency showing processing times for each type of form in each one of their offices; on each federal form you have the time that it will take you to fill it (incl. get the data you need, etc.); the MA DMV office has real-time waiting times for different services for all of its offices.
- Finally, I would consider reporting the broader performance of the government. E.g., How good my town's budget is managed? How many complaints per 1000 people served were submitted to interior affairs offices around the country?

Thanks for engaging me in this!

Anonymous said...

|
30 years ago regional promoting was performed generally by demonstrating static images of the nursery storefront inside of a thirty second advert that was accompanied by a track record, announcers voice that promoted typical nursery solutions made available with the advertiser. Earlier morning adverts are considerably less viewed, along with the adverts could be a reasonable $50.00 each, while, soap opera Television set and evening news stories draw 1000's the Television set screens, as well as adverts may well cost 100s of dollars each and every for a thirty second promoting exposure. Weekend promotion prices usually can be one-half or much less, in expenses, than Monday as a result of Friday promoting.

Anonymous said...

Ations gibt junge Frauen, kick the bucket treffen living room Zeitgeist. Julia [url=http://www.sreecgmath.org/louisvuittontaschen.php]Louis Vuitton[/url]
Saner ist hence eine. Sie sieht aus wie ein Engel, physician sie einer ist, kann ich nicht beurteilen. Sowie es ist nicht etwa [url=http://www.sreecgmath.org/louisvuittontaschen.php]Louis Vuitton[/url]
so, da脽 sich diese Menschen mit nur einer Uhr zufriedengeben. Es gab mal eine Zeit, when it comes to der fella sich mit einer Wristwatch begn眉gte. Mittlerweile ist ations jedoch 眉blich, f眉r jeden Anla脽 eine andere Uhr zu besitzen -- egal ob nun chevalier Dinnerparty, gesch盲ftliche Besprechung oder ein Nachmittag auf dems Golfplatz.
Eine der ersten Siedlerinnen, die sich ein Domizil mit gr眉nem Vista [url=http://www.sreecgmath.org/louisvuittontaschen.php]Louis Vuitton[/url]
g枚nnten, combat die-off Mutter der aufkeimenden Brand-new Yorker Modern culture, Mrs. Caroline Schermerhorn Astor. Ihrem tonangebenden Geschmack folgten alsbald gro脽e Different Yorker Familiennamen, darunter Vanderbilt, Carnegie, Kahn und Rockefeller.
Das. has been mankind heute gerne Einkaufserlebnis nennt, ist sein Werk: Er schafft R盲ume, stop working within Cremewei脽 und Encore gehalten sind, around denen drei Zentimeter dicke Teppiche living area Schall schlucken und ations nach feinem Kalbsleder duftet. Im ist der Erfinder der beigefarbenen Shoppingerfahrung. "Beige ist sch枚n sowie eignet sich stomach and intestines, wie Hintergrund, other andere Farben umso besser hervorzuheben", sagt ser.