Government Watch

In the Information Technologies (IT) market there is a domain called Application Monitoring (AM) and sometimes Business Process Monitoring (BPM). The company I work for (HP) has some products in that market (PDF here, lots of techinical language but some nice screenshots), and so do others. The paradigms of this practice deals with how to validate the quality and performance of IT systems over time, in production (which means – while the systems are being used). The paradigms has been shifting over the years from monitoring the technology to monitoring the process, and I wonder if we can nudge it just a bit further and make the same thing for monitoring human processes. And if so, can we use it to enhance our public, governmental systems?

Application Monitoring for Dummies (no offence meant)

Let me give a rough explanation on the AM or BPM paradigm (cool word, I’m gonna use it a lot!). The basic assumption of this domain is that IT systems, once deployed and used by real users, do not necessarily maintain their planned quality and performance. This does not require much explanation to anyone who has Windows. One approach to maintain good quality (which means that the process of using the system to achieve the user objective is executed as expected) and good performance (which means that the system is available at all times and any major action can be completed in the expected time frame) is to rigorously test the system before it is being used. Unfortunately for most systems it is unrealistic to test for all scenarios and all possible loads in advance.
The AM idea is to keep ‘testing’ the system even while it’s being used. It started out as a simple hardware monitoring – track the server and see if it works slowly. If it does, then you probably have an issue with your application and you should do something about it. After a while, monitoring added the software as another resource to look at for problems.
AM gradually shifted focus to the user experience and not the system resources. The idea was that what really interests the administrator of the system (in charge of its quality and performance) is not how a server was performing but how a user of the system felt using it. It may turn out that a server is doing fine, but the user still can’t complete the action in the 2-3 seconds he expects. The last stage of AM (usually called BPM) is to define a sequence of user actions on the same issue to be a ‘process’, and then to monitor the quality and performance of the process as a whole, and not to focus on a specific action. Note that these monitored processes may combine human actions and system actions. For example, buying a book at Amazon includes a human action (choosing and ordering the book) and a system action (issuing the book from storage to shipment).
In most cases, organizations which practice AM are doing all three approaches – monitoring the hardware and software resources, monitoring specific user actions and the user experience, and monitoring the business processes.
The end result of AM is usually statistical data shown as graphs and tables, indicating what can currently be done easily and what is difficult (or impossible) to complete. The more advanced products let the administrator look at the data overtime, allowing him to realize trends and come up with insights (for example, that most issues happen at 9am, when people get to work and start using the system).

From computers to people

So can we use similar techniques to monitor human systems? I have in mind the Israeli government offices, for example – can we monitor, visualize and report the time it takes to obtain an Israeli passport? Issue a construction permit? Submit tax returns? And if we could monitor these, what will we gain by doing so?
Most of these processes are not just partly human – they’re mostly (and even wholly) human. To realize how much time it takes to obtain a passport, you need to collect data on how much time it takes to stand in line at the Interior Ministry office nearest to you, how much time it takes for the clerk to go over your documents and sign an application, and how long will you wait until you get the passport in the mail. Unfortunately, only a small portion of this process is fully automated, and waiting until the government will automate it all may take some time (like a few decades). If we want to monitor this process, we probably need an external system, and not rely on the government to build one.
Let me first try and show that there is something to be gained by doing so, and then we can dive into the details of how it may work. My suggested implementation is probably only one way of doing it, but I will use it to help visualize what I’m aiming for.

This is how it feels when your word means something at all

One day you will go to a Web site, say http://www.govwatch.co.il/ (domain still vacant, if you’re the startup kind of guy) and have a look at an overview of the government offices performance. You may see that the Ministry of Interior offers 127 different procedures, completed on an average time of 2 months and 3 days, and completed successfully 63% of the times. The Ministry of Defense, on the other hand, has only 71 procedures, completed at an average time of a week and 4 days and at 84% success rate. This may interest you or may not – what you’re really interested to know is how much time it may take you to get your marriage certificate.
You search for ‘marriage certificate’ and soon you reach a graph which shows you the average time it takes to obtain one, with an activity breakdown – how much time it will take to schedule a meeting at the Rabanut, how much time it will take to wait there while the Rabbi is out for lunch, how long will you have to suffer the nonsense rumbling of the Rabbanit etc. Even better, you will have a location-based breakdown, showing you how much time it takes at Tel Aviv as opposed to Jerusalem or Sderot.The first obvious gain is that you can make better plans based on this information – decide when to apply for a marriage certificate, for example. The second gain is somewhat less obvious – you may have more trust in the government office, because your expectations will be met, or to be more precise, you will know what to expect and will not be disappointed because a process takes ‘too long’ or is likely to fail.
But user (a.k.a citizen) satisfaction is only the first step forward. Visibility into the governmental procedures, even offered by an external party, will likely drive the office in charge’s motivation to improve. After all, who wants to be publicly known as the slowest DMV office in Israel (backed up with fancy colored graphs)?! And the Web site even offers these offices information on how to better themselves – they can look at the activities and find out bottlenecks and potential improvements, they can see which offices are doing better and then call them up and get some tips. I’m sure that it is very naïve of me, but I think it may add another motivation to their work environment, more closely linked with their customer needs than the motivations the government currently provides.
To maximize on this vision, imagine that we have a multitude of such Web sites, one for each state. We then may have another way to measure states and indicate their efficiency and stability (similar to scales such as the Corruption Perceptions Index). One can use the information to decide where to live, but it may even be a diplomacy tool. Imagine the EU insisting that Ukraine will improve the ‘preliminary trial’ procedure time by 10% before joining the EU…

If you build it, they will come (I think)

From wuthering heights, let’s drop down to the possible mechanisms. Again, this is just one possible suggestion, to show that this is an achievable target, and not necessarily the best method.
Let’s assume that the technology to analyze the data and display the visualized results is not an issue. The main challenge of implementing such an approach would be collecting the data in the first place. For IT systems, data collection is done through agents reporting to a main server. These agents are usually small programs which run at a server or a user computer, and monitor the system resources or the user actions. The advanced tools offer automation – mostly of the user actions – so that you will not waste human resources on doing a monitor job.
Can we find analogues to these agents? Our Web site may employ people to server as agents. They will go through the procedures and then report the quality and performance. This is similar to food critics. The problem is that in order to reliably report a system we need a mass of repeatable results, which then can be measured statistically. It will take quite a work force to go through all the governmental procedures, even without repeating them every once in a while. Another disadvantage (which AM for IT systems is facing as well) is that processes in production needs real data and often result in real consequences. For example, you cannot use a fake ID to issue a passport, and when you use a real ID you cannot issue a new passport often enough to report the process.
A better approach would be to reply on customer feedback. Customers’ feedback is a proven tactic (with known disadvantages), for example in recommendations systems such as NetFlix and Amazon and in ‘customer watch’ sites such as Service Report. A person who is about to go through a procedure can register on the Web site, then issue a ticket (a unique id) for the process he’s about to undertake. We need to have a pre defined set of common activities, leaving room for remarks but enabling the users to focus on reporting the time and success of an activity (coupled with location and other meta-data). In a wiki-style, we can have some of the users define these activities and combine them into processes.
Now that the user has a ticket id matching his process, all he has to do is logon to the site every time he completes an activity and report it. The challenge here is to come up with a motivation for people to do so. I think there may be several approaches to achieve that:
A wiki-approach that offers ‘virtue as its own reward’ is a valid one. Its limitation is that we will only have a miniscule percent of the people do it, but that’s OK as we only need enough data to have meaningful statistics.
The personal value of using the Web site will be obvious and will encourage people to participate. One possible model would be to link participation with usage, although that has the risk of limiting the site to a ‘members club’ and not getting the greater benefits which I listed before.
The site can serve as an unofficial channel for people to communicate their governmental experiences. Dedicating time to resolve specific issues (as do ‘customer watch’ sites) and reward efficient offices and persons, the site may be able to gain credibility as an effective measure. The 2 main weaknesses of this approach is that the site will be biased towards the extreme cases and will have difficulty getting the needed masses of data; and in order to be effective, some governmental cooperation must be achieved.
Finally, the site may have money rewards for contributing. This can be financed by contributions, by global organizations (like the UN) budgeting and by charging for using the site (maybe for just some of the site services, e.g. location breakdown of procedures).
Of course, the more rewarding the contribution is, the greater the risk of abuse. That can be mitigated as well, for example validating that data is not submitted at once if it is likely to be gathered over time (to prevent people from just making up cases).
The customer feedback system can be further augmented by having the Web site employees gather objective data, such as the number of clerks at a given office, time limitations set by law, statistics of people waiting in line for a specific application etc. That is somewhat analogues to the hardware monitors that AM for IT systems use – it doesn’t necessarily indicate the quality of the procedure, but it can help to evaluate it.

I have a dream

Naturally there are a lot of details to discuss and questions to answer. Still it seems to me that the possible value of having such a tool at our disposal it so great, that it’s worth the effort, money and the shortcomings. I think this may harness the strengths of the Internet (masses, equity, anonymity, ease of use) to slightly advance our states to be more user oriented – meaning, being closer to what you and me want and expect.

So, do I hear anyone who wants to make the dream a reality?

* thans to Dubi, whose comments on incremental changes and giving feedback to the government encouraged me to post this. Unfortunately you'll have to work your way through an interesting but way too long a discussion to get to them.