Yves here. I have to confess that the big reason I’m partial to this article is it is consistent with my experience. Metricization of business, and worse, employee performance, has become a fetish. As I wrote in a long form treatment) in 2006:
Metrics presuppose that situations are orderly, predictable, and rational. When that tenet collides with situations that are chaotic, nonlinear, and subject to the force of personalities, that faith — the belief in the sanctity of numbers — often trumps seemingly irrefutable facts. At that point, the addiction begins to have real-world consequences.
It’s obvious to see how counterproductive simple measures can be. For instance, one push has been to require doctors to publish their success rates for surgeries. That’s often utterly misleading, since top doctors suffer from adverse selection: they attract the most difficult cases. My general bias is if you must measure outcomes, you need to use multiple measures. Over-reliance on a single figure to sum up a complex situation (Value at Risk is the poster child) is going to get you in trouble. But most people don’t like looking at a lot of data and having to weigh it to reach conclusions because that requires judgment….which is what they wanted to avoid relying on in the first place.
And here, it makes sense that government bureaucrats, who have to navigate through legislative and program requirements as well as internal and external politics, have sufficiently complex tasks that simple measures aren’t a great guide to performance. But even more important, these researchers found that the use of metrics to manage bureaucrats actually undermined results.
By Imran Rasul, Professor in the Department of Economics, University College London and Daniel Rogger, PhD Student in the Economic Department, University College London. Cross posted from VoxEU
Around the world, civil service reform is viewed as necessary to deliver public services effectively and to foster development. However, evidence is thin on how the management of bureaucrats affects the provision of public services. This column presents new evidence from Nigeria linking completion rates of government projects to bureaucractic management practices. Greater autonomy is associated with higher completion rates, whereas performance monitoring and incentive schemes seem to backfire. The most effective private-sector management practices may not be suited to public sector bureaucracies.
Since its inception in the 1850s, the British Civil Service has become a cornerstone of the executive branch of the UK government, translating the policy programme of the government into practice. Its practices have evolved gradually over the decades, but it is now in the midst of a major upheaval – in 2012, the Minister for the Cabinet Office and the Head of the Civil Service jointly published the Civil Service Reform Plan. The plan recognised the increased expectations on government to deliver public services in the context of perhaps permanently diminished government resources. To bridge this divide, it argued that the Civil Service would have to undergo significant (and controversial) reforms, aiming to become more skilled and less bureaucratic.
Britain is not alone in striving to reform its bureaucracy. A global drive to improve the functioning of bureaucracies is encapsulated in the ‘good governance agenda’ of the UN, the World Bank, the Commonwealth Secretariat, and other leading international organisations. Without such reform of bureaucracies, it is argued that longer-term goals in international development related to poverty reduction are unlikely to be met. Core to this agenda are the notions of strengthening the accountability of civil servants, and putting in place systems to effectively deliver basic public services. The Overseas Development Institute estimates that OECD governments spend more than $10 billion annually on such governance interventions, and to date, more than 40 developing countries are undertaking some form of civil service reform as part of this broad agenda (Goldfinch et al. 2012, Hasnain et al. 2012).
New Evidence from Nigeria
Despite these drives to reform public service delivery, the evidence base linking how bureaucrats are managed to the effectiveness of public service delivery remains thin. Our new working paper provides new insights into this relationship by studying the correlates of effective public service delivery in Nigeria – a country that has largely retained the British Civil Service structures since independence and the move to a Presidential democracy (Rasul and Rogger 2013).
To do so, we combine data sources linking the outputs of government bureaucracies with details of how bureaucrats are managed. For public service outputs, we use project level data measuring the completion, quality, and complexity of over 4,700 public sector projects implemented by organisations in the Nigerian civil service, including government ministries and other federal agencies. These projects relate mostly to construction projects such as roads, buildings, and small-scale dams – essentially the ‘nuts and bolts’ of rural infrastructure in Nigeria.
Project Completion Rates
Project completion rates were assessed by independent teams of engineers and members of civil society. Completion rates are scaled between zero and one, and provide an accurate measure of the extent to which public projects are completed relative to what was originally prescribed by the National Assembly in the technical documents for each project. Figure 1 shows a histogram of project completion rates for the 4,700 projects we hand coded information on.
Two striking features emerge:
• First, 38% of projects never even start.
An obvious explanation for this is corruption within bureaucracies – Nigeria ranks high on most international corruption indices.
• Second, only 31% of projects are fully completed.
Our study explores how these project completion rates correlate to the management practices bureaucrats operate under.
Figure 1. Project completion rates
To measure such management practices, we adapt the methodology set out in Nick Bloom and John Van Reenen’s pioneering work (Bloom and Van Reenen 2007, 2010). Earlier Vox columns have described the impact of management practices on explaining cross-country productivity differences in private sector firms, and on firm behaviour in developing countries. We adapt this method to measure management practices for bureaucrats in Nigeria, and then link these multiple dimensions of management to the quantity and quality of public services delivered by those bureaucratic organisations.
Our results confirm that two dimensions of management practice for bureaucrats matter above all.
• First, we document a significant positive correlation between the provision of autonomy to bureaucrats and public services delivered.
This finding provide support to the notion that public agencies ought to delegate some decision-making to bureaucrats, relying on their professionalism and resolve to deliver public services (Simon 1983). The evidence is less supportive of the notion that when bureaucrats have more agency or organisations are more flexibly structured, then they are more likely to pursue their own objectives that diverge from societal interests, resulting in fewer public services being delivered. In the Nigerian context, a strong prior might have been that granting civil servants more autonomy would necessarily lead to more graft and less effective public service delivery. This is not the case. Hence corrupt practices, while prevalent and important drivers of public service delivery, are not the entire story.
• Second, we find that the use of incentives for subjectively-determined ‘good performance’, the collection and use of key performance indicators, and efforts to monitor bureaucrats, all tend to decrease public service delivery as measured by project completion rates.
The broad use of incentive schemes tends to backfire in this public sector setting, because well-targeted incentive schemes are near-impossible to construct without creating other distortions. These distortions arise because bureaucrats are engaged in complex tasks, the relationship between bureaucratic inputs to outputs is uncertain, and there is considerable ambiguity in the design and implementation processes for many public projects. Hence the incentive systems we capture may place excessive regulatory burden or ‘red tape’ on bureaucrats, say through the use of mis-targeted key performance indicators. Such distorted incentives have long been argued to lead bureaucrats to mis-allocate effort towards less productive activities (Kelman 1990). Alternatively, our management practices related to performance incentives and monitoring might also pick up elements of subjective performance evaluation that lead to other dysfunctional responses among bureaucrats – especially engagement in influencing activities to curry favor with senior management.
Differences with the private sector
Our findings sound a note of caution towards the good governance agenda of reforming civil services throughout the world. Whilst some practices such as increased autonomy for bureaucrats might be encouraged, other dimensions such as the use of explicit incentive schemes or the drive towards payment-by-results schemes (that are occurring in the context of the British civil service) might not be optimal, and could even backfire. In short, there should be no simple import into the public sector of management practices that have proven to be effective in private sector settings. The nature of tasks varies between the private sector and bureaucracies, the objectives of bureaucracies are far less clear-cut than those of private sector firms, and there are fundamental differences in the contracting environment more generally (Dixit 2002).
One difference that is especially relevant is the lack of specialisation we see across civil service organisations in Nigeria. Civil service organisations are often tasked with implementing multiple project types, so that the same ministry might well be responsible for the construction of buildings, boreholes, and small-scale dams. For each type of project, the best management practices in terms of autonomy or incentive provision might well differ. Hence this lack of specialisation suggests organisational project portfolios might simply be too wide for them to be able to optimally fine-tune their management practices for an individual project. This non-specialisation may be optimal given complementarities in the production function of public goods. However, their varied workload may make the effective management of bureaucrats more difficult.
Our work also highlights other aspects of civil service architectures that likely interact with the effectiveness of management practices for bureaucrats. These include the process by which individuals are selected into the bureaucracy to begin with, and the labour market rigidities that characterise the civil service – the bureaucrats we study typically enjoy long tenure within organisations, with little movement across organisations. Such immobility can hinder the spread of best practices in such bureaucracies. Finally, a lack of competition and lack of price signals across organisations delivering the same project types reduces incentives for organisations to further improve their management practices. All such issues should be studied jointly in future work.
Understanding what drives the effective functioning of bureaucracies matters to economists. From a macroeconomic perspective, the effective functioning of the bureaucracy is an important determinant of poverty, inequality, and economic growth as stressed by the state capabilities literature (Besley and Persson 2010, Acemoglu et al. 2011). Effective public service delivery also matters from a microeconomic perspective – program evaluations of micro-scale interventions are often partly motivated by the assumption that successful interventions can be faithfully scaled-up by governments.
Our work is an attempt to provide among the first wide-scale evidence in a developing country context of how public service delivery relates to the management practices bureaucrats operates under. The results have implications for bureaucracies in developed and developing countries. In order to push forward this research agenda, future work will need to develop richer models of the contracting environment in which bureaucrats operate. To empirically test such models, we also need to advance methodologies and collect data that better measures bureaucrat activities and how they link to public service delivery. This defines a rich agenda for future work.
Authors’ note: This research was funded by the Federal Government of Nigeria, the International Growth Centre, the Economic and Social Research Council, the Institute for Fiscal Studies, and the Royal Economic Society.