Linking Diplomatic Performance Assessment to International Results-Based Management

Introduction

Performance assessment of the public sector has become increasingly common in both national governments and international organizations. It derives in large measure from the ‘reinventing government’ movement of the 1990s, which was intended to make governments more effective (Osborne and Gaebler, 1993). In this model, governments are expected to deliver goods and services that will achieve public objectives. Funding priorities should be based on the effectiveness of programs, with resources allocated to those programs that can demonstrate results.

Determining results had been a long-term concern in some international public activities. In the late 1960s, the United States Agency for International Development commissioned a private consulting firm to develop the logical framework, an approach to project design and evaluation that is now in common use.

Performance assessment is now part of the repertoire of most governments. It is intended to help show to parliaments that the funds appropriated for programs have been used wisely. In the United States, for example, the Program Assessment Rating Tool (PART) “was developed to assess the effectiveness of federal programs and help inform management actions, budget requests, and legislative proposals directed at achieving results” (United States Office of Management and Budget, 2004a, p. 1). Similarly, the Canadian Treasury Board mandates the use of what it terms ‘Results-based Management and Accountability Frameworks’ (Canada, Treasury Board, 2005).

The application of performance assessment has been particularly difficult in foreign ministries, especially when dealing with international organizations. The problem rests in determining what elements of the performance of international organizations can be linked to the output of the foreign ministries. Because the results obtained from funds sent overseas are not very visible either to the public or to the parliaments, foreign ministries are under considerable pressure to provide convincing evidence of performance.

To examine this problem, we look first at what an assessment by foreign ministries implies, with a case study of the United States State Department. Then, we examine how international organizations assess performance and explore whether a reasonable link can be drawn between national and international assessment.

Performance Assessment in Foreign Ministries

The main function of foreign ministries is to ensure that the nation’s interests are successfully pursued in relations with other nation-states and in international organizations. Determining whether these efforts have been successful, however, is not as easy as it would seem. Part of this is due to the fact that whether an expected result is obtained does not depend completely on the actions of the foreign ministry. Results are heavily conditioned by external factors such as the positions taken by other states, influential non-state actors, and the events in the external environment, all of which are outside the control of the foreign ministry.

Unlike domestic ministries, which can determine the effectiveness of programs by seeing whether roads and infrastructure are built, taxes are collected and services are used, all of which are measurable and can be influenced directly by the ministries concerned, the results of the work of foreign ministries are typically indirect, difficult to measure and not easy to influence directly.

The first problem in the performance assessment in foreign ministries is determining what the national interests are. For some nation-states, this is determined by tradition (what has always been the country’s position), but for most it is itself a complex issue. Beyond a general statement favoring peace and prosperity, more specific interests involve a complex relationship between history, values, immediate and longer-run advantages, as perceived by a nation’s leaders. Often these interests are neither clear nor consistent.

The second problem is determining the expected results in terms of these interests. Most foreign ministries are reactive rather than proactive. Their job is to defend national interests—as they are defined—when these are threatened. Promoting these interests requires taking advantage of propitious situations that cannot always be predicted, or seeking to structure the environment so that the interests can be promoted.

In bilateral relationships, a main expected result would be an agreement with another state. This can be measured by the existence of memoranda of understanding, treaties, joint statements, and the like. Separate indicators might include increases in trade after agreements have been signed, a reduction in illegal border crossings, increased extradition, all depending on the substance of the agreements reached.

The problem is greater at the international level, because the national interest is pursued in a complex multilateral environment. In many contexts the national interest is not at all clear. However, one obvious expected result would be that multilateral agreements reflect national concerns. Another would be that funds given to multilateral institutions were used in ways acceptable to the country providing them.

The difficulty in linking these international developments to national performance was illustrated by an effort by the United States Department of State to undertake a Performance Assessment Review Tool (PART) on some of its international programs.

The Case of PART

The United States federal government is mandated under the Government Performance and Results Act of 1993 “to identify both long-term and annual goals, collect performance data, and justify budget requests based on these data” (United States Office of Management and Budget, 2004b). In order to implement the law, the Office of Management and Budget in 2002 developed the Performance Assessment Rating Tool (PART). The PART was not meant to measure program performance per se (in terms of the outputs and outcomes), but rather was a measure of how well a program is designed, planned, managed, and achieves results. The results are measured in ways determined by the agency, which allows the ministry concerned to determine its own performance measures and report on them. The PART consists of approximately 30 questions (the number varies depending on the type of program being evaluated), asking for information which responsible federal managers should be able to provide. For instance:

• Is the program designed to have a significant impact in addressing the intended interest, problem, or need?
• Are federal managers and program partners (grantees, sub-grantees, contractors, etc.) held accountable for cost, schedule, and performance results?
• Has the program taken meaningful steps to address its management deficiencies?
• Does the program have a limited number of specific, ambitious, long-term performance goals that focus on outcomes and meaningfully reflect the purpose of the program?
• Does the program (including program partners) achieve its annual performance goals?

These proved difficult to implement, in part because the questionnaires were designed for domestic programs. Custodio (2006) undertook an analysis of the PARTs applied to two programs, Contributions for International Peacekeeping Activities and the United Nations Development Programme (UNDP). He found that in the first case the PART conclusion was positive, and in the second it was negative. In neither case did the results affect the funding for the program. Custodio argued that in part the legislature ignored the findings, but he also suggested that one problem was that the method of determining results was flawed, in that the indicators did not take into account the context in which results appear.

The question then becomes, how can measures be created that will demonstrate national foreign ministry performance, especially in multilateral organizations?

Resolutions and Decisions

States belonging to international organizations reach agreements and reflect them in resolutions and decisions. Every member state of the organization can, at some point, participate in the decisions. To the extent that the decisions reached reflect national interests, joining in the decision by voting for it would indicate that the foreign ministry staff members were functioning successfully. The difficulty in this is that most international organizations adopt decisions by consensus (without a vote) and the consensus positions are determined by groups of states. For example, the European Union develops a common position on most issues—especially in the economic and social area, and most members of the Group of 77 join the common positions agreed by the group.

A state that has a significant interest in a given issue will try to take a lead role in multilateral negotiations, first within its group and then, potentially, in the general negotiations. This can be measured in terms of who prepares the group’s negotiating draft, who is given the responsibility for coordinating positions, and who speaks during informal negotiations. In terms of expected outcomes, these might be expressed as follows: the foreign ministry demonstrates leadership in an issue of priority to the country. Its indicator might be the extent to which foreign ministry staff members are given those roles by other delegations.

By adding up all of the areas in which the foreign ministry was attempting to obtain a favorable decision, and calculating the number of instances where its personnel played a leadership role, the ministry can plausibly assess its performance.

Contributions to Funds

A more complex situation exists for financial contributions. In the case of the United States’ PART on peacekeeping (United States, Office of Management and Budget, 2004c), the problem was less since no peacekeeping operation could be authorized by the United Nations without the United States’ concurrence, because the decision-maker is the Security Council, on which the United States has veto power, and peacekeeping operations are funded by assessments. Even though the national funds go into a general fund for peacekeeping, a link can be drawn to national policy. The same is not true for other general-purpose funds based on assessment, where there may be programmatic elements about which a given state may not agree. Even voluntary contributions, where a state may decide to reduce or eliminate its contribution, suffer from this problem, except when earmarked funds are involved, where a state can determine the use for which the funds are allotted.

In fact, the only way to measure the performance of national contributions is by observing the performance of the fund or program into which the state is making a contribution.

RBM in International Organizations

International organizations have increasingly adopted results-based management. This has mirrored the development of RBM in many national governments. The RBM system in most international organizations is similar, as has been noted by the United Nations Joint Inspection Unit (2004). As part of the planning and budgeting process, the organization specifies its objectives and the expected outcomes of its work during the planning period that will lead to achieving an objective. At the end of the period, the organization reports on whether it has obtained the outcomes.

One intention of this process is to demonstrate to member states that their financial contribution to the organizations has been justified in terms of results. If the performance appraisal is sound, it will provide individual foreign ministries with the evidence that their policies have been useful and the funds allocated through them have been well spent.

Linking the National with the International

While foreign ministries cannot say that their input into the policymaking or the operations of international organizations has caused the observed results, they can say that they have been associated with success. In that sense, they have an investment in ensuring that the international organizations undertake proper performance appraisals that can demonstrate that success.

A weakness in the present system of international performance appraisals is that they are done separately for each organization, using slightly different methods. As a result, it is not easy for national authorities to draw a picture of the results of their financial contributions to the programs. The reporting done by foreign ministries is inevitably piecemeal, if it is done at all. The foreign ministries of all countries, regardless of the total amount of contribution to international organization funds, should have an interest in international performance appraisal, both in terms of methods and institutional arrangements. Because of the formula for allocating assessed expenses, every country essentially pays the same in real terms. Different countries have different priorities and want them expressed in programs and budgets. This is essentially the basis for the current impasse on UN reform, in which developing countries do not wish to relinquish their ability to influence programmatic decisions, while major contributors consider this an obstacle to management.

In fact, foreign ministries of both developing countries and major contributors have a stake in good performance appraisals at the international level, since both are responsible for convincing their parliaments that the funds provided to the international public sector have been well spent. A solution to the current reform impasse would be to strengthen the institutions that review appraisals and use that information to improve programs. In the United Nations itself, the body charged with this task is the Committee for Programme and Coordination. Considered by the Secretariat and by many of the major contributors to be an ineffective body, it has the potential to perform the appraisal function if its support is upgraded and its importance to foreign ministries is recognized, as I have argued previously (Mathiason, 2004).

The potential for using international performance results in national reporting by foreign ministries is illustrated by the use of PART in the United States. For the 2006 budget, a PART analysis was undertaken of the United Nations refugee program (Office of Management and Budget, 2005). Largely using information provided by UNHCR, including program performance data, the presidential budget office could conclude that the State Department input into the international program was effective.

An effort to strengthen the links between international and national performance reporting can serve to strengthen both the foreign ministries and the international programs with which they work.

References

Canada, Treasury Board of Canada Secretariat, “Preparing and Using Results-based Management and Accountability Frameworks”, www.tbs-sct.gc.ca/eval/pubs/RMAF-CGRR/guide/guide_e.pdf, January 2005.

Custodio, Roberto, “The Office of Management and Budget’s Program Assessment Rating Tool and Its Impact on International Organizations in the Congressional Budgetary Process”, Thesis submitted for a master’s in international relations, Maxwell School of Citizenship and Public Affairs, Syracuse University, May 2006.

Mathiason, John R., “Who Controls the Machine, III: Accountability in the Results-based Revolution”, Public Administration and Development 24, no. 1, 2004. pp. 61–73.

Osborne, David and Ted Gaebler, Reinventing Government: How the Entrepreneurial Spirit is Transforming the Public Sector, New York: Plume, 1993.

United Nations, Joint Inspection Unit, Overview of the Series of Reports on Managing for Results in the United Nations System, (JIU/REP/2004/5), Geneva: JIU, 2004.

United States, Office of Management and Budget, “What Constitutes Strong Evidence of a Program’s Effectiveness?”, www.whitehouse.gov/omb/part/2004_program_eval.pdf 2004a.

United States, Office of Management and Budget, “Rating the Performance of Federal Programs”, www.gpoaccess.gov/usbudget/fy04/pdf/budget/performance.pdf, 2004b.

United States, Office of Management and Budget, “Detailed Information on the United Nations High Commissioner for Refugees Assessment”, www.whitehouse.gov/omb/expectmore/detail.10001112.2005.html, 2005.

United States, Office of Management and Budget, “Programme Assessment: Contributions For International Peacekeeping Activities”, www.whitehouse.gov/omb/expectmore/summary.10002212.2005.html, 2004c.