New York State Assembly
     Albany, New York

    Legislative Commission on
    Government Administration

 Sheldon Silver, Speaker
 Marty Luster, Chair

Performance measurement has a long history extending back to early attempts to apply scientific management principles to government. Different forms of it have been attempted at the state, federal, and local levels on a regular basis since the 1950s.

The central issues, then and now, revolve around accurately defining and developing measures that reflect the activity they are designed to measure and providing the necessary conditions and incentives for such measures to be used in decision-making. One of the goals of performance measurement is to simplify the task of assessing the effectiveness and efficiency of organizations and programs. Part of this simplicity is the assumption that implementation should be easily undertaken. Yet the task historically has proven anything but simple as vast amounts of time have been consumed in the process of performance measurement rather than in doing the mission of the program.

While few disagree in principle with the concept of performance measurement and performance budgeting, New York and other states have not had a great deal of success implementing it. These setbacks have not, however, dampened enthusiasm for revisiting the subject framed as it has been most recently by the “reinventing government” movement.

Performance measurement can be a difficult task for any organization, public or private. In general, the private sector has had greater experience and greater success with performance measures. In the private sector there is often better understanding of the relationship between the underlying technology of the organization and its outputs. And the outputs of the organization are subject to the discipline of the market.

In the private sector:

 market prices produce valid measures;

 the corporate structure involves all policymakers using the same valid measures; and

 the measures lead to budget choices and policy making.

In the private sector it has generally been assumed that the discipline of the market moves firms towards cost efficiency and profits. The market system, although imperfect, still provides for economic (as distinct from social) indicators of performance with such measures as profits, rates of return on investment, and market shares. The market provides measures that can be valid across different firms or organizations as well as across levels within firms. Performance measurement cannot fully succeed in the public sector until such validity has been achieved; the reported measure truly reflects the targeted activity and can be compared or summed across organizations or activities.

Definition of Measures

Performance measures can be represented in a range of forms including effectiveness, efficiency, workload, cost effectiveness and productivity. The most important measures focus on effectiveness and efficiency:

 Effectiveness emphasizes how well a program meets its goals or citizen needs. This is often discussed in terms of the quality of the service provided.

 Efficiency compares the quantity of service provided to the resources expended; is the service provided at a rea- sonable cost? Efficiency measures concentrate on how well a task is being performed, not whether the task itself is correct.

A common problem in performance measurement is the reporting of process measures (workload) or input measures instead of output measures (such as effectiveness and efficiency). Such indicators as client/staff ratios, resources per capita, number of reports issued may be useful in understanding what an agency does or how busy it is, but they do not measure results. Such data, however, are easily gathered and, without clear definitions or communication, are often reported as performance measures.

Most programs cannot be summarized with one indicator. Most well-regarded efforts rely on multiple measures and multiple forms of measurement. The simple application of a proven measure in one jurisdiction to another may also be a bad strategy, given the variance in jurisdictions and services delivered. Considerable experimentation may be required to develop an elegant solution: a relatively small number of indicators, easily produced, that capture the intended agency activity.

Performance Measurements and Performance Budgeting in Other States

Many state and local governments have attempted performance measurement systems as fiscal pressures have increased and efforts have revived to assess government activity and improve accountability for progress and outcomes in acheiving results. Despite widespread experimentation, the record on well-regarded performance measurement systems is still limited, particularly at the state level. For further discussion of other states' experiences as well as new state initiatives, please see recent reports by the National Conference of State Legislatures (“The Performance Budget Revisited: A Report on State Budget Reform” Legislative Finance Paper #91, February 1994) and the United States General Accounting Office (Performance Budgeting: State Experiences and Implications for the Federal Government, February 1993).

While few disagree in principle with the concept of performance measurement and performance budgeting, New York and other states have not had a great deal of success implementing it. Reviews of successful performance measurement and budgeting often must focus on the municipal level where services and citizen feedback are more easily measured. (Reinventing Government, for example, although intended for every level of the public sector draws many of its cases from local government experience.)

The problem of definition and use of measures is exacerbated at the state level by the breadth and complexity of state government:

 State level tasks are often less directly involved in service provision compared to local governments. Therefore, it is often harder to develop measures that relate inputs and out- puts.

 State government is more organizationally complex, and systemic change is harder to implement.

 The greater number of local governments provides a greater base for comparison and successful experimentation.

Several states, including Oregon and Texas, have recently inaugurated new performance budgeting systems. Others are debating such initiatives.

Experiments with performance measures have been undertaken in many state and local governments with a range of experiences. The concept of performance measurement is widely endorsed, but successful implementation depends more on specific circumstances such as institutional experience, external pressures and political leadership.

Conditions for Use

An equally important issue in performance measurement is connecting indicators to the decision-making process. Efforts may be made to precisely define the ideal measures, but success demands a linkage to the decision-making process. And this must be done at different levels for different decision-makers. Precise and well-defined measures are of considerably less value if not connected to decision-making. Legislators look to performance measures to provide the basis for decisions across programs and agencies. Agency managers are more likely to be concerned with internal efficiencies and with a shorter perspective. Employees need to be able to relate measures back to their own performance. It is very unlikely that the same measures will be equally useful at all levels; measures that are not relevant then fall into disuse. Unless those responsible for producing the measures see some utility in the measure they have little incentive to produce accurate good indicators. A core problem for public sector performance measurement is the inability to aggregate measures. There is no equivalent to the bottom line in the marketplace. Even though it may be imperfect, the market still provides economic indicators of performance about managers’ decisions, and experience and economic theory have shown that there is a relationship between elements such as quality control, turnover or computing costs and profits.

Key Item Reporting

The 1992 legislative session saw the end of New York state’s recent experiment with performance measurement with the signing of Chapter 762. Observers and users agreed that the system did not provide useful information on agency or program performance. The repeal of key item reporting (KIRS) reflects some of the central problems in designing and implementing performance measurement systems.

 Design and Intent

The central concept was the “key item.” According to the KIRS approach, agencies were to develop specific indicators that represented performance of specific programs. Rather than a comprehensive effort to develop indicators for all agencies or impose a program structure, these key items would serve as representative, not summary, indicators of agency activity. To avoid implementation problems, key items would be developed on a pilot basis gradually expanding to cover more agencies.

 Implementation

Despite the graduated implementation, KIRS experienced problems. Although the number of agencies reporting key item measures increased, basic issues concerning the definition and use of such measures were not resolved. Measures were not well defined or explained and often represented workload or other process-oriented indicators rather than efficiency or effectiveness. Split responsibility for system implementation created confusion and little incentive for users: agency managers developed measures according to standards set by the Division of the Budget for principal use by the legislature. The formal opportunity for joint legislative input into the specification of key item elements was not realized.

The measures developed by the agencies did not meet legislative needs for measures that reflected program activity. Measures as presented reflected only parts of programs or activities and generally could not be aggregated to provide an overall program or agency measure that could be directly related to legislative decisions. Agency programs, as presented in the budget, contain several activities, and only part of one activity was represented in the key item report. The measures could not be related to the budget. The KIRS reports covered only results or projections for the current year. This limited perspective made judgements about relative progress from year to year impossible. From the agency perspective, little incentive existed to develop measures that were not directly related to internal agency management and decisions. KIRS was further handicapped by the existing accounting system which does not accurately attribute costs to programs. For example, fringe benefits are reported as General State Charges rather than allocated to the specific program where the costs were incurred.

Failure to resolve these problems led to decreased use and decreased support in the executive and legislature. KIRS was developed by the legislature, and necessary long term cooperation between the branches to implement such systems never emerged. KIRS was an early recognition of the importance of performance measurement, and in some cases it did provide useful information about the activities and performance levels of agencies.

Conclusion

Performance measurement and performance budgeting are important issues on the public agenda and there is little disagreement about their worthiness. In order to succeed, performance measurement systems should:

 sustain the necessary long-term cooperation between the executive and legislature;

 keep the measures connected to budgetary decisions; and

 have reliable and valid measures of costs and activities.

Few governments have been able to develop, implement and maintain a comprehensive approach. A more likely strategy for most jurisdictions seems to be an approach of further refinement of measures within functional areas and success based on local experience and the voluntary adoption of measures that have been shown to be effective. Performance measurement can be used to inform decisions, but the state of the art has not yet developed far enough to be an equivalent of the market.

Carter, Karen. February 1994. "The Performance Budget Revisited: A Report on State Budget Reform." Legislative Finance Paper #91. Denver, Colorado:National Conference of State Legislatures.

Hatry, Harry P. et al. (editors). 1990. Service Efforts and Accomplishments Reporting: Its Time Has Come: An Overview. Norwalk, Conn.: Governmental Accounting Standards Board. (See also specialized volumes under same general title for Elementary and Secondary Education, Water and Wastewater Treatment, Mass Transit, Sanitation Collection, Fire Department Programs, and Public Health and a general Concept Statement.)

Osborne, David and Ted Gaebler. 1992. Reinventing Government. Reading, Mass.: Addison-Wesley Publishing Company.

South Carolina Commission On Government Restructuring. 1991. Modernizing South Carolina State Government For The Twenty-First Century. Columbia, S.C.: State of South Carolina.

United States General Accounting Office. February 1993. Performance Budgeting: State Experiences and Implications for the Federal Government. Washington, D.C.