When agents of change go to extraordinary lengths to facilitate collaboration among mental health and criminal justice stakeholders, which leads to the development of new and exciting initiatives to improve the systems' response to people with mental illness, it is essential that they measure and evaluate the impact of these efforts.  Too often, policymakers exhaust time and resources planning and implementing a new program, policy, or statute without taking the steps to ensure that they will know the results of the initiative.  By then, administrators need additional resources to sustain the initiative, yet appropriators are insisting upon some evidence describing the impact of the program before authorizing the expenditure of additional funds.

Indeed, policymakers and organization executives are right to demand such information.  It often rewards the initial decision to authorize the allocation of resources to a particular initiative with data illustrating the benefits of a new program.  The results of an objective, thoughtful evaluation also signal how an initiative can be improved.  Furthermore, the evaluation process itself facilitates quality control; not every good idea is implemented well.  Sometimes the results of a study reveal that a new program, policy, or legislation has had a negligible impact on a problem, or occasionally even exacerbated it.

The section of the Introduction to this report entitled "Getting Started" explains that an essential first step for any jurisdiction interested in improving the response to people with mental illness is to identify the problem (or problems) that leaders in the criminal justice and mental health community can agree to address.   This chapter assumes the existence of such an agreement about the problem; the first policy statement underscores the importance of establishing practical measures of success, which will allow program funders and program administrators to determine whether they have addressed the problem. [1]  The second policy statement in this chapter reviews the elements of a program or policy that will support the data collection needed to measure the outcomes identified. The last policy statement in the chapter assumes the change agent has helped analyze an initiative's successes and failures and discusses disseminating the findings.

Evaluations can be extraordinarily complex and expensive undertakings.  The policy statements in this chapter suggest how policymakers and practitioners can measure the impact of an initiative practically and efficiently.  That said, any effort to obtain reliable and useful information describing an initiative's outcomes requires some resource allocation.  Examples cited elsewhere in this report sometimes include a provision requiring state or local government officials to use a portion of the funds allocated to evaluate the impact of the program. [2]  Partnering with local universities is one way to conduct an evaluation and maximize the use of existing resources. 

The value and usefulness of a program evaluation often corresponds to the degree to which various stakeholder groups are involved in identifying outcome measures, developing a data collection process, and disseminating the findings.  Extensive collaboration inevitably enhances the quality and efficiency of the evaluation.  Equally important, it vastly improves the likelihood that significant segments of the community will accept the findings that the evaluation yields.  This chapter does not address the oversight of the evaluation.  (For a discussion about how to collaborate effectively and establish and institutionalize partnerships, see the section of the report Introduction entitled "Getting Started" and Chapter V: Improving Collaboration.)