Effective policy does not end with implementation; programs, initiatives and policies must be continuously evaluated to make sure they are reaching their milestones and goals. At PSC, consultants work with clients to design and conduct evaluations in the policy areas of health, education and environment and energy. Evaluation may be focused at the program, initiative or policy level, and can be used to inform decisions made by program administrators and staff, governing boards, funders and policymakers. Findings can also convey the value of programs and services to potential customers.
When it comes to the evaluation process, PSC takes a participatory approach — meaning the people who are going to use the evaluation results help inform the process and remain involved throughout. By including these key individuals, PSC’s team is able to develop the best possible plan, and participants are more invested in the results.
“We work very closely with our clients and all the program stakeholders to ensure our evaluation work meets their needs for information and that it contributes to the improvement and success of their initiative,” says Jane Powers, vice president in PSC’s health division. “Ultimately, we want what the stakeholders want — the best outcomes for the people they’re intending to serve.”
The PSC evaluation framework incorporates traditional tools — logic models and theories of change — to identify the outcomes that the program, initiative or policy is intended to achieve. The framework also includes measures that will be used to determine if an initiative did or didn’t work and why that happened.
Once the basic components of the evaluation plan are in place, the PSC team uses the process, intermediate outcomes and long-term outcomes to determine what methodologies will be used to collect information — including surveys, focus groups, key informant interviews, background materials and client reports. After the PSC team gathers and analyzes all the necessary information, the findings are synthesized and provided to the client and stakeholders in regular reports that help the client stay on track.
“Evaluation allows the people who are implementing programs or initiatives to track how well they’re meeting their goals and objectives throughout the implementation and planning/development stages,” says Powers. “Information is fed back as part of a continuous improvement cycle so they can make adjustments and be more successful in their efforts.
“We strive to give stakeholders the information they need to achieve their goals and objectives.”
Detroit Economic Growth Corporation uses PSC evaluation assistance for D2D program
The Detroit Economic Growth Corporation’s Detroit-to-Detroit (D2D) program was designed to grow local business-to-business procurement opportunities. The program works to increase business readiness and competitiveness in order to secure contracts.
When funders were interested in evaluating the impact of the D2D program, DEGC turned to PSC, who had worked previously with the DEGC on program evaluation.
PSC identified the strengths of the D2D program (such as the networking opportunities provided), and offered recommendations on ways to help improve the program (such as how to strengthen data collection processes). The findings can be used to help guide future modifications so the program can continue to achieve its goals.
Melissa Gibson, senior consultant at PSC, said the DEGC did a very good job of listening to respondents throughout the D2D program’s development and implementation stages. The DEGC also responded to what businesses said they needed by making adjustments to the program, she added, which is why they achieved the results they did.
“The D2D program is designed around making local business connections. It doesn’t distribute money to encourage participation, but people keep coming back because they’re getting value for their businesses,” says Gibson. “They are willing to take the time to attend D2D events because of the networking and business development opportunities offered.”
Michigan Health Endowment Fund initiatives utilize PSC’s evaluation expertise
The Michigan Health Endowment Fund (MHEF), created through passage of Public Act 4 of 2013, supports efforts to improve quality of health care, as well as reduce costs, and benefit the health and wellness of children and seniors throughout the state.
PSC has been contracted for evaluation work by three of these organizations: the Area Agencies on Aging Association of Michigan, the Michigan Association of United Ways, and the Michigan Association for Local Public Health. PSC works with key stakeholders in each of these initiatives, such as state-level administrators and governing boards, as well as local program-level directors.
PSC began their work at the start of these initiatives and will continue to work closely with throughout the entire project period — this includes completing evaluation design, data collection and analysis, summary reports to stakeholders, and annual and final reports, all of which will be used to determine progress and guide modifications. PSC’s evaluation work will also be used to provide updates to the MHEF board regarding progress and results.
Better communication with boards and funders is one of the common benefits of PSC’s evaluation work. But beyond that, evaluation work truly effects change when clients use PSC findings to make modifications to meet or surpass their goals.
“The value is that evaluation will improve the implementation of the initiative and the outcomes,” says Powers. “In the case of the Michigan Health Endowment Fund initiatives, when an initiative can run the best way that it can, then we’re contributing to better health for people in Michigan.”