Health-Related Evaluation Methods Flashcards Preview

Directing Benefits Programs > Health-Related Evaluation Methods > Flashcards

Flashcards in Health-Related Evaluation Methods Deck (21)
Loading flashcards...

What is the formative realist case evaluation example

Involved an evaluation of a difficult to describe large scale "transformation" of health services and its related report.


What are the overall questions considered in all realist evaluations?

What works, for whom, under what circumstances?
What are transferable lessons about effective change?O


Theory or logic theory whose aim is to explain how an intervention is understood to work. It used mixed data sources and collection methods of ethnographic observation, semi-structured interviews and document analysis

Realist context-intervention-mechanism theory


Summarize the "lessons" that were reported related to Mechanism 1 from the formative realist case evaluation example

1. Relationships between organizations are trusting

2. Approaches to integration are imaginative, locally responsive, negotiable and supported by technology rather than rigid and driven by tech

3. External incentives are designed to reward collaborative performance and do not make organizations compete

4. The strategy for integration includes both "soft" and "hard" approaches

5. Solutions are participatory rather than developed by one party and imposed on others


The generalizability of the findings in this type of formative realist case evaluation example can be increased by ___________________.

Providing details of the change actions and of the context


The description of the mechanisms or change principles and how they were triggered by the intervention in the setting enables others to try to ________ the mechanism in their different settings, possibly using different ___________.



How does this type of formative realist case differ from many other forms of evaluation, particularly experimental evaluations?

Gives a narrative understanding of the process of change in an environment rather than showing a few measured outcomes attributed to a change


What difficulties are involved with formative realist case evaluations?

1. Distinguishing between context and mechanism
2. Is there an end point?
3. When to "close"/
4. Time-consuming
5. Can realist evaluation studies be replicated?
6. How does realist evaluation differ from traditional mediator/moderator analysis?


Describe the reasons to evaluate a reform before it starts, during its implementation and after its implementation

Assess whether it is feasible, to predict problems and to help plan how to implement it


During a reform, an evaluation is to help make corrections _________ implementation



After the reform, an evaluation helps to explain the lessons for _______ changes or reforms and ________ to scientific knowledge that could help the reformers explain, understand or predict the effects of _____________ changes.



List and explain the usual steps for evaluation a reform

1. Describing the reform "instrument"
2. Gathering data about changes in health service performance and possibly also in health
3. Assessing whether these data really are outcomes of reform


The aim of the reform is often to do the following

1. Present evidence of the results of the reform
2. Describe the process of implementing the reform
3. Assess the strengths and weaknesses of the reform as implemented and judge the value of the results
4. Recommend improvements


In most reforms, the intention is to change the organization, procedures, financing methods and other "things".

Immediate subject or target


Reform that is often aimed to improve patients health or well-being

Long term aim


Intervention that does not work directly on the _________ _________ (patients or the population) but works through changing the immediate subject (health organization) in order to achieve the ultimate benefits or better health.

Ultimate beneficiaries


4 designs typically utilized to evaluate health reforms

1. Implementation designs
2. Comparing achievements with reform objectives
3. Before/after design
4. Comparing one site with one of more others implementing the reform


Designs that report on strengths and weaknesses of the process and make explicit the criteria used to assess the process. For example, one criterion might be whether all stakeholder groups were involved.

Implementation designs


Compares the intended goals of the reform with the extent to which they are achieved

Comparing achievements with reform objectives


This design describes the reform implementation process but also gathers data before the reform and then after the reform has been proceeding for enough time to have an effect.

Common criteria are costs, efficiency, access, equity, quality and service comprehensiveness.

Before/after design


This comparative design gathers data about different sites or areas implementing the same reform. It can be used cross-nationally to compare similar types of reform.

Comparing one site with one or more others implementing the reform