Home
Horjus & Partners
Onderzoek & Advisering


H&P


You are connected to the site of Horjus & Partners. H&P is a small firm with potential. This site cannot contain the full range of it so you will have to be content with just a glimpse of it.

Most of the research we do has to do with the local and regional situation, but there is one subject that is interesting for the international market. It concerns effect-evaluation.
In cooperation with dr. C. Baerveldt of the University of Utrecht we developed a model to evaluate and to compare interventions
The method is described in the following publication that you can find on the list of publications. We give here the summary of the article.

Transparent boxes. Group approachment or the achievement of competence. Presentation and illustration of a method to compare effects of interventions in residential institutions.

In many cases the results of effect studies of larger prevention programs and institutes cannot be interpreted because the fact is neglected that those institutes combine various interventions, which often exceed the official philosophy or program theory of the institute. Here a method is presented that explicitly tackles this problem. Also, the method accounts for the fact that interventions are often directed at behaviour within the institute, whereas the objective is to change behaviour outside the institute. The method presented is first used in 1995 to make a comparison of two juvenile correctional institutes. This “transparent box” method is used several times in program evaluations in the years to follow. In 2001 it has been used a pilot-study involving two Dutch institutes, an institute for achievement of competence, and an institute following a strict group interaction model. You can find documentation in our list of publications and further on our site (up to this moment in Dutch only).

To compare effects of programs with different program theories and interventions we need:

  1. a model in which all programs fit
  2. an analysis of the practice done in the programs in the terms of the model in 1
  3. an effect evaluation done in the programs using the terms of the model in 1
  4. a data analysis to relate 2 to 3

1 The model all kind of programs can fit in
To describe the effort and effects of institutions the mechanism of a “minimal theory” is used. This minimal theory must be very general and have little detail so that it can be applied on a broad range of program theories, intervention programs and also on the target group. The minimal theory that we found most appropriate for our purpose is a variant of the theory of planned behaviour of Aizen (figure 1).


Figure 1: The ASE Model.



This theory gives us some terms that determinate behaviour (attitude, perceived social norms, effectiveness and a threshold). The attitude towards an object or behaviour is founded on one’s values and the perceived costs and merits of the object behaviour. The next term is the perceived social norms of relevant others. People tend to take this in account when planning their behaviour. The (perceived) effectiveness or self-efficacy is an important influence on behaviour. If one doesn’t believe he is able to show a certain behaviour he will not come to the intention to do so. When on the other hand these aspects are favourable toward a behaviour it is likely that we develop an intention or plan to act.
By adding the threshold to this model we can take two situational aspects in account. Firstly the aspect that a threshold can keep people from showing certain behaviour (e.g. the closed door of the correctional institute). Secondly the aspect concerning proximal effects and distal effects. For instance behaviour learned in one situation under optimal conditions is hard to transfer to a real life situation. The influence of an intervention on the threshold can give insight in these transfer (im-)possibilities.

2 Program analysis in the terms of the model in 1
All goals of interventions/programs/program theories can be translated in these terms. We give an example in figure 2.

Figure 2: The profile of some interventions in correctional institutions

Intervention

Attitude

Social

norms

Efficacy

Threshold

Values

Weighing merits and costs

 

 

 

x

 

 

 

 

x

x

x

 

Social skill training

 

 

x

 

x

 

Family therapy

 

x

 

x

 

 

Individual coaching

 

 

 

x

 

 

Job coaching

 

 

x

 

x

x

Being locked up

 

 

 

 

-x

x

Confrontation with consequences of the crime

x

x

 

-x

 

Note that these interventions above are not necessarily given in one institution. Being able to place the interventions in the table shows that it is possible to compare them. An analysis is done by dismantling the complex intervention into components. One by one the interventions are observed and described. After this a panel of experts is asked to rate their effect on the terms in the model. The outcome can be seen as a quantitative hypothesis of effect. In our method these findings can individually be coupled to the persons in the experimental group. Individually, because we found that not to everyone in a program the same interventions are applied. The model can also take in account that programs tend to change during the research period. In a black box design this is lethal to validity. In this model new or changed interventions can get new ratings.


3 An effect evaluation using the terms of the model in 1
Effect evaluation can be done by introducing the items in a quasi experimental design.
Result of this is that target groups can get a profile in these terms.


4 A data analysis to relate 2 to 3
The outcomes of the effect evaluation can be related to an analysis of the practice in the program.
In analysis the model has some advantages
• we can identify the differences in effects between the programs in the comparison in terms that are alike
• we can identify the interventions in the programs that are more and less effective
• we can make assumptions about which interventions should be added to make programs more successful
• we can identify the aspects that are most likely to be influenced successfully for each target group. For instance hard-core criminals are expected to found their choices on the opportunistic weighing the costs and merits of their behaviour (Attitude). Youngsters from marginal groups tend to see themselves as insufficient to fully participate in the conventional society and can be influenced through empowerment (Effectiveness) and group activities (Social Norm = conforming to the social norms of peers). In this way we can also make a distinction between the several target groups of the programs in the comparison.


Keywords: delinquency, effect-evaluation, correctional institutions, adolescents


This description is short, but if you think this is interesting and you and want to know more of this model and its applications, please contact us on H&P.