The European Evaluation Society (EES) has its biennial conference “Building for the future: Evaluation in governance, development and progress” on October 1-3, 2008
in Lisbon, Portugal.
CPPE Head Daniel Tsygankov
presented a paper “Policy makers’ evaluation use - between legitimating and rationalizing”
on Thursday, October 2, 2008:
Stream 7 “Encouraging evaluation use”
Paper Session 7.6. “Shaping policies?” (17.00-18.15)
Although in recent decades the number of evaluation research carried out is rapidly growing, the use of evaluation results in political practice is still not a general rule. The information on evaluation, its concepts and arguments is gradually percolating through society by means of force groups, expert influence and mass-media.
Simplifying the policy makers taking political and administrative decisions pursue two groups of objectives. The first group is scientifically centralized objectives (knowledge may increase the rationality of political practice), the second one is applied objectives (the legitimating of decisions, political win). In accordance with this fact the supporters of amplification of use/integration of evaluation into public sector speak of challenges of modern administration “efficiency”, the critics refer to the “verification rituals” of modern auditing society (Michael Power). Any way no one denies forming of new audit culture (Marylin Strathern) which can affect social reality (e.g. by means of forming of performance indicators system).
The advocates of evaluation implementation themselves perceive both weak professionalism in some countries and “shapeless” evaluation (everything that names itself in such a way is accepted just so by those as well) with an anxiety, and neatly characterize them as pseudo-evaluations or quasi-evaluations (Daniel Stufflebeam). Here we put aside those evaluations and surveys which are objectively useless due to their methodological mistakes.
Therefore the development of evaluation as public practice and research strategy can be analytically placed on the axis between the poles of “Rationalizing”/“Legitimating”, as for further analysis of chances of its institutionalization the new dimensions can be introduced (institution environment, waves of public sector reforms, development of boundary disciplines, structure of the power field / the intellectual field, etc.).
The understanding of consistent patterns of such development possesses not only pure scientific interest, but it is also important in principle for the countries in transit which did not enter the influence orbit of countries and international organizations with formed evaluation institutes. For such countries there is a risk the relegation of evaluation to the “Legitimating” pole is high.
The peculiarities of formulating of conditions and evaluation topics by policy makers can point out such a “removal” from rational pole (the topic is contradictorily, confusingly and at length formulated; customer’s representative avoids explaining research topics; research reports are never published; the companies which are absolutely unknown in the market participate in tenders; the price of the contract obviously contains “a corrupt constituent”).
Therefore there is a question whether the existing approaches (creation of national association and work-out of evaluation standards, development of master programs and programs of further education, publication of magazines and books in mother-tongue, etc.) will be enough for risks reduction or whether it will be necessary to take non-standard steps which will contribute to more rational use of evaluation as cross-border discipline.