Starting from the nineteen-eighties, whilst the new network of community psychiatric services was being constructed, the concept of evaluation has been progressively modified.

Previously, a substantially self-referential model was applied for evaluation, based on the willingness\inclination of the work group to evaluate itself.

The metaphor of the half full\half empty glass, which allows both points of view (the glass is half full, the glass is half empty) to correspond to fact, has in reality justified the detachment with which the evaluation process was regarded, having been judged an ambiguous procedure.

The expansion of the new services has given rise to new knowledge and belief regarding the usefulness of "community based" services and the possibility of demonstrating their superiority in guaranteeing accessibility, continuity, and practical effectiveness as well as user satisfaction.

With this, the willingness to share respective experiences and adopt objective instruments of measurement and evaluation has grown considerably.

This open attitude has above all allowed Mental Health Services to be linked to other fields of medicine more used to applying epidemiological indexes and indicators of performance. This has in turn allowed selection of the types of intervention which are most appropriate and practically effective in relation to the needs of the population.

In a parallel way, the awareness that "therapy is service"; that is, that integrating treatment processes with coherent systems of therapy is a fundamental factor in achieving efficacy; has made the evaluation process extremely permeable to any and all information regarding successful organisations and the changes which such organisations have gone through in order to attain this status.

Attention to successful business strategies has also been prompted by two additional factors:

This change has been summarised in the phrase: concentrate on the client in efficiently re-modelling the system of producing Health Services.

This can be summed up in a single key-word : Quality

The new model based on quality assessment has given the Service the chance to cast aside their traditional self-referential attitude.

Well, what is quality?

From the different points of view of the operators in this field - professionals, managers, consumers – several different characteristics of quality are proposed, which can be summed up as follows:

Organizational Quality

Make up a single service quality profile and should thus be considered complementary (as indicated in the National Mental Health Services project) even though it is inevitable that the manager considers mainly how much he/she is involved in the processes, while the users ca

These characteristics taken together re about results. It is the operators in the middle who have to maintain a unitary vision of the quality profile.

An existing formula allows quality to be measured; the formula proposed by Donabedian. Quality is expressed by the ratio between the improvement achieved in health conditions and the best attainable improvement in the same conditions (given our actual state of knowledge and the best available technologies and considering the condition of the patient).

When this ratio is one to one, what you have is excellence. This corresponds to the maximum health improvement attainable in the given context.

To attach values to the numerator and the denominator of Donabedian’s formula, one must, obviously, have a data flow available; an information system.

We’ve used Wing’s distinctions of top down systems, bottom up systems and integrated information flow systems.

While top down flows are useful for the programmer, who can compile Donabedian’s formula in relation to the entire population he is concerned with, and bottom up systems are suitable for measuring quality in a single given case, the third – integrated – position proposed by Wing seems to be the most suitable when you want to apply the formula of Donabedian at a single system level of treatment: Hospital x, District y, Department z.

As you can see the integrated flow system is associated with accreditation (or in a wider sense, with evaluation of services), much as the other systems are suitable for epidemiological analysis (on which Health programming is based) and for clinical analysis (which lays the foundations for awareness of therapeutic practice).

The integrated approach has brought professionals to acknowledge the usefulness of the tools of voluntary evaluation, with equal responsibility based on reciprocal exchange and geared towards improvement, in the awareness of the concrete dimension which must incorporate the concept of excellence.

With respect to other evaluation activities, accreditation is characterised by the fact that it is periodic in nature (time is a significant variable) as well as systematic (the dimension of service as therapy is conserved), and that it uses standards ("minimum" or "excellence") which refer to legal regulations, but not only; in other words, it is an open and dynamic instrument.

In a diachronic vision we can describe the development of periodical evaluation programmes, including accreditation, as the growth of two fundamental and indispensable components: The objectivity of criteria, indicators, and standards, and the consensus with which the evaluation is participated in by the object of the evaluation, that is, the operators themselves.

As we all know, the use of the term accreditation was used for the first time in a professional context in 1917, when Dr. Codman guided his colleagues in evaluating their own working performance in order to identify a consensus regarding the best procedures, to be recommended and extended, with the aim of improving the capacity of the professionals and the efficacy of their interventions. This consensus, indispensable in order to increase the number of departments and hospitals which practised the best available interventions, was based mainly on the objectivity of the gathered evidence. Consensus in participating in the programme aided in gathering the objective elements which in turn facilitated the extension of this very consensus.

Subsequently in the United States, but also in Canada and Australia, the development of accreditation programmes was motivated more by economic and business considerations than by ethical or professional ones.

The pairing of consensus and objectivity was extended to Health Organisation fields, to professional enterprises, and to insurance companies. No longer groups of professionals but entire systems of health care production had to be in agreement, and express consensus regarding the objectivity of the criteria which would exclude them from or include them in the systems by which performance is recompensed.

In the last few years all the accreditation systems have attempted to go back to their roots.

The new awareness of the fundamental role played by professionals, as well as the entry into the field of user organisations, has returned "the voluntary nature" of self-evaluation to centre stage; in the sense of involving internal groups in the organisation, and of focusing on the client/citizen in order to activate the processes of continuous quality improvement.

We are far away from the so-called wormy apple model, in which accreditation was the means for removing the worm-eaten apple from the heap (with an emphasis on external function, inspectional, "una tantum").

The awareness of the indissolubility of the consent/objectivity team can be traced to the systems of obligatory institutional accreditation as well.

Let's take the Dutch model for example. An obligatory government-regulated model based on procedures of responsible self-certification and self-appraisal.

In much the same way, in the Italian institutional accreditation system, which at first was essentially proposed as an instrument for allowing access to the health market, the demand for improving the link between objectivity and consensus is currently gaining ground.

Both the ruling (229/99) regarding re-organisation of the national health system as well as the project aimed at national "mental health" have expressed this concept.

In our system, the main role is assigned to the Regions, which are responsible for financing (as of 2001) the programming and management of the Health Service. And therefore of accreditation activity management as well, both for the purpose of guaranteeing quality as well as with the aim of selecting providers of health services.

Over the last ten years, the experience of professional accreditation of the quality of mental health departments has been growing in Italy.

Following a typically bottom-up trend, this experience has progressively extended to most of the Italian regions and has enjoyed the support and financing of the High Institute of Health, a Scientific Institute connected with the Health Ministry.

The various projects that have taken place have implemented the objectivity of the instruments of evaluation and the consensus of those who have adopted them.

The following three goals have been sought:

 

1. Application and development of a peer methodology, through consensus procedures aimed to define criteria, indicators and standards to be used in the evaluation process
2. Professionals training through the self evaluation process and the accreditation visit.
3. Implementation of periodic pear accreditation visits, and promotion of a culture of quality improvement and good practices

This research project has produced the following results:

 1 – An accreditation manual, with criteria, indicators and standards.

The approved Accreditation Manual is divided into five chapters, which refer to the organisation and to the various structures of the DSM. The introduction to the chapters includes instructions for the correct use of the manual. There are 76 standards concerning the General organisation of the Mental Health Department, 128 concerning the out-Services for adults, 100 concerning the Hospital psychiatric Services, 86 concerning Day Hospital centres, 80 concerning residential structures. The requisites concerning MCQ activity have been re-distributed in the lists relative to the various Department subdivisions.

Each questionnaire is, in turn, developed in ten standards of targeted areas, with a standardised description of the basic information related to the structure: general policies, management areas, organisation, training and management of the staff, integration, facilities and equipment, safety of patients and staff, delivery of care, clinical documentation, individualized care, relationship with the families and relationship with the social workers.

A computerised version of the manual has been produced in order to facilitate self-appraisal and training procedures.

As you can see, the accreditation process is developed through interaction between an internal component (training of support operators, self evaluation, improvement procedures) aimed at self appraisal, within the structure, which wants to be accredited and which reflects on the quality of its own performance, and an outside component (training of external visitors, visit, final report) external to the structure, the authority and the visitors, which guarantees the objectivity of the evaluation.

This cyclical process foresees at the very least a periodic repetition of the activity of self-evaluation and of the visit, even though in our experience, critical areas are constantly identified, areas which require improvement and which must be followed by a new visit to confirm accreditation.

The application of the professional accreditation procedure has shown a high training potential both towards the operators who are directly involved in the research as well as, in general, towards all those who are involved with connected services.

Particularly, the phase of self-evaluation has permitted the various teams involved in the services to focus attention on the practical significance of the concept of the criteria of good quality and to develop within the service environment the aptitude for seeking improved performance by comparing one’s own organisation and one’s own practices with those of other services, as well as aiming towards the same criteria of good quality.

The accreditation procedure can be used for two different aims.

To self-evaluate one’s own DSM.

To obtain professional accreditation.

Self-evaluation, at least in Italy, consists of an activity that is far from being secondary. The huge variability which still exists must, first of all, be brought to the attention of the operators themselves.

An exemplary case can be illustrated in one of the big mental health departments in northern Italy. If we analyse the ratio of self-evaluation in the three areas we’ve mentioned, that is: organisation, assistance, and integration, we can confirm what has been stated.

The organisation curve, in fact, which is the less accentuated, is off by nearly two points (forty per cent possibility)

The assistance curve shows a variability of sixty per cent.

Finally, the integration curve, the most accentuated, shows an oscillation of a good four points (eighty percent).

The emphasis put on the activity of self-evaluation must not, however, let us forget that the aim of accreditation is that of obtaining the status of an accredited service from an external authority. This is also the aim of professional accreditation, even though in this case the external authority is made up of professionals at the same level.

Let's see however, what are the strong points which the accreditation visit has identified in the general organisation of one of the Italian departments which is most well-known, also at international level.

 A clear and the shared mission aimed toward social psychiatry

They were found lacking in several specific treatment techniques, in their computer system, and in their relations with other external agencies and institutions.

The impact of the accreditation programme

At a distance of some time after carrying out the visit, a survey was done (G. Agnetti, A. Righi, E. Re) regarding the impact that participation in the accreditation programme (at the same level of competence) has had on the participating services.

Apart from a generally positive attitude regarding the experience, a significant number of services discussed the appraisal procedure used among themselves (40%), along with the significance of the evaluation criteria (30%) to establish the good quality of the service.

Most of the services (60%) have also thought of introducing some changes with respect to the critical areas identified as a consequence of the visit and a fair number (20%) have also put some of these changes into action, with the intention of improvement.

In conclusion we can confirm:

1. Its voluntary nature
2. Professionals involvement and participation
3. Emphasis on quality improvement
4. Self-evaluation
5. Attention to users, cares and their satisfaction

 Some of these features are specific to professional accreditation, for example, the willingness of the structure to undergo the procedure of self evaluation, while others are common to accreditation with certificatory functions, but some aspects are more strongly developed and valorised.

In particular, professional accreditation takes into consideration:-

1 The interest of the professionals to develop and improve their professional qualities and the quality of the services they provide.

2 The necessity to involve the operators in the evaluation process, the object but also the subject of evaluation.

3 The tendency of the entire structure and the work groups to try to improve the quality of the services provided.

4 The central role played by providing an answer to the needs and demands of their own clients.

We believe that the accreditation programme success is related to

The procedure of professional accreditation, the aim of which is to promote

© 2002, Associazione Italiana Qualità e Accreditamento in Salute Mentale 
Scrivi alla nostra Associazione          Contatta il Webmaster          Aggiornamento 04/12/2004