While Indonesia's family planning program is one of the most successful among developing countries, contraceptive discontinuation rates are high: One-fourth of couples who adopt a method stop using it within a year.¹ In response, the National Family Planning Coordinating Board (BKKBN) has launched a national initiative to improve the quality of family planning and other reproductive health care. One major component of this initiative is extensive refresher training for family planning service providers, to improve their interpersonal communication and counseling skills.

In a marked departure from past practice in Indonesia, the training curriculum promotes a client-centered approach to care, drawing on the client-centered therapeutic relationship described in counseling and psychotherapy, as well as new models of medical care that have emerged as alternatives to the conventional physician-centered approach.² Client-centered care calls on providers to respect clients' point of view, encourage clients to express their needs and preferences, tailor information to clients' situation and help clients make informed decisions, rather than tell them what to do. Thus, the training curriculum demands substantial changes in providers' attitudes and behavior.

Health care workers face many obstacles in applying newly learned skills on the job, and improvements in job performance after training are not necessarily sustained over time.³ To realize the potential of training, providers need reinforcement and a supportive environment when they return to their jobs after training.⁴ Because funds available for conventional forms of reinforcement (such as refresher training or close supervision) are limited, BKKBN sought to design and test a low-cost intervention to help providers consolidate and apply newly acquired skills.

Prior research suggested that self-assessment and peer review would be effective, low-cost strategies.⁵ Self-assessment asks individual providers to judge their own job performance against a set of outside standards. In peer review, groups of providers give one another feedback and share experiences and ideas. Providers' efforts to learn and improve by themselves is a critical element of both activities. After identifying their weaknesses, providers set personal goals for behavior change, try out new behaviors and assess the outcomes of their efforts; this is a continuous process, in which providers repeat the cycle until they are completely satisfied with their performance. The success of both self-assessment and peer review depends on providers' motivation, ability and diligence in completing the tasks required, since little or no supervision is involved.

While research on whether self-assessment and peer review improve health workers' performance is limited and has proven inconclusive, results suggest that these approaches may improve providers' communication skills. Studies in developed countries that have examined the reliability of self-assessment as a substitute for or complement to supervisors' evaluations generally suggest that it is useful in training health care providers. However, they have questioned the reliability of self-assessment because its quality varies widely.⁶

An evaluation of an Indonesian program that sent trained midwives to observe and assess their colleagues and to give them direct feedback found that this intervention enhanced midwives' interpersonal and clinical skills.⁷ Some key elements of the peer-review approach also have been tested as part of the COPE (client-oriented, provider-efficient) intervention, in which clinic staff work as a group to assess and solve problems. Results suggest that providers are intrinsically motivated to offer better services, consider themselves responsible for self-improvement and can continue to learn through group self-assessment and support.⁸

The purpose of the study described in this article was to assess whether training, self-assessment or peer review encouraged client-centered counseling in the Indonesian program--that is, whether, after the intervention, providers worked more with clients as partners, clients participated more actively and clients felt more satisfied with the counseling process.

Methods

Study Sample

The study was conducted in two provinces of Indonesia: East Java and Lampung. Within each province, investigators selected three districts (or kabupaten) that were matched in terms of size, proximity to the provincial capital and clients' socioeconomic characteristics. From the list of all public service clinics (or puskesmas) in the six districts, 191 clinics were randomly chosen for the study. One or two providers at each clinic were asked to participate, for a total of 233. Each provider asked six family planning clients to participate: two before training, two immediately after and two at a later date. Over the course of the study, 21 clinics, 32 providers and 55 clients were dropped because providers did not attend the training workshop or information was incomplete. Data on 170 clinics, 201 service providers and 1,210 clients are analyzed here.

The clinics, 77% of which were located in rural areas, varied widely in size, staffing and client flow. In June 1998, for example, the top 20% of clinics with respect to family planning caseload registered 90 or more family planning visits apiece, while the bottom 20% each recorded 18 or fewer. A few of the busiest clinics recorded more than 200 family planning visits that month.

All of the providers were women; most were married with children. Ninety-three percent were midwives,* while the remainder were nurses. The providers were responsible for a wide range of health services in addition to family planning; 45% spent less than half their time on family planning. Four-fifths had more than five years of experience offering family planning services, and half saw more than 10 family planning clients per week.

Clients were chosen randomly on the day of data collection; as a result, they mirrored the general family planning population. All but two were women, 99% were married, 70% had at least two living children and 55% were aged 25-34. More than half (57%) had at least some secondary education; this proportion is slightly higher than national levels among women 25-44 years old (21-44%), reflecting that family planning users in Indonesia are better educated than nonusers.⁹ Three-fourths (76%) of participants were continuing clients, about half of whom were returning for routine checkups and resupply; the remainder came to the clinics because they had a problem with their method. The most commonly used contraceptives were the injectable (used by 28% of participants), IUD (27%), implant (17%) and pill (16%).

Interventions

Providers attended five-day training workshops, held in December 1997 and January 1998, that emphasized client-centered counseling and covered skills such as establishing rapport with clients, encouraging dialogue and helping clients make decisions. To assess the impact of the training and reinforcement, we randomly assigned each district to one of three groups; every clinic and every provider within a district belonged to the same group. Providers in one group, who served as controls, attended the training workshop but did not receive any subsequent reinforcement. Those in a second group participated in self-assessment after the training, while providers in the third group both participated in self-assessment and attended peer-review meetings. Workshops held in the self-assessment and peer-review districts included a half-day of instruction on these interventions. The reinforcement interventions began one month after the workshops ended and lasted 16 weeks, from February to June 1998.

For the self-assessment intervention, providers spent about 15-20 minutes each week evaluating a single counseling session, using a two-page form. Each week's form covered a single skill area from among these eight: listening, being responsive to clients, expressing positive emotions, eliciting information, verifying clients' decisions, communicating during clinical procedures, giving information and encouraging clients' participation. First, providers reported on their own and the client's behavior, using a checklist with such items as "Gave full attention to client's fears and anxieties" and "Client asked for clarification." Then, two open-ended questions prompted the providers to reflect more deeply about the interaction, asking, for example, what other questions or concerns the client might have had and what else the provider could have said. The providers finished the form by listing two specific behaviors, related to that week's skill area, that they wanted to change during the following week. Space also was provided to record the outcomes of their efforts to improve.

Providers in the peer-review group attended a weekly 30-60-minute meeting with three or four other participants. No moderator or facilitator was assigned, but providers were given a brief discussion guide that echoed the topic of the weekly self-assessment form. Participants were expected to discuss issues that emerged from the self-assessment exercises but not to identify a specific case or to share their self-assessment forms.

Data and Variables

Research assistants collected data from four sources: interviews with providers about their training and service experience; audiotapes of family planning counseling sessions; exit interviews with clients about their perceptions of the session; and clinic records of the number of clients served and the availability of contraceptive methods. Prior to data collection, providers and clients signed a confidentiality and voluntary participation consent form; their names were not known to the analysts. Data were collected at three points: before the training workshop (baseline), during the one-month gap between the end of the training and the start of the reinforcement interventions (posttraining) and 1-2 months following the 16 weeks of reinforcement (follow-up).

Two key variables, for which information was drawn from the audiotaped sessions, examined provider-client interactions. One of these, which we call providers' facilitative communication, refers to communication that promotes an interactive relationship between client and provider by fostering dialogue, rapport and client participation. The other, termed clients' active communication, refers to communication that allows the client to participate in the consultation and help shape its direction or that indicates that she feels comfortable and is speaking openly with the provider. Increasing both types of communication was the primary goal of the interventions.

To measure the communication variables, we used an adaptation of the Roter Interaction Analysis System (RIAS), which assigns a code to each utterance (i.e., complete thought, usually a phrase or sentence) made by a client or provider so that these may be grouped into categories for analysis. RIAS has been employed extensively in both developed and developing countries.¹⁰ The adaptation was based on an earlier client-provider interaction study we conducted in Indonesia, in which we reviewed 38 videotaped counseling sessions.¹¹ Coders for the current study were Indonesians who understood the local language and received special training in RIAS; they assigned one of 47 mutually exclusive codes to each utterance, using a computerized data entry screen while listening to the audiotapes. The codes were combined into the categories listed in Table 1.

The analysis of changes in providers' facilitative and clients' active communication presented a methodological challenge, because the average length of the counseling sessions varied between clients and fluctuated significantly from one round to the next. Data on the proportion of utterances contributed by the provider and the client allow sessions of different lengths to be compared. However, an increase in one proportion means that the other must decrease, since the total is always 100%. Yet if sessions last longer, the number of contributions of both may increase. Frequency data provide a more accurate assessment of these kinds of changes than do proportions. Frequency data also capture changes in important types of communication that occur too seldom to achieve significance in percentage tables. To present a full picture of the changes in communication patterns, we report both percentages and frequencies.

Three other variables, derived from interviews with clients, assessed their perspectives on the counseling experience. These were self-efficacy, self-expression and satisfaction. Self-efficacy is a precondition to behavior change. In general, itdenotes the extent to which a person believes that she or he is able to act;¹² here, it refers to clients' belief that they can say what they want to the provider. Self-expression is clients' assessment of how much they spoke and what they said during the consultation. Two aspects of clients' satisfaction with the quality of care were assessed: the personal attention they were shown and whether they received the help that they came for.

To measure these variables, research assistants read a series of statements to clients, who had a choice of four responses: strongly agree, agree, disagree or strongly disagree. For the purposes of analysis, a neutral option was added to the scale; thus, these responses were tabulated on a five-point scale, with five indicating strong agreement and one indicating strong disagreement. Multiple interview items were grouped to create the indicators shown in Table 2. Statistical testing found that reliability for these indicators was high. Cronbach reliability coefficients were .79 for self-efficacy, .84 for self-expression, .85 for satisfaction with attention shown and .69 for satisfaction with needs met.

Results

Baseline Communication Patterns

Baseline counseling sessions show how providers and clients in Indonesia usually interact. On average, sessions lasted six minutes, but they varied widely in length (from one minute to 72 minutes). Likewise, the total number of utterances spoken by both provider and client ranged widely (from 11 to 512), averaging 95. Total utterances in sessions with new clients exceeded those in sessions with continuing clients (119 vs. 87; p<.0001). Providers dominated the interchange: They spoke 64% of all utterances, and their utterances generally contained more words than clients'.

The bulk of provider communication, as expected, was a straightforward delivery of information on family planning and medical matters. Such information and questions about health and family planning accounted for about two-thirds of providers' utterances (Table 3). One-fourth of providers' communication was facilitative.

More than three-quarters of clients' utterances came in direct response to the provider. Clients predominantly either answered providers' questions about medical, family planning and routine matters (56%) or signaled agreement with or understanding of providers' remarks (24%). Only about 10% of clients' utterances were active; half of these were questions. This translates into clients' asking an average of 1.6 questions per session.

Clients' baseline ratings of their counseling experiences were skewed to the upper end of the five-point scale: 3.9 for self-expression, 4.0 for a self-efficacy and 4.1 for satisfaction. These results are not surprising, given the strong social pressures in Indonesia against expressing disagreement. Thus, in this setting, even relatively small shifts in client assessments may be meaningful.

Posttraining Communication Patterns

The most dramatic impact of the training workshop observed in the month afterward was that the average length of counseling sessions almost doubled, to 11 minutes. Similarly, the total number of utterances by both provider and client jumped 86%, to 177. (Sessions grew shorter over the months that followed, but were still 40% longer at follow-up than they had been at baseline.)

Providers used most of the extra time to give clients additional information and counseling on medical and family planning issues. The number of utterances in this category rose from 27 to 58, and the proportion climbed from 39% to 48% (Table 3); both differences were highly statistically significant (p<.0001). Providers also asked more medical and family planning questions after training than before (18 vs. 16; p<.005), even though these questions accounted for a smaller proportion of their utterances. Since Indonesian providers generally give family planning clients limited information,¹³ the dramatic increase in information delivered represents an improvement in one aspect of counseling. However, the quality of information provided is also important, and the RIAS coding scheme does not permit us to assess the clarity, accuracy and relevance of information.

Training also had a positive impact on providers' facilitative communication. Between baseline and the posttraining period, the average number of utterances in this category doubled from 15 to 30; as a proportion of providers' contributions, they increased from 25% to 28% (p<.0001 for both increases). In percentage terms, providers increasingly acknowledged what clients said and offered information on lifestyle and psychosocial issues.

After training, clients' active communication remained steady in percentage terms (Table 3) but more than doubled in frequency, from 3.3 to 7.0 utterances per session (p<.0001). Most of the increase was in acknowledging what the provider had said, as a consequence of the sharp rise in providers' information-giving. However, the average number of questions per session also doubled (from 1.6 to 3.3; p<.0001), and that number held steady during the four-month follow-up period, even as consultations grew shorter.

All client ratings showed small increases after training. The self-expression rating rose to 4.1, self-efficacy to 4.1 and satisfaction to 4.2 (p<.0001 for each increase).

Impact of Self-Assessment

As Figure 1 shows, self-assessment alone had a significant impact on providers' facilitative communication. The proportion of providers' utterances that fell into this category increased from 28% to 33% in the group that conducted self-assessment only, while remaining essentially unchanged in the control group. A closer examination of the data (Table 4) shows two patterns at work. Some types of communication (partnership-building and asking questions) increased in the self-assessment group but stayed at the same level in the control group. Others (positive emotion and acknowledging clients' remarks) declined significantly in the control group but not in the self-assessment group. Only the proportion of utterances devoted to personal or social conversation increased in both groups.

The level of clients' active communication increased significantly in the self- assessment group but not in the control group (Table 5). The change was due to the substantial increase in clients' social conversation in the self-assessment group and the decrease in their expressions of concern in the control group. Self-assessment also had a positive impact on other types of client communication (the provision of lifestyle, psychosocial, medical and family planning information), which is important for client-oriented care.

Client ratings of self-expression increased significantly (p<.0001) in the self-assessment group (from 4.0 to 4.2) but held steady in the control group. Similarly, in the self-assessment group, but not in the control group, significant increases were observed in client satisfaction with provider attentiveness (from 4.3 to 4.4) and with needs met (4.1 to 4.3). Self-efficacy did not change in either group.

Additional Impact of Peer Review

Peer review made an impact beyond that of self-assessment on facilitative communication (Figure 1). The total level of facilitative communication rose from 28% to 37% in the peer-review group (Table 4). Peer review had a greater effect than self-assessment alone on social conversation and lifestyle or psychosocial information; it had less of an influence on partnership-building and a similar impact on lifestyle or psychosocial questions. The number of facilitative utterances declined among providers in the self-assessment group (from 29 to 25, p<.05) but remained stable (30-32) in the peer-review group.

Clients' active communication also increased more sharply in the peer-review group (from 12% to 17%) than in the self-assessment group (from 12% to 15%), mainly because of differences in levels of social conversation (Table 5). During the final round of data collection, clients in the peer-review group made an average of 8.5 active utterances, including 4.9 questions; by comparison, those in the self-assessment group made 5.8 active utterances, including 3.3 questions.

Client ratings show a different pattern. Two of the three variables changed in the self-assessment group: Satisfaction increased from 4.2 to 4.4, and self-expression rose from 4.0 to 4.2 (p<.0001 for both comparisons). However, none of the client ratings changed significantly in the peer-review group.

Differential Impacts on Subgroups

We conducted a series of multiple regression analyses and Wald tests to assess the impact of the reinforcement interventions in various subgroups. The subgroup analysis examined six variables: new versus continuing clients, clients' education and age, providers' experience, province and clinic size.†

Self-assessment and peer review had a stronger impact on providers' facilitative communication in sessions with continuing clients than in those with new clients, perhaps because sessions with continuing clients tend to be shorter and focused more on asking clients questions than on giving information. In contrast, reinforcement had a stronger impact on clients' active communication and satisfaction in sessions with new clients; this result may reflect that new clients have fewer preconceptions about appropriate behavior during family planning consultations.

Clients' characteristics also proved significant in determining the effect of the interventions. Self-assessment and peer review had more impact on both providers' facilitative and clients' active communication among clients with at least a secondary education than among those with less schooling. Better-educated clients engaged in active communication more often than their less-educated peers at baseline, and this gap widened over the study period. Education may give clients the confidence to take advantage of any opportunities to speak that providers offer, while providers may be more open and responsive to better-educated clients, with whom they may identify. Reinforcement interventions also had a greater impact on providers' facilitative communication in sessions with clients aged 35 or older.

The interventions were associated with a greater improvement in both providers' facilitative and clients' active communication among providers with more than 10 years of experience offering family planning services than among their less-experienced counterparts. This finding implies that experienced providers, far from resisting change, were better able to understand, carry out and apply lessons learned from reinforcement activities than were those with less experience.

Self-assessment and peer review had a stronger effect on levels of providers' facilitative communication in clinics with 50 or fewer client visits per week than in those with smaller caseloads. In contrast, client satisfaction was more affected in larger clinics. Client satisfaction differed substantially in the two provinces studied at baseline, and it was more affected by reinforcement activities in Lampung than in East Java.

Costs and Cost-Effectiveness

Direct costs of training and reinforcement (including materials, trainers' honoraria, providers' per diems and transportation) totaled approximately U.S.$70 per provider for training, $2 for self-assessment and $9 for peer review.‡: The need to transport providers between the scattered rural clinics that participated in the study boosted peer-review costs. Including opportunity costs (the cost of employee time diverted from regular duties to program activities) and the cost of a single supervisory visit, the total cost of training and reinforcement was $90 per provider for the control group, $106 for the self-assessment group and $122 for the peer-review group. (To put these figures in context, the providers' average monthly salary at the time was $76.) Development costs for the training curriculum, self-assessment forms and peer-review guides were not included because they were onetime expenditures.

To calculate cost-effectiveness, we used the proportion of providers' utterances that were classified as facilitative communication as the outcome measure. The cost of raising facilitative communication one percentage point in the control group (i.e., through training alone) was $500. In comparison, the cost was $12 for the self-assessment group and $10 for the peer-review group. Thus, adding peer review to self-assessment proved cost-effective despite its higher cost.

Discussion

Benefits for Providers

While the training workshop raised the quality of providers' communication in the short term, without reinforcement, their newly learned skills tended to erode over time. This confirms the observations of health care trainers in both developing and developed nations that the transfer of skills from formal training to everyday practice is extremely limited without continuing support and reinforcement.¹⁴

Indeed, providers in the control group found little support for their new counseling skills when they returned to their clinics after training. Routine supervision visits were irregular, and supervisors--who were not included in the training--rarely addressed client-provider communication. Untrained colleagues expressed skepticism when providers tried out new counseling behaviors--for example, by asking "Why are you acting so funny now?" Continuing clients also were bewildered and uncomfortable when providers acted friendlier or offered lengthier explanations than they had in the past. Providers found it easier to try out new communication behaviors with clients whom they had not seen before. Faced with these obstacles to behavior change, providers in the control group may have stopped trying to apply their new skills before they had a chance to consolidate them.

Self-assessment supplied the reinforcement and continuing learning that providers needed to change their behavior on the job. Not only did it help maintain providers' posttraining performance, it also served as a mechanism of continuous quality improvement. Some key communication skills, such as partnership-building, continued to improve throughout the study period. More experienced providers were better able to translate self-assessment and peer review into behavioral change.

Prior studies suggest that by clarifying standards, self-assessment forms reduce providers' anxiety and confusion, increase motivation, focus attention and facilitate learning.¹⁵ The list of weaknesses that providers commonly identified during their self-assessment exercises matched outside assessments of their deficiencies. This suggests that self-assessment helped providers discriminate between good and poor skills and allowed them to evaluate how well they were performing. The process of behavior change built into the weekly self-assessment exercises also served as a constant reminder to providers to apply new skills. During their final interviews, providers agreed that the self-assessment exercises had increased their confidence and their ability to apply the communication skills they learned during training.

Adding peer review to self-assessment heightened the impact on some aspects of provider communication. The effectiveness of peer groups for learning depends upon the quality of the group process and the richness of the discussion. However, the contents of the peer-review discussions were not systematically studied. According to anecdotal reports, the providers got to know each other well over the 16-week period, enthusiastically shared their experiences and gained self-confidence from their participation. If this is so, then one important benefit of the peer-review meetings may have been the psychological and emotional support that participants derived from one another.

Impact on Clients

Most of the gains in clients' active communication were in personal and social conversation, suggesting that the intervention was more successful in promoting rapport between providers and clients than in boosting client input. This inference is borne out by the relatively small change in client self-expression ratings. However, clients were asking twice as many questions by the end of the study as they had been at baseline--a meaningful change, given the passive behavior of most Indonesian clients.

Although these data suggest that training and reinforcement are related to improvement in providers' and clients' behavior, they do not support the theory that raising providers' facilitative communication increases clients' confidence in their ability to express themselves, since self-efficacy grew only slightly over the study period. Rather, providers' and clients' behavior seem to be more directly linked, with providers offering clients more opportunities to speak and prompting them to talk. Better-educated clients were able to take greater advantage of these opportunities than their less-educated peers.

Theoretically, increases in providers' facilitative and clients' active communication should make clients feel more satisfied with their consultations. Although there was a significant rise in client satisfaction, it was quite small. Also, contrary to expectations, satisfaction increased most in the self-assessment group, although the peer-review group displayed the greatest gains in provider and client behavior. Given the many methodological difficulties in defining and assessing client satisfaction,¹⁶ it is not clear whether these results reflect measurement problems or real differences in client satisfaction. Substantial differences in client satisfaction between Lampung and East Java also complicate interpretation of the results.

Ultimately, the success of this intervention should be measured by its impact on clients' correct and continuing use of a contraceptive method, since high discontinuation rates were BKKBN's motivation for launching the project. However, no data were collected on clients' contraceptive practices after they left the clinic. In theory, increased client participation and more active communication should help clients make more realistic, better-informed contraceptive choices and should increase their confidence in and commitment to their decision--all of which would raise their likelihood of continuing to use the method.¹⁷ Studies in medical settings in developed countries have found that health outcomes improve when clients take a more active role in consultations,¹⁸ but only a single study in Egypt has investigated the link between client-centered counseling and contraceptive continuation.¹⁹ Further research is needed to determine whether and how facilitative counseling and increased client participation affect contraceptive continuation in developing countries.

Ingredients for Success

Past studies have reported problems in implementing self-assessment because of participants' unfamiliarity with their role in the intervention, resistance to self-evaluation or distrust of the assessment.²⁰ By contrast, although participants in this study conducted the self-assessment exercises in isolation from one other and without support from supervisors or coworkers, nearly all of them completed the 16-week series. Similarly, absenteeism for the peer-review meetings was extremely low, even though two-thirds of providers had to travel to other clinics for the meetings. Why were these providers so diligent in carrying out their self-assessments and peer review?

The brief, uncomplicated design of the self-assessment exercises and form made them easy to complete, even though the form took longer than planned to fill out. Providers initially had some difficulties with the cycle of defining concrete behavior goals, trying out new behaviors and assessing the outcomes, but a single problem-solving meeting proved sufficient to overcome them.

Furthermore, the contents of the curriculum, self-assessment forms and peer-review guides were based on formative research findings and were completely consistent with one another. The training workshop clearly defined the behaviors to be evaluated and their performance criteria, so providers understood what each behavior meant and how to evaluate its quality. Equally important, the workshop taught providers how to complete the self-assessment forms. Without this kind of training, self-assessment might not work.

Providers' high level of motivation also was essential. Training instilled providers with the desire to offer better service, as evidenced by the increased length of sessions immediately afterward. The short (one-month) time lag between the training workshop and the start of the reinforcement interventions allowed self-assessment and peer review to capitalize on this strong motivation. Further motivation may have come from Indonesian providers' identification with the service delivery system and their tendency to comply with their institutional obligations.

Study Strengths and Limitations

By collecting data at three points in time (including a baseline) and creating a control group, we were able to study several issues at once: the impact of training on job performance, retention of skills over time, the impact of self-assessment alone and the impact of self-assessment combined with peer review. The large sample size provided the opportunity to compare impacts in several ways and allowed the quantitative findings to be interpreted with a high degree of confidence.

The use of multiple data sources and indicators to measure the impact of the interventions enriched the interpretation of the findings. Audiotaping the sessions had several advantages over conventional observation and interview methods: It provided in-depth qualitative and quantitative data, allowed investigators to listen and recode as needed, and covered client and provider behavior equally. Interviews probing clients' perspectives on the counseling sessions enhanced our understanding of client-provider interactions and the impact of interventions. Conflicting findings from different data sources provided important insights on measurement issues.

A critical limitation of this study is that we cannot relate quality improvements with outcome behaviors, such as continuing use of a contraceptive method. We have data showing that the number of family planning visits to study clinics dropped during the four-month follow-up period; however, this statistic was so greatly affected by economic and social disruptions in Indonesia that it may not reflect changes in quality. ²¹ Riots, contraceptive shortages and new charges for contraceptives during this period all discouraged clients from attending public clinics.

Refining the Interventions

Both training and reinforcement overlooked a critical issue in raising the quality of care: ensuring that providers give equally good care to all clients, regardless of their education, socioeconomic status or ethnicity.²² Although the training curriculum and reinforcement materials encouraged respect for clients, they did not ask providers to reflect on how cultural biases affected their interactions with poor or uneducated clients. Revised materials should directly address these issues and encourage providers to treat all clients with equal respect and empathy.

Other changes in the strategies tested here could help give providers the supportive work environment that experience in both developing and developed countries has found to be so necessary after training.²³ Training all of the providers in a clinic--rather than just one--and orienting the rest of the staff to the curriculum would help ensure that coworkers understood and encouraged providers' efforts to change their approach to counseling. It would also substantially lower the cost of peer review by eliminating the need for providers to travel between clinics. If training were extended to managers and supervisors, then they too could help oversee and advise providers on interpersonal communication and counseling. (AVSC International has long advocated this kind of whole-site training.²⁴)

Clients also contribute to a supportive environment, as demonstrated by providers' difficulty in changing their style of counseling with continuing clients. This finding suggests the importance of a strategy that raises clients' expectations and improves providers' communication and counseling skills. Family planning programs in Nepal and Egypt employing radio and television to model client-centered interactions have changed cultural norms about what is appropriate behavior during family planning consultations.²⁵

While providers had little difficulty completing the self-assessment forms, it took them longer than planners had estimated, and some did not have enough time to fill out the form between clients. But some providers had trouble remembering the details of the consultation if they waited until later in the day to complete the form. One solution is for providers to fill out the first section of the form (which asks specific questions about their performance) immediately after seeing the client, but to save the rest for later.

Peer review proved to have a weaker design than self-assessment. Giving and receiving feedback is the critical learning component in group work, but providers lacked the expertise to offer the kind of accurate and concrete feedback that can best influence knowledge, attitudes and behavior. Adding a more skilled person to the group, such as a supervisor or trained peer mentor, might have improved the quality of the feedback. Feedback would have been more concrete and powerful if the group listened to an audiotaped segment from one of their own sessions or participated in a role-play, rather than basing their discussion on providers' accounts of their experiences. Audiovisual materials and a more detailed discussion guide might have focused and enriched the peer review. Finally, Indonesian cultural values may have made providers reluctant to give specific feedback, for fear of being impolite. While giving constructive criticism is always risky behavior, it is especially so in Indonesia.

Meeting the Need for Reinforcement

Self-assessment and peer review can make training more effective by helping providers apply new skills in the workplace, giving them an opportunity to continue learning and motivating them to improve their performance long after training is over. The learning cycle built into these reinforcement mechanisms also can support the kind of ongoing quality improvement sought by many primary health care programs in the developing world. Given the high cost of training, spending a small additional amount on self-assessment or peer review makes good sense.