The literature on the evaluation of adolescent sexual health interventions is growing in both developed¹ and developing countries.² However, because few evaluations have shown an impact on behaviors, rigorous assessments of interventions that target adolescents are still needed.

This study assesses the performance of four adolescent sexual health interventions implemented in Sub-Saharan Africa under the five-year Social Marketing Adolescent Sexual Health project, financially supported by the U.S. Agency for International Development's Africa Bureau. It is unique in that it uses a quasi-experimental design, covers programs in multiple countries—Cameroon, Botswana, South Africa and Guinea—and presents findings within the parameters of a well-articulated analytic framework, the Health Belief Model.

The Health Belief Model is one of the most widely used approaches to understanding individual health behaviors; studies in Sub-Saharan Africa indicate that it is good at explaining intentions and behavior there.³ It is used for this study because its components broadly reflect the goals of social marketing efforts (i.e., to raise risk awareness, reduce barriers to safer sex and increase perceived benefits of prevention), and it allows the examination of a range of variables that are common to several widely used conceptual models.

The model is based on a cognitive approach that uses a cost-benefit perspective to understand preventive health behavior. It assumes that individual behavior change depends upon perceived severity of risk, perceived susceptibility to risk, perceived benefits of preventive action, perceived barriers to preventive action and perceived ability to take preventive action (self-efficacy).⁴

THE INTERVENTIONS AND STUDY DESIGN

All four interventions evaluated in this study were nested within national social marketing programs. The national programs sold subsidized, branded condoms to traditional outlets, such as pharmacies and clinics, and to nontraditional outlets, such as supermarkets, kiosks and street vendors. They promoted the use of their brand to sexually active adults through mass media advertising and through information, education and communication tools such as billboard messages about condom use and point-of-sales materials (stickers and posters) at condom outlets.

The most rigorous evaluation design is the true experiment, in which individuals are randomly assigned to intervention and comparison groups. In many situations, however, implementing a true experiment may be impractical.⁵ Since the interventions being evaluated here rely on the use of mass media, for example, it was not possible to randomly assign some individuals living in a given neighborhood to be exposed to the intervention and others not to be. As a result, this evaluation used the nonequivalent comparison group design, in which towns, rather than individuals, were compared.

Cameroon

In Cameroon, the two towns selected for this study were Edea and Bafia. Edea, where the intervention was implemented, is a city about 40 miles south of Douala, on the main road to Yaoundé. At the time the intervention was implemented, it had a population of about 86,000. The comparison location, Bafia, is about 80 miles north of Yaoundé, and had a population of 73,000. In both towns, the populations are ethnically mixed and are roughly evenly divided between Christians, Muslims and others. Both towns have access to AIDS services.⁶

The 13-month intervention included peer education, youth clubs, mass media advertising and the distribution of informational and educational materials. Twenty-eight adolescents (16 males and 12 females) were trained as peer educators to promote behavior change and motivate adolescents to use the contraceptive products available through the social marketing program. They led discussion groups about a wide range of topics—for example, abstinence, pregnancy prevention, HIV and AIDS, other sexually transmitted diseases and how to communicate about sexual issues—and sold subsidized condoms to discussion-group participants.

Members of the youth clubs wore promotional items such as T-shirts and caps carrying messages about contraceptive use. Youth club events (e.g., condom use demonstrations at soccer games) were advertised on community radio, which reached a large segment of the population of Edea. On live talk shows, adolescents discussed sexual health topics such as how to inform a partner about a sexually transmitted disease, the importance of being faithful to one's partner and condom use. In addition, radio spots were used to broadcast preventive health messages targeted at adolescents.

Baseline and postintervention survey data were collected by l'Institut de Recherche et des Etudes de Comportements (IRESCO). The 1996 baseline survey included a random sample of 1,606 respondents (805 in the intervention city and 801 in the comparison location), and the 1997 follow-up survey contained data on 1,633 respondents (811 and 822, respectively). A multistage probability sample was used; 30 clusters were drawn with probability proportional to cluster size in each location. Households were randomly selected within each cluster, and of household members aged 12-22, the one whose birthday was most recent was chosen to participate.⁷

South Africa

In South Africa, the intervention site was Soweto, and the comparison site was Umlazi. Both are urban areas near major urban centers (Johannesburg and Durban, respectively). Soweto has a population estimated at more than one million people, while Umlazi has a population of fewer than 50,000. Umlazi has a higher HIV prevalence level than Soweto (21% vs. 12% of women visiting prenatal clinics). One of the factors that may have contributed to the higher HIV prevalence in Umlazi is its proximity to Durban, which is a major seaport.⁸

The 11-month intervention in Soweto included campaigns promoting safer sex, which were broadcast on the community's new radio station. On live weekly talk shows, project staff held discussions with guest experts (e.g., family planning providers and counselors) on topics such as ways to prevent pregnancy, HIV and other sexually transmitted infections; dating; relationships; sexual violence; contraception; and how to resist peer pressure to have sex.⁹

Seventy adolescent volunteers were trained to participate in media development, condom distribution and peer education. (Twelve of these young people later became paid marketing assistants for the project and promoted or sold condoms to adolescents in their communities for a commission.) In addition, 300 new condom outlets that sold the program's condoms were opened in Soweto.

Adolescents were randomly selected for the surveys through multistage probability sampling; the South African Medical Research Council collected the data. At the first stage, township maps for each city were used to randomly select 450 residential plots with equal probability of selection. Young people aged 16-20 were enumerated in each selected plot, and a random sample of youth was drawn.¹⁰Because data quality checks revealed that most interviewers used for the male follow-up sample were not reliable, the analysis is restricted to females. In addition, because very few 16-year-old females were selected for the follow-up sample, the analysis for the intervention site is restricted to those aged 17-20.¹¹ The baseline survey was conducted in 1996 among 118 young people in Soweto and 103 in the comparison location; the postintervention survey took place in 1997, and included 101 respondents in Soweto and 103 in the comparison town.

Botswana

An eight-month intervention was carried out in Lobatse, a town of 30,000 that is very near Botswana's southern border with South Africa; the comparison town, Francistown, has a population of 50,000 and is near the eastern border with Zimbabwe. The towns are similar in terms of ethnic composition, urbanization and access to government services (including AIDS prevention services). They have similar levels of HIV infection among women attending antenatal clinics—estimated at 40% at the time of the intervention.¹²

The intervention was designed to persuade adolescents that reproductive health services were for them, and not solely for adults. To complement activities aimed at creating demand, the intervention developed youth-friendly outlets. Retailers who agreed to participate attended a workshop on adolescent sexual health counseling and were given a sign that identified their outlet as being adolescent-friendly. Providers at retail outlets offered advice to adolescents and referred them for counseling to Ministry of Health clinics where health workers had been trained to counsel youth. The clinics also displayed signs indicating that they were adolescent-friendly. During the intervention, project staff made monitoring visits to follow up on the performance of adolescent-friendly outlets.

The mass media component included radio messages and information campaigns directed at parents, teachers and young people, and a radio call-in show on which adolescents could discuss issues related to AIDS, family planning and contraceptive use. However, because the same radio station reached both Lobatse and Francistown, adolescents in the comparison area were also exposed to these radio shows and messages. Peer educators who were part of the intervention were given standardized training and refresher courses.¹³ Because of the relatively small population of Lobatse, a substantial proportion of adolescents (70%) were exposed to the intervention.

Survey respondents were 13-18-year-old men and women who were randomly selected through multistage sampling. Data collection was conducted by the Social Impact Assessment and Policy Analysis Corporation (SIAPAC-Africa). With the use of numbered maps, 30 starting households were randomly selected in both the intervention and the comparison locations. Each household had an equal probability of selection. Other households in that cluster were randomly selected through a systematic pattern of walking. For the 1994 baseline survey, 1,002 adolescents were selected. For the follow-up, in 1995, the sample was increased to 2,396.¹⁴

Guinea

In Guinea, the program's objective was to conduct the intervention in the capital city, Conakry (population, 700,000). However, the country has no city of similar size that could serve as a comparison site: The second largest city (Kankan) has a population of less than 100,000. Therefore, the intervention was implemented in selected neighborhoods in each city, while other neighborhoods were used as comparison sites.

In intervention areas, 28 male and 28 female volunteer peer educators were trained on issues related to HIV and AIDS, family planning and communication techniques. During the intervention period (about six months in Kankan and 8-9 months in Conakry), they held discussions about reproductive health with young adults and organized educational theater, film presentations on reproductive health, and dances and athletic events during which presentations on reproductive health issues (including condom use demonstrations) were made. Most of these events were conducted at locations frequented by adolescents, such as schools, sports fields and video clubs. Peer educators distributed flyers on family planning, HIV and AIDS, and promotional items (T-shirts, stickers, badges and caps) carrying messages about contraceptive use. They also distributed free contraceptives to adolescents. Young people developed a logo proclaiming "My Future First," which was used to identify youth-friendly retail outlets where young people could purchase condoms and other contraceptives. Because each city included intervention and comparison areas, the intervention did not use mass media communication specifically targeted at adolescents.

The Association Stat-View collected data, using a multistage sampling process. The sample was drawn from a list of enumeration areas and households obtained from the Planning Department for Demographic Surveys. All enumeration areas in the intervention and comparison locations were included in the sample frame. Two households were randomly selected per enumeration area, and one adolescent was randomly selected in each household. For the 1997 baseline survey, the sample consisted of 2,016 respondents; for the 1998 follow-up, 2,005 respondents.¹⁵

INDICATORS AND ANALYSES

The variables used in this study serve as proxies for various indicators of the Health Belief Model, measuring both perceptions and behaviors related to sexual health and risks. Although the dependent variables differ somewhat across the four countries, many of the variables used are identical and enable a relatively easy comparison across countries.

Multivariate logistic regression was used in this study. In a nonequivalent comparison group design, respondents in the intervention and comparison locations can differ by age and educational level.¹⁶ Therefore, all regression analyses controlled for these factors. In addition, all analyses were stratified by gender.

For each gender, the analysis was done in two steps. First, for each country, the baseline and postintervention data from each location were merged, resulting in two data sets (i.e., one set containing two rounds of data on respondents in the intervention location and the second containing two rounds of data on respondents in the comparison location). Identical logistic regression models were then run to examine whether the intervention and comparison sites experienced significant changes over time in the indicators being tested; results are presented as odds ratios.

Because beliefs and attitudes can change as a result of factors beyond the control of any intervention, to determine the true effect of an intervention, it is important to assess its impact net of secular trends. Therefore, this step of the analysis also examined whether positive trends in the intervention and comparison locations were significantly different from each other.

For the second step, the two data sets for each country were merged into one, containing both rounds of data from both locations. The statistical model used with these data sets included all variables included in the first step of the analysis plus an interaction term that indicated whether trends in the intervention and comparison locations were significantly different from each other. Results indicate

whether the intervention had a net positive effect on an indicator, a net negative effect or no net effect.

Several combinations of results can be interpreted as demonstrating a net positive effect of the intervention: (a) a significant improvement in a given indicator in the intervention location but not in the comparison location, and an interaction term showing that these trends were significantly different from each other; (b) a significant improvement in both locations, and an interaction term showing that the change was significantly greater in the intervention location than in the comparison location; (c) a negative trend in the comparison location, no trend in the intervention location and an interaction term showing that the trends in the two locations were significantly different from each other; and (d) no trend that reached statistical significance in either location, but an interaction term showing that the trends were significantly different from each other.

A net negative effect of the intervention can be inferred if there was an improvement in a particular indicator in the comparison location but not in the intervention location, and the interaction term showed that these trends were significantly different from each other.

Finally, the model suggests a zero net effect of the intervention if the interaction term showed no significant difference in trends between the intervention and comparison locations. Thus, the intervention had no net effect if trends in both the intervention and the comparison locations were significantly positive or if trends in both locations were significantly negative (as long as the trends were not significantly different from each other). For any trend to be considered positive or negative, it had to be statistically significant at p<0.05.

RESULTS

Reach of the Interventions

Data from the postintervention surveys indicate that the interventions reached widely varying proportions of their target audiences. In Cameroon, 91% of respondents to the postintervention survey reported exposure to the intervention. This level of exposure probably reflects the relatively small population of Edea and the multiple channels of communication used for the intervention.¹⁷ Similarly, in Botswana, 71% of young men and 68% of young women who responded to the postintervention survey reported exposure to the intervention; exposure was high because the program had a strong peer education component that was able to reach the intervention town's small population.¹⁸

By contrast, only 25% of respondents to the postintervention survey in South Africa reported exposure to the intervention,¹⁹ probably because of a number of limitations that project implementation in Soweto faced: The peer education volunteers were not closely monitored,²⁰ and the social marketing organization periodically ran out of condoms during the intervention period.²¹ Furthermore, because the community's radio station was new, it reached only an estimated 10% of the population.²²

And in Guinea, 38% of young men and 15% of young women who responded to the postintervention survey reported having participated in any intervention activity.²³ The low proportions in Guinea likely reflect that the peer education program was not closely monitored,²⁴ and most exposure to the intervention came through attendance at soccer games, where reproductive health messages had a low educational component.

Effects on Perceptions and Beliefs

• Women. Table 1 (page 113) shows, for young women, changes between the baseline and postintervention surveys in perceptions regarding risk and prevention. In Cameroon, Botswana and Guinea, young women's perception that sexual activity carries the risk of AIDS was essentially the same before and after the interventions (i.e., odds ratios were all statistically nonsignificant). In South Africa, this perception improved in the comparison site (as indicated by the statistically significant odds ratio of 4.0) but did not change in the intervention location. The interaction term showed that the result was a net negative impact of the intervention.

The intervention in Cameroon had no net effect on the belief that sexual activity carries the risk of pregnancy, despite a significant negative trend in the comparison site. In Botswana and South Africa, significant positive trends in the intervention locations produced a net positive impact on the women's likelihood of perceiving that sexual activity carries the risk of pregnancy. In Guinea, this belief was unchanged in the intervention site, but because of a significant positive trend in the comparison site, the net impact of the intervention was negative.

The interventions in Cameroon, Botswana and South Africa had significant net positive impacts on the perceived benefits of engaging in various protective behaviors. In Cameroon, both locations exhibited positive trends in the belief that abstinence protects against pregnancy, but the trend was significantly stronger in the intervention location. The belief that condom use protects against pregnancy also increased significantly in the intervention location in Cameroon, resulting in a net positive intervention impact. The belief that other contraceptives protect against pregnancy increased in both sites in Cameroon, but again, the trend was significantly stronger in the intervention location.

In Botswana, there was a net positive impact of the intervention on the belief that abstinence protects against AIDS because of a significant positive trend in the intervention site. The belief that condoms protect against pregnancy increased significantly in both locations, but the trend was significantly stronger in the intervention location. The belief that condoms protect against AIDS increased significantly only in the intervention location, resulting in a net positive impact.

In South Africa, because of an improvement in the intervention location, the intervention had a net positive impact on the belief that abstinence protects against pregnancy. The belief that condoms protect against AIDS increased significantly only in the intervention site, and the intervention had a net positive impact. Between surveys, the belief that other contraceptives protect against pregnancy became more common in the intervention site and less common in the comparison site, resulting in a net positive impact of the intervention on this belief.

The intervention in Guinea had no net positive impact on women's perceptions of the benefits of abstinence or condom use. In fact, the belief that condom use protects against AIDS significantly increased in the comparison area but not in the intervention area, resulting in a net negative impact of the intervention on this belief.

In Cameroon, the intervention had a small positive impact on perceived barriers to practicing safer sex, because women in the comparison area became less likely to believe that it is normal for a woman to propose condom use. The interventions in Botswana and South Africa had no net impact on the perceptions of barriers; data were not collected on these indicators in Guinea.

The intervention in Cameroon had a net positive impact on the belief that AIDS is avoidable because of negative trends in the comparison area. Although the discussion of sexuality was unchanged in both areas, the net impact of the intervention was statistically significant and positive. The interventions in Botswana and Guinea had no net impact on discussion of sexual matters with a partner. In South Africa, there was a net reduction in confusion about sexual matters because of a significant reduction in confusion among those in the intervention location.

• Men. Table 2 (page 114) shows the impact of the interventions on these indicators for young men in each country except South Africa, where the data were of too poor quality to be included. The interventions had no net impact on the perceived the risk of AIDS or pregnancy from sexual activity in Cameroon or Botswana (despite positive trends in both the intervention and the comparison areas in Botswana). In Guinea, the perception that sexual activity entails a risk of pregnancy increased significantly between the baseline and postintervention surveys among young men in the comparison location, resulting in a net negative impact of the intervention.

In Cameroon, the perceived benefits of the use of condoms and other contraceptive methods increased in the intervention site and declined or remained unchanged in the comparison site, resulting in a net positive impact of the intervention. The intervention had no net impact on perceived benefits of abstinence or condom use in Botswana. In Guinea, because the belief that condom use protects against AIDS increased in the comparison site and did not change in the intervention site, the intervention had a net negative impact.

The interventions in Cameroon and Botswana produced no net positive impact on perceived barriers to abstinence and condom use. In Guinea, data were not collected on these indicators. The Cameroon intervention had a net negative impact on opposition to sex before marriage, because this attitude declined in the intervention area and did not change in the comparison location.

The interventions produced only one net change on self-efficacy among men: In Cameroon, the intervention had a net positive impact on discussion of sexuality, because of a significant reduction in discussion in the comparison location.

Effects on Behavior

• Women. Table 3 shows that the interventions in Cameroon, Botswana and South Africa had no net effect on women's sexual partnerships. The intervention in Guinea showed a negative effect on the likelihood that women had had multiple partners in the past month and a positive effect on their likelihood of having fewer partners as a means of preventing AIDS.

In Cameroon, Botswana and Guinea, some aspects of contraceptive use improved as a result of the interventions. In Cameroon, a variety of patterns of change yielded positive effects: Ever-use of condoms increased significantly in the intervention location and did not change in the comparison location; current use of condoms for pregnancy prevention increased in both sites, but the trend was stronger in the intervention site; and use of abstinence for pregnancy prevention declined in the comparison location and increased in the intervention site. In Botswana, a significantly higher likelihood of having taken any measure to protect against pregnancy was reported in the intervention location after the intervention than at baseline, but there was no change in the comparison location, resulting in a significant positive impact of the intervention. In Guinea, a significant reduction in use of condoms at last intercourse in the comparison area and no significant change in the intervention location produced a net positive impact; in addition, the intervention had a net positive impact on ever-use of the pill, because of a positive trend in the intervention area.

• Men. Table 4 shows that young men in the intervention locations in Cameroon and Botswana were less likely to report two or more partners or casual partners in the postintervention survey than at baseline, resulting in a net positive impact of the interventions on these measures.

In Cameroon and Guinea, the interventions had a net positive impact on contraceptive use. Use of a modern method for pregnancy prevention increased significantly in the intervention area in Cameroon but not in the comparison area, and the difference in trends was statistically significant. Reliance on abstinence for pregnancy prevention increased in both areas in Cameroon, but the trend was significantly stronger in the intervention area. In Guinea, because of a significant positive trend in condom use in last sex in the intervention area, the intervention had a net positive impact on this indicator.

CONCLUSIONS

The four adolescent sexual health interventions evaluated in this study relied to different degrees on mass media, sponsored events, peer education and youth-friendly contraceptive services. While the interventions were broadly similar, the reach of their programmatic elements differed in substantial ways. Some of these differences were determined by the realities of implementation or by factors beyond the control of the social marketing program (such as the reach of community radio), but they had an important impact on the outcome of the evaluation.

The Cameroon intervention, with its combination of mass media and peer education, reached most of the adolescent population of Edea. This intervention was also implemented for longer than the other three and had the greatest impact on different dimensions of sexual health beliefs and behavior. It produced a net increase in perceived benefits of protective behavior and in self-efficacy among both males and females, and a reduction in perceived barriers to protective behavior among females. Consistent with these changes, it was also associated with a reduction in risky sexual behavior among young men and an increase in contraceptive use among young men and women.

In Lobatse, Botswana, a closely monitored peer education program reached a substantial proportion of adolescents. However, because of the absence of a community radio station, adolescents in the comparison site were also exposed to the mass media component of the intervention. Thus, mass media are unlikely to have had a net impact in the intervention site in Botswana. Women's perception of the level of risk involved in sexual activity and of the benefits of preventive measures increased in the intervention site, as did their contraceptive use. Among men, there was evidence of a reduction in casual partnerships.

In Soweto, South Africa, mass media communication would have been important in promoting changes in beliefs and behavior, but the newly established community radio station reached only a small fraction of the population. Thus, the intervention had very few net positive effects among young women. The poor quality of data for men precluded the evaluation of the full impact of this intervention.

The intervention in Guinea relied primarily on sponsored events such as soccer games and concerts, which usually have low educational content and limited reach. Furthermore, because young men are more likely than young women to attend soccer games, a relatively small proportion of adolescents in the intervention sites were exposed to sexual health messages. Moreover some young persons from comparison neighborhoods may have attended proj-ect-sponsored events in the intervention neighborhoods (since both were in the same cities), and this may have diluted the impact of the intervention. The intervention in Guinea had the least impact on Health Belief indicators.

This evaluation is unique in that it has assessed multiple sexual health interventions targeted to young men and women in Sub-Saharan Africa, using the same conceptual and statistical models, a quasi-experimental study design and similar indicators. However, the study had important limitations. One is that the selection of intervention and comparison sites was based on logistical considerations (rather than being random). The multivariate analyses adjusted for differences in respondents' characteristics by controlling for age and education, and the analyses were stratified by gender. Only this limited set of control variables was used in order to keep the statistical models the same in all four countries; it is therefore possible that individual-level differences between intervention and comparison sites were not adequately controlled for. However, the findings from the South Africa and Cameroon assessments were very similar to findings from evaluations of the same interventions conducted by others²⁵ with a larger number of control variables, and this similarity supports the validity of these findings.

The social and epidemiological contexts of HIV differed in these four countries. Intervention and comparison sites in Botswana and South Africa had much higher levels of HIV prevalence than sites in Cameroon and Guinea. The absolute level of risk perceived by individuals in different social contexts may be quite important in determining what preventive measures they adopt. Moreover, adolescents in Botswana and South Africa would have come across more HIV prevention messages than adolescents in Cameroon and Guinea (because the former two countries were experiencing an HIV epidemic). How, if at all, these factors may have affected the likelihood of success of these sexual health interventions is unknown.

Overall, more positive change occurred among young women than among young men, a finding that is consistent with results of earlier research.²⁶This may reflect a better ability of these adolescent sexual health interventions to address the concerns of women than of men, or a greater receptivity to such interventions among young women than among young men. It is critical to develop a better understanding of how to address sexual health concerns of young men.

These findings suggest that when adolescents are exposed to substantial levels of an intervention that includes mass media and interpersonal communication in a context where contraceptives are available (in this case, both through the intervention and through a national social marketing program), it is likely that their attitudes and behavior related to preventive sexual health behavior will improve.