Not so long ago, if demographers and program planners wanted to know the level of contraceptive prevalence, they had to create estimates from family planning program statistics.1 Such indirect measures may seem imprecise in the age of large population-based surveys like the Demographic and Health Surveys (DHS), but program statistics often offered the only data on contraceptive use.
The accuracy of such estimates was difficult to determine, as it depended both on the accuracy of record-keeping and on assumptions about the size of the population served, the consistency of method use, continuation rates and even (in the case of coitus-dependent methods) coital frequency. And as large, nationally representative, population-based surveys have become common in the last few decades, many people have come to believe that they provide definitive information on contraceptive prevalence and patterns of method use.
In the few instances in which information collected in a national survey has been compared with data gathered through service statistics, though, discrepancies in use rates have often been dramatic.2 For example, in 1977, Paraguay's program statistics indicated a level of contraceptive use about 40% greater than that shown in results from a national survey.3
Large discrepancies between program statistics and survey data may serve as warnings to program planners, especially if time trends differ markedly. Yet even when such discrepancies emerge, there is no established way to resolve them. We review here one past effort to investigate the causes of such a situation, and reflect on our own more recent attempts to do so.
Bangladesh's Condom Gap
A "condom gap" was detected in Bangladesh in the early 1980s, when program statistics suggested a much higher level of condom use than did national survey data.4 If both data sources were accurate, then many more condoms were being distributed than were being used; if one source was inaccurate, then it was important to determine which it was.
Researchers advanced the following 10 hypotheses to explain the condom gap:5 Wives interviewed in surveys underreport regular condom use; wives underreport irregular condom use; retailers overstock condoms; a delay occurs between a retail sale and actual use; couples use more condoms per couple-year of protection than the conventional estimate of 100 units; condoms purchased or distributed free are not used; condoms are used for noncontraceptive purposes (e.g., as balloons); a substantial share of condom use occurs outside of marriage; condoms are smuggled to neighboring countries; or, finally, sales figures from social marketing programs are inaccurate.
It was not possible to systematically evaluate all of these possibilities, but researchers did at least conduct a large-scale study in Bangladesh to evaluate whether the condom gap could be attributed to women's underreporting of condom use.6 They took a convenience sample of households in which a married couple was currently residing and the wife was 18-37 years of age. When both husband and wife were present at the time of the interview, they were interviewed separately; when only one spouse was present, he or she was interviewed.
The researchers assumed that couples interviewed at the same time would provide more valid information than respondents interviewed individually, because of each spouse's assumed concern that his or her response could be verified against their spouse's reply.
When the analysts compared the responses of the men and women who were interviewed without their spouse to those of the couples, they found that both men and women interviewed when their partner was not at home tended to report less condom use than those interviewed when their partner was home. The magnitude of underreporting was greater among women, however, and was greater still among residents of semirural areas than among those living in urban areas. They concluded that much of Bangladesh's "condom gap" could be explained by this underreporting, and they urged the development of improved data collection methodologies. However, the researchers did not consider selection bias as a possible explanation; couples who were at home together at the time of the interview may simply have had higher coital frequency than couples in which one partner was absent at the time of the interview. Coital frequency in turn may be related to contraceptive choice.
Ghana's Foaming Tablet Gap
More than a decade after the discovery of Bangladesh's condom gap, the West African country of Ghana confronted a similar scenario, this one involving vaginal foaming tablets. Information from the Accra Mission of the U.S. Agency for International Development indicated that the number of vaginal foaming tablets distributed during the past decade had increased 10-fold, from 186,000 units per quarter in 1987 to more than 2,000,000 units per quarter in 1995. DHS data showed, however, that the prevalence of vaginal foaming tablet use had remained about constant between 1988 and 1993 (about 1% in each year).7
At the same time, local program personnel produced numerous anecdotal accounts of people's having misused foaming tablets, including as shampoo, detergent or a vaginal cleansing agent. Given the dramatic difference in trends between program statistics and survey data, these reports sparked interest in searching for an explanation of Ghana's apparent "tablet gap." Therefore, we undertook a short survey of family planning providers and of users of vaginal foaming tablets, both to assess the extent of misuse in Ghana and to ascertain if misuse might explain the gap.
We limited our investigation to providers and clients in the Ghana Social Marketing Foundation distribution network in two metropolitan areas—Accra and Kumasi—for three reasons. First, the Ghana Social Marketing Foundation is the largest distributor of vaginal foaming tablets in Ghana (having 44% of the market share), and distributes a large proportion of its tablets in Accra and Kumasi (55%). In addition, this social marketing program requires the consumer to pay for the contraceptive; thus, we could assume that product wastage was minimal. Finally, the increase in tablet distribution in the Ghana Social Marketing Foundation program in the previous decade was similar to the increase seen at the national level.
We drew a random sample of 100 pharmacies and chemist shops serviced by the Ghana Social Marketing Foundation, and interviewed the first available shopkeeper or clerk in each. In addition, we chose a convenience sample of 12 outlets from among these 100, and conducted a client-intercept study of 300 customers who entered the outlet to purchase vaginal foaming tablets.
Sales figures from the Ghana Social Marketing Foundation showed that 1.23 million units of vaginal foaming tablets (a brand known as Kamal) had been sold in Accra and Kumasi in the six months prior to the study. The outlets that were sampled reported having sold an average of 440 tablets per month. Since there are 470 outlets overall in the Kumasi and Accra areas, this suggests that they sold about 1.24 million units (440 tablets per month x 470 outlets x six months). This close agreement between the actual and estimated sales figures provides confidence in the sample's representativeness.
According to both the providers and the clients, vaginal foaming tablets were being used almost exclusively for family planning. One provider said that on rare occasions the tablets were also used to prevent AIDS and other sexually transmitted diseases (STDs), and a small proportion of clients (15%) reported having heard of the tablets being used against STDs, with only one having heard of them being used against AIDS.
Fewer than 5% of clients had heard of people misusing vaginal foaming tablets, with 4% (12 clients) saying that ground-up tablets are mixed with soap and used as a skin bleach. Two of these individuals also said they had heard of the tablets being used as shaving cream, and one client said that she had heard that the tablets were inserted after sex to prevent STDs. Only two clients admitted having themselves used vaginal foaming tablets as a skin bleach.
Our investigation thus provides no evidence that the discrepancy between unit distribution and contraceptive prevalence of vaginal foaming tablets was due to widespread misuse of the method. Unfortunately, many alternative explanations for the discrepancy remain unexplored: The findings of our analysis might not be generalizable to other clients (especially to those in areas other than Accra and Kumasi or to clients attending other family planning outlets); unit distribution figures may be inaccurate; the DHS estimates may lack precision; vaginal foaming tablet use may differ over time (i.e., use might have become more consistent, or people may use the tablets with different types of partners); or supplies may be smuggled to neighboring countries, where vaginal foaming tablets reportedly are more expensive.
It could be that all five of these factors play a role, to varying degrees. The only firm conclusion that we can draw from our small study is that since misuse seems not to be a major problem, the local program should probably not devote scarce resources to attempts at reducing product misuse.
Conclusion
Just as earlier investigators did when faced with a discrepancy between program statistics and survey data, we used operations research to explore one of many potential explanations for the tablet gap in Ghana. Those researchers concluded that to resolve discrepancies such as the condom gap, surveys used to assess trends in fertility and contraceptive practices needed to be improved and couples, not individuals, ought to be interviewed (despite the added difficulty and expense).8
We draw a different conclusion. We agree that some attempts should be made to improve data collection. We also believe, though, that researchers are apt to expend considerable time and resources by conducting large-scale surveys to evaluate the various possible explanations for such divergences between program statistics and survey data, without drawing any clear programmatic implications.
When a country is faced with a "contraceptive gap" like that seen a decade ago in Bangladesh or more recently in Ghana, two important questions should be answered. First, is the product being misused? Second, are products expiring in the distribution pipeline or simply not being used by the consumer? In most settings, answers to these two questions can be obtained quickly and for a minimal cost, as is demonstrated by our survey in Ghana. Moreover, answers to these questions may readily translate into programmatically relevant recommendations.