Collection of accurate behavioral data is critical for understanding the transmission dynamics of HIV and other STIs and for designing and evaluating interventions aimed at containing their spread. Researchers usually rely on self-reported data, which are subject to participants’ recall bias, question misinterpretation and reluctance to respond truthfully when asked about private behavior.¹ Social desirability bias is particularly problematic in studies involving sexual behavior, as respondents may deliberately answer questions inaccurately, either by underreporting stigmatized activities or by overreporting normative ones, if their actual behavior would be considered socially unacceptable.²

Potential respondent biases may be reduced or exacerbated by the method of questionnaire delivery. Surveys administered via face-to-face interviews, for instance, can be influenced by interviewers’ inaccurate rewording of questions, directive probes, nonverbal cues or inappropriate feedback.³ Real or perceived judgment on the part of the interviewer, or concerns participants have about the privacy of their responses, may prevent them from providing honest answers to sensitive questions.

Although the bulk of survey data from developing countries—including Demographic and Health Survey (DHS) data sets—is traditionally collected in face-to-face interviews, increasing use has been made of technology-based alternatives, such as audio computer-assisted self- interviewing (audio-CASI).⁴ With this method, participants listen to questions through headphones and enter responses using an electronic device, such as a laptop or handheld computer. This technique provides greater standardization than other interview methods by administering an identical script to each respondent and by eliminating interviewer effects. Audio-CASI also affords greater privacy to participants than other modes, and thus has the potential to reduce social desirability bias in the reporting of sensitive or stigmatized behaviors.

Use of audio-CASI has been shown to be feasible in low-resource settings (e.g., Kenya,⁵ Malawi,⁶ Zimbabwe,⁷ Brazil,⁸ India⁹ and Vietnam¹⁰) and has led to higher reporting of some outcomes. However, these and other studies have shown that results may vary by setting and sample. A systematic review of questionnaire delivery modes in developing countries found that, overall, respondents were less likely to report sensitive behaviors in face-to-face interviews than in audio-CASI,¹¹ while a second review and meta-analysis of quantitative interviewing tools in low- and middle-income countries concluded that results depended on the sensitivity of the outcome and the population under study.⁴

Motivations to report behavior truthfully may also vary by gender, particularly regarding sexual activity in contexts where gender norms create different expectations for males and females about socially acceptable behavior.¹² In Malawi, as in much of Sub-Saharan Africa, premarital and extramarital sex are considered symbols of masculine prowess,¹³ but for females, sexual activity outside of marriage is discouraged, and the social costs of reporting it are higher.¹⁴

In a study that compared reporting by males and females via face-to-face interviews at a demographic surveillance site in Malawi’s Karonga district, significant gender differences were found in the reporting of sexual partnerships, demonstrating that some combination of male overreporting and female underreporting occurred.¹⁵ Results from an earlier study at a Tanzanian demographic surveillance site were similar.¹⁶ Men have also been found to consistently report higher levels of premarital sex and risky sexual activity than have women in the DHS and other surveys.^17,18

To determine whether interview mode influences levels of reporting for each gender, the present study compares responses to sexual behavior questions administered via audio-CASI with those obtained via traditional face-to-face interviews. Given the social norms surrounding relationships and sex in rural Malawi, we expected to observe different patterns of reporting between interview modes for males and females. Specifically, we hypothesized that males’ reporting of relationship experience and sexual activity would be higher in face-to-face interviews than in audio-CASI, and that the opposite would be the case for females.^[10,19,20] However, for socially proscribed behaviors, we expected reporting to be higher in audio-CASI for both genders.

METHODS

Study Design and Samples

The audio-CASI method featured in this analysis is used in the Malawi Schooling and Adolescent Study (MSAS), a longitudinal survey administered yearly since 2007. The baseline sample comprised 2,650 adolescents—1,312 males and 1,338 females—aged 14–16 in January of that year. Two-thirds of the adolescents (889 males and 875 females) were students attending standards (grades) 4–8 at recruitment and were randomly selected from enrollment rosters at 59 primary schools in Balaka and Machinga, two contiguous rural districts in the southern region of the country.* The probability of a particular school being selected was proportional to its enrollment in 2006. Approximately 30 students stratified by gender and age were selected from each school. The remaining third of participants (423 males and 463 females) were not attending school when first interviewed in 2007.† These adolescents, who lived in the catchment villages of the participating schools, were identified through key informants at each study school or in randomly selected communities in each school’s catchment villages. Follow-up interviews have been conducted annually through 2011, and approximately 90% of adolescents in the original sample have been reinterviewed each year.

The audio-CASI data used here come from the third round of the MSAS, conducted in May through July 2009, during which 1,165 males and 1,219 females were interviewed. For the majority of respondents (94%), this was the third time they had been exposed both to the use of audio-CASI and to the series of sexual behavior questions featured in this study.

A supplementary random sample of adolescents was selected in 2009 to serve as a comparison group for this methodological experiment. A household census conducted in catchment villages of the MSAS schools provided the sampling frame. Twelve of the 59 study schools were randomly selected, with a probability of inclusion proportional to enrollment in 2006. For each selected school, four catchment villages were randomly chosen, with the exception of one school with particularly large catchment villages, for which two villages were selected. Each household in the selected villages was approached for participation in the census, during which basic social and demographic information about the household and its members was collected.‡ From the household rosters, adolescents aged 16–18 at the start of 2009 were identified and followed up for interview. This age-group was chosen to make the sample directly comparable to the MSAS participants. In total, 407 eligible adolescents (200 males and 207 females) completed the interview, representing a completion rate of nearly 80%.

As a longitudinal study, the MSAS is subject to attrition across rounds, such that the subset of the baseline sample interviewed in 2009 does not necessarily share the characteristics of a purely random sample.§ In addition, as noted earlier, the baseline MSAS sample was constructed according to schooling status, and for current students it was restricted to those attending standards 4–8. Since the supplementary sample did not share this design, we might expect some background characteristics to vary between the two groups.

The MSAS instrument consists of an extensive set of questions on household and family characteristics, schooling and employment history, marriage and childbirth, health and sexual behavior, as well as literacy and numeracy assessments. Participants in our study completed the questionnaire using a combination of interview modes. For the MSAS group, nonsensitive questions were administered via face-to-face interviews, and sensitive questions related to puberty, sexual behavior, physical abuse, and drug and alcohol use were asked using audio-CASI. The entire MSAS questionnaire was administered to the supplementary sample via face-to-face interviews. A local team of trained interviewers, matched to participants by gender, administered the survey to the MSAS sample; a subset of these interviewers were selected to conduct the face-to-face interviews with the participants in the supplementary sample. All interviews were offered in Chichewa and Chiyao; nearly all participants (98%) chose Chichewa, Malawi’s national language.

Written informed consent was obtained from all participants. For individuals in the MSAS sample who were enrolled in school, teachers were provided a letter to read to standards 4–8 and students were asked to inform their parents of potential participation in the study. Participants were asked to confirm their parents’ approval as part of the informed consent process. For out-of-school participants in the MSAS sample, and for all individuals in the supplementary sample, consent was first obtained from participants’ parents or guardians, and then from the participants themselves. Interviews for both the MSAS and the supplementary sample were conducted at school, the household or a public place, according to where participants could be located. The research protocol was approved by the Population Council institutional review board in New York and the University of Malawi Research and Publications Committee in Zomba.

The present analysis, which focuses on premarital sexual behavior, includes 1,104 males and 646 females in the MSAS (audio-CASI) sample and 195 males and 116 females in the supplementary (face-to-face interview) sample who had never been married and for whom all relevant data were available.**

Outcome Variables

To assess differences in the reporting of relationships and sexual behavior according to mode of interview, we focused on three outcomes.

•Ever had a boyfriend or girlfriend. Participants were asked if they had ever had a chibwenzi, which in Chichewa signifies "boyfriend" or "girlfriend."††

•Ever had sex. After defining sex as "full penetration" and making clear that episodes of forced sex should be included, respondents were asked to report sexual activity in two ways. First, they were asked their age at sexual debut, with the option of answering "never had sex." Next, all respondents—including those who had not reported an age at first sex—were asked a sequence of questions about whether they had ever had sex with specific types of partners, including a boyfriend or girlfriend, a "hit and run" (a person with whom the respondent had had a single sexual encounter), a relative, a teacher or any other partner. Although both audio-CASI and face-to-face interviews lend themselves to the inclusion of skip patterns, and such patterns were used in other sections of the survey, we decided to ask the series of partner questions to all respondents, regardless of whether they had provided an age at first sex. This decision reflected previous research by the authors in Kenya⁵ and Malawi⁶ that had demonstrated that some adolescents who did not initially report having had sex subsequently acknowledged sexual activity when further probed about specific partners. Interviewers reconciled inconsistent responses to the sexual behavior components, but in the audio-CASI interview, it was possible to report an age at first sex but no partner, and vice versa. We therefore considered an adolescent to have ever had sex if he or she answered affirmatively to any one of the questions on age at first sex or partner.

•Ever had sex with a stigmatized partner. We define a stigmatized partner as a relative or a teacher, owing to the particular disapprobation associated with these relationships for adolescents. We determined whether an adolescent had had sex with such a partner from responses to the series of partner questions.

For purposes of comparison between the interview groups, we also assessed two measures unrelated to relationships and sex: whether a respondent had drunk alcohol or stolen from a kiosk or vendor.‡‡ For students, these questions referred to the past school year (which began the preceding January); for those not enrolled, questions referred to the period since January 1, 2009.

Independent Variables

Participants provided information on a number of social and demographic characteristics: age, ethnic group, religion, coresidence with parents and household assets.§§ They also answered questions about current school status and highest grade ever attended, and completed exercises measuring their literacy and numeracy skills. Respondents were considered literate if they could successfully read two simple sentences in Chichewa, taken from the 2004 Malawi DHS. A numeracy score was generated by summing the number of correct answers in a 12-question exercise that involved ordering numbers; performing addition, subtraction, multiplication and division; and solving simple word problems, all of which were drawn from the Malawi Institute of Education achievement tests for primary school standard 3.

Analysis

We used two-sided z tests for proportions and t tests for means to compare the social and demographic characteristics of the two samples, separately for males and females. We also compared reports of each relationship, sexual, alcohol and theft outcome across interview modes. We then estimated multiple logistic regression models by gender to identify associations between interview mode and reporting of ever having had a boyfriend or girlfriend, ever having had sex and ever having had sex with a stigmatized partner. Standard errors were adjusted for clustering within interviewer.

All regression models controlled for variables related to educational attainment and skills, both because of the known link between school attendance and sexual activity²¹ and as a proxy for the respondent’s ability to understand and respond to survey questions. The models also included the other social and demographic characteristics presented in Table 1, as these could influence adolescents’ reporting of sexual behavior,²² and serve to control for observed differences between the two samples. In addition, to test whether multiple exposures to the MSAS questionnaire may have affected responses in the audio-CASI group, we compared the reporting of participants with complete data who were aged 16 at the start of 2007—when they were first exposed to the audio-CASI instrument—with that of individuals in the supplementary sample who were 16 at the start of 2009. These groups would be expected to have comparable levels of premarital sexual experience. Finally, the internal consistency of reporting sensitive behavior using audio-CASI was assessed. Analysis was conducted using Stata version 12.1.

RESULTS

Sample Characteristics

Overall, participants had a mean age of 17 (Table 1). The majority were currently attending school and were literate in Chichewa. Approximately 40% were Muslim, corresponding with the proportion belonging to the Yao tribe; the remainder were predominantly Christian, and members of the Chewa, Lomwe and other tribes. Two-thirds of participants resided with their mother, and almost a fifth reported that their mother had died; about one-quarter of participants had lost their father.

For both genders, significant differences in social and demographic characteristics were found between the supplementary and MSAS samples, but given the differences in sampling design, these were relatively modest. MSAS males were slightly older than their counterparts in the supplementary sample, but a higher proportion of the latter group were attending school. Among females, a larger proportion in the MSAS sample than in the supplementary sample were proficient in Chichewa and math. Regarding household characteristics, a slightly higher proportion of MSAS males reported that their father had died.

Relationship and Sexual Behavior Outcomes

A greater proportion of males than of females reported having had sex, irrespective of questionnaire delivery mode (not shown). However, differences in reporting between face-to-face interviews and audio-CASI for both genders were also observed.

Overall, the proportion of males participating in face-to-face interviews who reported ever having a girlfriend was significantly greater than that of males who used audio-CASI (69% vs. 49%— Table 2, page 17). The difference was smaller among females (44% vs. 38%) and not significant. A greater proportion of males reported their age at first sex in face-to-face interviews than in audio-CASI (70% vs. 55%), while for females the difference was smaller (40% vs. 37%) and not significant. For those who reported an age at first sex, there was no difference in the mean age of initiation across modes of interview for either gender.

Although greater proportions of males acknowledged having had sex with a girlfriend (63% vs. 40%), a "hit and run" (21% vs. 13%) or any partner (70% vs. 52%) in face-to-face interviews than in audio-CASI, the level of reporting was not universally higher in the former group for each partner type. Notably, greater percentages of males who used audio-CASI reported having had sex with a relative (8.2% vs. 3.1%) or a teacher (2.6% vs. 0.0%). Females in the audio-CASI group were also more likely than their counterparts to report having had sex with a relative (3.7% vs. 0.0%). When responses to the age at first sex and partner questions were used in combination to determine sexual experience, 70% of males reported premarital sex in face-to-face interviews, compared with 62% in audio-CASI. The opposite pattern was observed for females, but the difference was not significant. Reporting of alcohol use and petty theft was universally low and not significantly different across interview modes for either gender.

The multiple logistic regression results for each outcome largely corroborate the findings from Table 2. In adjusted models, males in the audio-CASI group were significantly less likely than those in the face-to-face interview group to report ever having had a girlfriend (odds ratio, 0.4—Table 3), but not to have ever had sex. Neither outcome varied significantly by interview mode for females. The finding that males in the audio-CASI group were more likely than those in the other group to report ever having had sex with a stigmatized partner was highly significant (3.5). Since no females reported sex with a relative or teacher in face-to-face interviews, interview mode perfectly predicted reporting of this outcome.

Robustness Findings

In multivariate analysis that compared reporting from 348 males and 245 females when they were 16 and first exposed to the audio-CASI instrument in 2007 with reporting from 74 males and 48 females in the supplementary sample who were 16 at the start of 2009, males were less likely to report ever having had a girlfriend in audio-CASI than in face-to-face interviews (odds ratio, 0.5—Table 4), but more likely to report ever having had sex with a relative or a teacher (3.0), although the latter finding was only marginally significant. Females were more likely to report ever having had sex in the computerized interview (1.8).

In an overall assessment of the internal consistency of reporting sexual behavior, among males in the audio-CASI group who said they had had premarital sex, only 71% reported both an age at first sex and at least one partner; 17% reported only an age at first sex, and 12% reported only a sex partner. Among females who used audio-CASI, consistency was lower: Sixty-two percent of those who reported ever having had sex gave both an age and a partner; 21% gave only an age and 17% only a partner. We did not observe this problem in the face-to-face interviews, because interviewers were instructed to reconcile discrepant responses to the sexual behavior components.

DISCUSSION

Our results suggest that the method used for collecting sensitive data may influence the reported prevalence of key behaviors, particularly for males. As expected, never-married males were less likely to report having had a girlfriend in audio-CASI than in face-to-face interviews, while sexual encounters with stigmatized partners were reported to a greater degree when using the former method. These results conform with similar reporting patterns in other interview mode experiments involving male samples.^19,20 Although it is not possible to know respondents’ actual behavior in the context of this study, the differences in reporting likely stem from social desirability bias rather than from other sources of error. Males’ reporting of other behaviors, including alcohol consumption and petty theft, was not significantly lower in audio-CASI, suggesting that magnified accounts of relationship and sexual experience in face-to-face interviews were not the result of universal overreporting in that mode. Given that males were more likely to report sexual encounters with a relative or teacher in audio-CASI than in face-to-face interviews, it seems improbable that they would underreport more conventional sexual activity when using the computerized method. If we conclude, then, that males exaggerate sexual experience with traditional partners in face-to-face interviews, but underreport their more stigmatized encounters, this has implications for the interpretation of behavioral data collected using this method. These patterns may also help explain the absence of a clear link between self-reported risky behavior and STIs.^23,24

Contrary to our hypothesis, but reflective of unexpected findings in other studies,^6,20 females did not report having a boyfriend or premarital sex to a significantly greater degree in audio-CASI than in face-to-face interviews. In a Kenyan study, Luke and colleagues¹⁹ posit that it is no longer socially unacceptable for young women to engage in monogamous sexual relationships, and therefore we should not expect reporting of traditional sexual partnerships to differ in face-to-face interviews relative to other interview modes. Recent research in Malawi, however, has noted females’ continued reluctance to discuss their relationships.¹⁴ Since we cannot determine actual levels of sexual or relationship experience, we cannot definitively assess the accuracy of reporting in either interview mode, but it is notable that some females acknowledged sexual encounters with a relative or teacher in audio-CASI, whereas none did so in face-to-face interviews. This suggests that audio-CASI was successful in eliciting reports of the most stigmatized behaviors, if not of other outcomes.

It is important to note that the audio-CASI arm of this experiment was drawn from a sample of adolescents who had been interviewed repeatedly as part of the Malawi School and Adolescent Study. Therefore, the 2009 survey data assessed here represent at least the second and often the third occasion that members of the MSAS sample had responded to the same series of sexual behavior questions. Panel conditioning theory suggests that as participants become more comfortable with the researchers and the study instruments through repeated interviews, they may feel less pressure to provide socially desirable responses.²⁵The past experience of MSAS participants with the study, the questionnaire and the use of audio-CASI may therefore have influenced their reporting in ways that would not have affected individuals in the supplementary random sample, who were participating for the first time. Specifically, according to our hypothesis regarding gender differences in divulging sensitive behavior, we might expect reporting of relationships and sexual activity to be lower among MSAS males who had been interviewed three times relative to their baseline reports, and reporting of stigmatized sexual encounters to be higher, all other things being equal. For females, we would expect reporting of all outcomes of interest to increase with the number of interviews.

However, we found that the pattern of reporting among 16-year-old males in their first audio-CASI interview in 2007 was consistent with that observed for the wider cohort participating for their second or third time in 2009. In contrast, 16-year-old females were more likely to report having had sex in their baseline audio-CASI interview than were their peers who did face-to-face interviews. Hence, the reporting differences observed in this study do not appear to be attributable to panel conditioning.

One limitation of this study is the absence of biomarker data, which provide a more objective means of validating self-reported sexual activity. Identifying STIs among respondents who did not report ever having sex, or measuring the strength of the association between infection and sexual behavior among those who did report it, could shed further light on the accuracy of reporting in each interview mode.^[16,26,27] In addition, some differences in the reporting of age at first sex and sex with specific partners between interview modes could stem from interviewers reconciling discrepant responses to the sexual behavior questions in the face-to-face interviews.

Nevertheless, our results suggest that questionnaire delivery mode influences the reporting of relationships and sexual behavior and that this possible influence differs by gender. On balance, audio-CASI seems to have elicited more moderated reports of overall sexual activity for males, and greater reports of sexual encounters with stigmatized partners for both genders. However, comparison of the two components of the aggregate "ever had sex" indicator reveals that the level of within-round consistency in audio-CASI is low, a finding that has been replicated elsewhere.^6,20

Conclusions

The self-administered nature of audio-CASI increases the privacy afforded to participants and may therefore reduce the influence of societal expectations on reporting, but it also removes the opportunity for interviewers to probe respondents or clarify questions. Researchers working with self-reported data should continue to develop innovative methods of data collection that attempt to address social desirability bias without compromising other aspects of data quality.^14,19 Sexual behavior research would also benefit from greater use of biomarker testing as an objective means to validate self-reports.^4,28

REFERENCES