BERNHARDT ET AL. 771 Alcohol Assessment Among College Students Using Wireless Mobile Technology* JAY M. BERNHARDT, PH.D., M.P.H.,† STUART USDAN, PH.D.,† DARREN MAYS, M.P.H.,† RYAN MARTIN, PH.D.,† JENNIFER CREMEENS, PH.D., M.S.P.H.,† AND KIMBERLY JACOB ARRIOLA, PH.D., M.P.H.† National Center for Health Marketing, Centers for Disease Control and Prevention, 1600 Clifton Road NE, MS E21, Atlanta, Georgia 30329 ABSTRACT. Objective: This study used a two-group randomized design to assess the validity of measuring self-reported alcohol consumption among college students using the Handheld Assisted Network Diary (HAND), a daily diary assessment administered using wireless mobile devices. Method: A convenience sample of college students was recruited at a large, public university in the southeastern United States and randomized into two groups. A randomly assigned group of 86 students completed the daily HAND assessment during the 30-day study and a Timeline Followback (TLFB) at 30-day follow-up. A randomly assigned group of 82 students completed the paper-and-pencil Daily Social Diary (DSD) over the same study period. Data from the daily HAND assessment were compared with the TLFB completed at follow-up by participants who completed the HAND using 95% limits of agreement analysis. Furthermore, individual growth models were used to examine differences between the HAND and DSD by comparing the total drinks, drinking days, and drinks per drinking day captured by the two assessments over the study period. Results: Results suggest that the HAND captured similar levels of alcohol use compared with the TLFB completed at follow-up by the same participants. In addition, comparisons of the two study groups suggest that, controlling for baseline alcohol use and demographics, the HAND assessment captured similar levels of total drinks, drinking days, and drinks per drinking day as the paperand-pencil DSD. Conclusions: The study findings support the validity of wireless mobile devices as a daily assessment of alcohol use among college students. (J. Stud. Alcohol Drugs 70: 771-775, 2009) H EAVY ALCOHOL USE among college students is a public health problem in the United States (Substance Abuse and Mental Health Services Administration, 2004; Wechsler et al., 2000). Despite significant advances in alcohol surveillance, prevention, and intervention research, the measurement of alcohol use among college students remains a challenge for alcohol researchers because students’ drinking behaviors are characterized by infrequent, heavy drinking episodes (Dawson, 2003; Greenfield, 2000; Substance Abuse and Mental Health Services Administration, 2004). In part to overcome some of the methodological limitations of existing assessments of alcohol use, the National Institute on Alcohol Abuse and Alcoholism identified the need to determine the most effective uses of computer-based technologies to help prevent dangerous alcohol use in college settings (Task Force for the National Advisory Council on Alcohol Abuse and Alcoholism, 2002). For instance, wireless mobile devices (MDs) such as handheld computers, cellular phones, and smart phones provide a number of advantages over traditional paper-and-pencil methods of measurement and thus may be useful in improving the assessment of alcohol use among college students (Bernhardt et al., 2001). Previous research has demonstrated that the use of MDs, such as handheld computers, for collection of sensitive behavioral data (e.g., drug and alcohol use, sexual behavior) from students resulted in substantially fewer missing data points compared with paper-and-pencil questionnaires and face-to-face interviews (van Griensven et al., 2006). Research also suggests that electronic daily diaries may be a useful method for alcohol assessment; however, only a few prior studies have used the wireless capabilities of MDs for alcohol assessment among college students (Bernhardt et al., 2007; LaBrie et al., 2006; Weitzel et al., 2007). By using existing daily reporting methods of alcohol assessment and incorporating innovative wireless mobile technology, it may be possible to create an accurate assessment of drinking behaviors among college students and provide opportunities Received: February 10, 2009. Revision: June 18, 2009. *This research was supported by National Institute on Alcohol Abuse and Alcoholism grant 5R21AA013969-03. The preparation of this manuscript was supported in part by an appointment to the Research Participation Program for the Centers for Disease Control and Prevention (CDC) administered by the Oak Ridge Institute for Science and Education through an agreement between the Department of Energy and the CDC (Darren Mays). The findings and conclusions in this article are those of the authors and do not necessarily represent the views of the U.S. Department of Health and Human Services or the Centers for Disease Control and Prevention. †Correspondence may be sent to Jay M. Bernhardt at the above address or via email at: jbernhardt@cdc.gov. Stuart Usdan is with the College of Human Environmental Sciences, University of Alabama, Tuscaloosa, AL. Darren Mays is with the Rollins School of Public Health, Emory University, Atlanta, GA. Ryan Martin is with the Division on Addictions, Cambridge Health Alliance, Medford, MA. Jennifer Cremeens is with the Department of Health Education and Promotion, East Carolina University, Greenville, NC. Kimberly Jacob Arriola is with the Rollins School of Public Health, Emory University, Atlanta, GA. 771 772 JOURNAL OF STUDIES ON ALCOHOL AND DRUGS / SEPTEMBER 2009 to apply this technology for tailored interventions (Bernhardt et al., 2005, 2007; Weitzel et al., 2007). This study sought to evaluate the validity of measuring self-reported alcohol consumption among college students using daily Handheld Assisted Network Diary (HAND) assessments for alcohol use administered on wireless MDs by comparing data collected using the HAND with two validated methods of alcohol assessment. First, reported alcohol use on the HAND was compared with a paper-andpencil, self-completed Timeline Followback (TLFB; Sobell et al., 1986) assessment completed at follow-up by the same participants. Second, data collected using the HAND assessment were compared with data collected among a randomly assigned comparison group of participants who completed a paper-and-pencil Daily Social Diary (DSD) over the same 30-day period. Method Design and procedures The study used a pre-test, post-test design with randomized group assignment to examine the validity of the HAND for measuring alcohol use among college students at a large, public university in the southeastern United States. The study procedures were approved by the institutional review board at the participating institution, and all participants provided signed informed consent before taking part in the study. The methods of the study are described in detail elsewhere (Arriola et al., 2009). Briefly, a convenience sample of college students (n = 170) was randomly assigned to complete a daily assessment of their drinking using either the HAND (Bernhardt et al., 2007; Weitzel et al., 2007) or the paper-and-pencil DSD each day for 30 days. Participants were asked to complete a baseline TLFB, the daily HAND or DSD for 30 days, and a follow-up TLFB after 30 days. Measures Daily drinking was measured on the TLFB, HAND, and DSD by asking participants whether they drank alcohol on the previous day and, if so, how many standard drinks they consumed. Standard drinks were consistently defined across the three assessments as 12 oz of beer, 5 oz of wine or champagne, 3 oz of fortified wine, or 1.5 oz of distilled spirits. Conversion factors were also provided to estimate the number of standard drinks consumed based on state-specific alcohol distribution laws at that time that required drinking establishments to serve distilled spirits in single-serving bottles rather than using conventional serving methods. The TLFB administered at baseline and follow-up was self-completed (i.e., paper-and-pencil). Similar to prior research (Maisto et al., 2008), participants were provided with the TLFB calendar and instructions and were asked to complete information about their alcohol use each day for the previous 30 days. The reliability of the TLFB has been previously established by comparing it with electronic forms of measurement, including handheld computers, interactive voice response programs, computer-based assessments, and telephone interviews (Carney et al., 1998; Searles et al., 2000; Sobell et al., 1996). The questions examining drinking on the daily HAND and DSD assessments were identical; the only difference between the two assessments was the mode of administration. The DSD was based on the drinking self-monitoring log, a validated daily diary-type assessment for alcohol use (Sobell et al., 1989), and was administered in paper-and-pencil format. At baseline, DSD group participants received a packet containing a one-page assessment for each day of the study period to be completed over the course of the study, which they then turned in when completing the follow-up TLFB. The HAND was designed using EntryWare Designer software (Techneos Systems, Inc., Vancouver, BC) and administered using Palm Tungsten W smart phones (Palm Inc., Sunnyvale, CA), which were enabled to wirelessly transmit data to a secure web-based server after each assessment was completed. Participants who did not complete the HAND assessment by 12:00 PM each day were sent an automated email from the researchers reminding them to complete the HAND. Participants who still had not completed the assessment by 5:00 PM each day received an automated phone call from the researchers reminding them to complete the HAND. Between administrations of the daily HAND assessments, the MDs were locked at a waiting screen until the next day’s assessment was available, which prevented participants from completing multiple HAND assessments in 1 day. Statistical analyses Data were analyzed using SAS 9.2 (SAS Institute, Cary, NC). Variables created for the HAND, DSD, and TLFB assessments included total drinks consumed, number of drinking days, and drinks per drinking day. The 95% limits of agreement method (Bland and Altman, 2003) using a correction factor for sample sizes of less than 100 (Ludbrook, 2002) was used to assess agreement between the HAND and the follow-up TLFB. Three individual growth models were created using PROC MIXED (Singer, 1998) to compare differences in total drinks, drinking days, and drinks per drinking day between the HAND and DSD. Each outcome variable was aggregated at weekly intervals (Weeks 1 and 2 with 7 days, Weeks 3 and 4 with 8 days), totaling four time points for each individual growth model. The primary independent variable of interest was a group variable coded as “1” for the HAND group and “0” for the DSD group. Individual growth models also BERNHARDT ET AL. 773 TABLE 1. Unstandardized parameter estimates and 95% confidence intervals (CIs) comparing the Handheld Assisted Network Diary (HAND) to the Daily Social Diary (DSD) for alcohol-related variables Variable Assessment (1 = HAND, 0 = DSD) Baseline TLFB Log (1 + total drinks)a Drinking days Drinks/drinking day Gender (1 = male, 0 = female) Age Null model likelihood ratio test Total drinks b (95% CI) Drinking days b (95% CI) Drinks/drinking day b (95% CI) -0.15 (-2.52, 2.21) -0.09 (-0.37, 0.19) 0.50 (-0.41, 1.41) 7.45† (6.06, 8.90) – – – 0.18† (0.14, 0.23) – -2.97* (-5.41, -0.54) -0.25 (-0.53, 0.04) 0.42 (-0.38, 1.35) 0.14* (0.03, 0.24) χ2 = 90.9, 9 df, p < .001 χ2 = 105.41 9 df, p ≤ .001 0.66† – – (0.49, 0.83) -0.58 (-1.52, 0.37) 0.29 (-1.52, 0.37) χ2 = 50.07 9 df, p ≤ .001 Notes: TLFB = Timeline Followback. aNatural log transformation was applied because the baseline total drinks variable was strongly kurtotic. included age, gender, and the corresponding baseline TLFB drinking-variable controls. The total drinks variable from the baseline TLFB was transformed using a natural logarithm (natural log + 1) because it was strongly kurtotic (Emerson and Soto, 1983). Two participants were excluded from the analyses reported because they had extreme outlier values on all assessments of alcohol use. Comparison of the HAND with the DSD When controlling for the respective alcohol-use variables on the baseline TLFB, age, and gender, there were no statistically significant differences between the HAND and DSD in total drinks, drinking days, or drinks per drinking day captured each week during the 30-day study period (Table 1). Results Discussion Participants Of the participants included in the analyses (n = 168), 50.3% were female, the majority were white (77.7%), and the mean (SD) age was 19.96 (1.35) years. Demographic characteristics did not differ significantly between the two study groups. Among the participants in the DSD group (n = 82), the mean number of completed daily assessments on the DSD was 29.29 (0.91), with an average completion rate of 97.6%. Among participants in the HAND group (n = 86), the mean number of completed assessments on the HAND was 25.75 (5.90), with an average completion rate of 85.8%. Comparison of the HAND with the TLFB Overall, the mean total drinks (p = .21), drinking days (p = .56), and drinks per drinking day (p = .54) captured on the HAND and TLFB, respectively, did not differ significantly. The mean difference in drinks per drinking day between the HAND and TLFB was 0.07. The adjusted 95% limits of agreement using the small sample size (n < 100) correction factor (Ludbrook, 2002) were -3.68 to 3.83 standard drinks. In total, 94.2% of data (81/86 respondents) fell within this agreement interval. This study sought to evaluate the validity of measuring self-reported quantity and frequency of alcohol use among college students using the MD-based HAND. Our analyses suggest that the HAND assessment captured similar levels of alcohol use over the course of the study compared with two valid paper-and-pencil assessments of alcohol use, the TLFB and DSD. This study contributes to a growing body of literature on the assessment of alcohol consumption using a number of different communication technologies (Dawson, 2003). Specifically, this study suggests that wireless MDs may represent a valid method of assessment of daily drinking among college students. Technical aspects of the design of the two daily assessments may have affected the comparisons reported. For example, the average completion rate among DSD participants (98%) was substantially higher than among HAND participants (86%). However, the design of the HAND prevented participants from completing assessments that were missed, and HAND group participants received reminders to complete daily assessments. Neither of these features was incorporated into the design of the DSD, which likely contributed to the observed differences in completion rates in particular. These technical aspects notwithstanding, the completion rate 774 JOURNAL OF STUDIES ON ALCOHOL AND DRUGS / SEPTEMBER 2009 observed for the HAND indicates that, despite the potential burden for participants of completing the daily HAND assessment, the HAND can be completed daily with little deviation. Although research suggests that technology such as wireless MDs offers many benefits for behavioral assessment (Bernhardt et al., 2001, 2005), the evolving technology of wireless MDs, including enhanced multimedia and improved wireless capabilities of more recent MDs, may expand the benefits of applying such technology for alcohol assessment. In particular, although preliminary evidence suggests that wireless MDs may be an effective tool for delivering individually tailored messages to reduce drinking among college students (Weitzel et al., 2007), future research is needed to examine both the assessment and intervention capabilities of more recent, more advanced wireless MDs over longer time periods. The results of this study should be considered in the context of several limitations. The study used a convenience sample of college students; therefore, generalizations to broader populations should be made with caution. All drinking behaviors are based on participant self-reports. Although research suggests that self-report assessments are reliable (Del Boca and Darkes, 2003), the results should be interpreted with this limitation in mind. Because of the nature of the DSD, it could not be determined when the daily assessment was completed, thereby limiting the comparisons that can be drawn between the DSD and the HAND. Finally, participants’ reported alcohol use on the follow-up TLFB may have been affected by the fact that they completed a daily assessment of alcohol use over the preceding 30 days. The comparisons of the HAND with the follow-up TLFB in particular should be interpreted with this limitation in mind. Two primary research implications should be considered based on the present study. First, future research is needed to examine both the alcohol assessment and intervention capabilities of MDs, in particular investigating their use over longer study periods. In addition, research is needed to examine how advanced multimedia and communication capabilities of more recent wireless MDs can be incorporated into assessment and intervention methods for alcohol use and potentially other behaviors. Acknowledgments The authors are indebted to Richard E. Dixon and James B. Weaver III for their thoughtful feedback on this manuscript. References ARRIOLA, K.D., USDAN, S., MAYS, D., WEITZEL, J.A., CREMEENS, J., MARTIN, R.J., BORBA, C., AND BERNHARDT, J.M. Reliability and validity of the Alcohol Consequences Expectations Scale. Amer. J. Hlth Behav. 33: 504-512, 2009. BERNHARDT, J.M., STRECHER, V.J., BISHOP, K.R., POTTS, P., MADISON, E.M., AND THROP, J. Handheld computer-assisted self-interviews: User comfort level and preferences. Amer. J. Hlth Behav. 25: 557-563, 2001. BERNHARDT, J.M., USDAN, S.L., AND BURNETT, A. Using handheld computers for daily alcohol assessment: Results from a pilot study. J. Subst. Use 10: 347-353, 2005. BERNHARDT, J.M., USDAN, S., MAYS, D., ARRIOLA, K.J., MARTIN, R.J., CREMEENS, J., MCGILL, T., AND WEITZEL, J.A. Alcohol assessment using wireless handheld computers: A pilot study. Addict. Behav. 32: 30653070, 2007. BLAND, J.M. AND ALTMAN, D.G. Applying the right statistics: Analyses of measurement studies. Ultrasound Obstet. Gynecol. 22: 85-93, 2003. CARNEY, M.A., TENNEN, H., AFFLECK, G., DEL BOCA, F.K., AND KRANZLER, H.R. Levels and patterns of alcohol consumption using the Timeline Follow-Back, daily diaries, and real-time “electronic interviews.” J. Stud. Alcohol 59: 447-454, 1998. DAWSON, D.A. Methodological issues in measuring alcohol use. Alcohol Res. Hlth 27: 18-29, 2003. DEL BOCA, F.K. AND DARKES, J. The validity of self-reports of alcohol consumption: State of the science and challenges for research. Addiction 98 (Suppl. No. 2): 1-12, 2003. EMERSON, J.D. AND SOTO, M.A. Transforming data. In: HOAGLIN, D.C., MOSTELLER, F., AND TUKEY, J.W. (Eds.) Understanding Robust and Exploratory Data Analysis, Hoboken, NJ: John Wiley & Sons, 1983, pp. 97-128. GREENFIELD, T.K. Ways of measuring drinking patterns and the difference they make: Experience with graduated frequencies. J. Subst. Abuse 12: 33-49, 2000. LABRIE, J., EARLYWINE, M., LAMB, T., AND SHELESKY, K. Comparing electronic-keypad responses to paper-and-pencil questionnaires in group assessments of alcohol consumption and related attitudes. Addict. Behav. 31: 2334-2338, 2006. LUDBROOK, J. Statistical techniques for comparing measures and methods of measurement: A critical review. Clin. Exp. Pharmacol. Physiol. 29: 526-536, 2002. MAISTO, S.A., CONIGLIARO, J.C., GORDON, A.J., MCGINNIS, K.A., AND JUSTICE, A.C. An experimental study of the agreement of self-administration and telephone administration of the Timeline Followback interview. J. Stud. Alcohol Drugs 69: 468-471, 2008. SEARLES, J.S., HELZER, J.E., AND WALTER, D.E. Comparison of drinking patterns measured by daily reports and Timeline Follow Back. Psychol. Addict. Behav. 14: 277-286, 2000. SINGER, J.D. Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. J. Educ. Behav. Stat. 24: 323-355, 1998. SOBELL, L.C., BROWN, J., LEO, G.I., AND SOBELL, M.B. The reliability of the Alcohol Timeline Followback when administered by telephone and by computer. Drug Alcohol Depend. 42: 49-54, 1996. SOBELL, M.B., BORGARDIS, J., SHULLER, R., LEO, G.I., AND SOBELL, L.C. Is self-monitoring of alcohol consumption reactive? Behav. Assess. 11: 447-458, 1989. SOBELL, M.B., SOBELL, L.C., KLAJNER, F., PAVAN, D., AND BASIAN, E. The reliability of a timeline method for assessing normal drinker college students’ recent drinking history: Utility for alcohol research. Addict. Behav. 11: 149-161, 1986. SUBSTANCE ABUSE AND MENTAL HEALTH SERVICES ADMINISTRATION (Office of Applied Studies). Overview of Findings From the 2003 National Survey on Drug Use and Health, DHHS Publication No. 04-3963, Rockville, MD: Substance Abuse and Mental Health Services Administration, 2004. TASK FORCE OF THE NATIONAL ADVISORY COUNCIL ON ALCOHOL ABUSE AND ALCOHOLISM. A Call to Action: Changing the Culture of Drinking at U.S. Colleges, NIH Publication No. 02-5010, Bethesda, MD: National Institute on Alcohol Abuse and Alcoholism, 2002. BERNHARDT ET AL. GRIENSVEN, F., NAORAT, S., KILMARX, P.H., JEEYAPANT, S., MANOPAIBOON, C., CHAIKUMMAO, S., JENKINS, R.A., UTHAIVORAVIT, W., WASINRAPEE, P., MOCK, P.A., AND TAPPERO, J.W. Palmtop-assisted self-interviewing for the collection of sensitive behavioral data: Randomized trial with drug use urine testing. Amer. J. Epidemiol. 163: 271-278, 2006. WECHSLER, H., LEE, J.E., KUO, M., AND LEE, H. College binge drinking in VAN 775 the 1990s: A continuing problem. J. Amer. Coll. Hlth 48: 199-210, 2000. WEITZEL, J.A., BERNHARDT, J.M., USDAN, S., MAYS, D., AND GLANZ, K. Using wireless handheld computers and tailored text messaging to reduce the negative consequences of drinking alcohol. J. Stud. Alcohol Drugs 68: 534-537, 2007.