General

The Principles of Effectiveness: Early awareness and plans for implementation in a national sample of public schools and their districts

Description
The Principles of Effectiveness: Early awareness and plans for implementation in a national sample of public schools and their districts
Categories
Published
of 5
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
Share
Transcript
  Research Papers The Principles of Effectiveness: Early Awareness and Plans for Implementation in a National Sample of Public Schools and Their Districts Ashley P. Simons-Rudolph, Susan T. Ennett, Christopher L. Ringwalt, Louise Ann Rohrbach, Amy A. Vincus, Ruby E Johnson ABSTRACT: The US Department of Education’s Principles of Effectiveness require recipients of Safe and Drug-Free Schools and Community Act,funds to: a) base drug and violence prevention programs on needs assessment data, b) develop measurable program goals and objectives, c) implement programs with research evidence ($effectiveness, and d periodically evaluate programs relative to their goals and objectives. This paper reports the extent of awareness of the Principles of Effectiveness and plans for their imple- mentation among public school districts and schools in the United States in the year ollowing their announcement. Results showed a greater percentage of public school districts than individual schools knew about the principles and planned for implementation, but baseline levels of awareness or both districts and schools were relatively low). Schools were more likely to know about the principles when their associated school district also knew. Results sugg st a need for greater communication about the principles to school districts, and in turn, a need for greater communication between district and school-level substance use prevention staff. J Sch Health. 2003;73(5): 81-185) n July 1, 1998, the US Department of Education 0 romulgated the Principles of Effectiveness and made compliance with the principles prerequisite for public schools to continue receiving funds through the Safe and Drug-Free Schools and Community Act (SDFSCA).’ The SDFSCA represents the single largest source of funding for school drug and violence prevention programs,* and the principles seek to ensure accountability for those funds by school district recipients. In brief, the four principles, outlined in Figure 1, require a school district to: a) base its drug and violence prevention programs on needs assess- ment data, b) develop measurable program goals and objec- tives, c) implement programs with research evidence of effectiveness, and d) periodically evaluate its programs relative to their goals and objectives. The principles mandate that schools implement science-based drug and violence prevention programs. Because school districts often do not follow science-based practices, such as the tendency to implement programs without demonstrated effecti~eness,’.~ he principles may induce change in the Ashley P. Simons-Rudolph MPP Research Analyst, RTI International, PO Box 12194, Research Triangle Park, NC 27709; (apsr@rti.org); Susan T nnett PhD Associate Professor, Dept. of Health Behavior and Health Education, CB 7440, Universiry of North Carolina at Chapel Hill, Chapel Hill, NC 27599; (sennetteemail.unc.edu); Christopher L. Ringwalt DrPH Center Director, Pacific Institute for Research and Evaluarion, 1229 E. Franklin St., Chapel Hill, NC 27514; (ringwalt@pire.orgj; Louise Ann Rohrbach PhD Assistant Professor, Research Institute for Prevention Research, University of Southern Cali,fornia, 1000 S. Fremont Ave., Alhambra, CA 91803; (rohrbac@hsc.usc.edu); Amy A. Vincus MPH Research Analyst, RTI International, PO Box 12194, Research Triangle Park, NC 27709; (avin- cus@rti.org); and Ruby E. Johnson MS Statistician, RTI International, P.O. Box 12194, Research Triangle Park, NC 27709; (rej@rti.org). This paper was supported by Grant 5 R01 DA11492 from the National Institute on Drug Abuse. This article was submitted July 19, 2002, and revised and accepted for publication February 18,2003. types of drug and violence prevention programs that schools implement. This project assessed the extent of awareness of the Principles of Effectiveness, and determined plans for their implementation among public school districts and schools in the United States in the year following their announce- ment. The findings provide baseline values against which to compare dissemination of the principles over time. Prevalence rates both for school districts and schools were estimated, as well as key demographic characteristics potentially associated with awareness and implementation at each level. The district level was examined because school districts receive SDFSCA funds, and districts repre- sent the primary target audience for the principles. Districts also are responsible for disseminating information to schools and guiding overall district substance use preven- tion practices. Schools, however, often retain considerable autonomy in their prevention programs and ultimately constitute the sites where the principles are implemented. Examining both districts and schools proved important for assessing whether they communicate effectively about the principles. As the primary recipients of the principles, a greater proportion of districts than schools was expected to know about and have plans to implement them. With the presumption of communication between districts and their schools, schools located in districts where the administra- tion knew about the principles and made plans for imple- mentation were expected likewise to know about the principles and to have plans to implement them. METHODS The School-Based Substance Use Prevention Programs Study (SSUPPS), a national survey of substance use Journal of School Health May 2003 Vol. 73 No. 5 181  prevention practices in middle grades for the 1998-1999 school year, was the source of the data. The SSUPPS included two samples: a random sample of all regular public and private schools in the 50 states and District of Columbia that included some subset of middle grades, and a sample of the school districts in which the selected public schools were nested. Separate surveys concerning substance use prevention programs and practices were mailed to respondents in the individual school and school district samples. Samples for this investigation included only public schools and public school districts. Samples The sampling frame for schools was constructed using the QED National Education Database.s Eligible schools included grades seven or eight, or were limited to grade six, or to grades five and six. Schools were excluded if they were nonregular schools (for example, alternative), govem- mental schools (for example, Bureau of Native American Affairs), schools with fewer than 20 students, and schools with no substance use prevention program (3.2% of sampled schools). A random sample of public schools, stratified by school district poverty index 5 lo , 11% - 40%, 2 41% households below the federal poverty line), population density (urban, suburban, rural), and school size I 00, 200-599, 2 600 middle school students) was selected with equal probability within each stratum. Of 2,273 eligible public schools, 1,656 (72.9%) participated in data collection. With random sampling, results may be generalized to US public schools that include middle grades. After the sample of schools was selected, the associated Figure 1 US Department of Education’s Principles of Effectiveness for Drug and Violence Prevention Programs Effective July 1998, a recipient of Title IV funds shall: 1 Base its programs on a thorough assessment of objective data about the drug and violence problems in schools and communities served. 2. With the assistance of a local or regional advisory council where required by the SDFSCA, establish a set of measurable goals and objectives and design its programs to meet those goals and objectives. 3. Design and implement its programs based on research or evaluation that provides evidence that the programs used prevent or reduce drug use, violence, or disruptive behavior among youth. 4. Evaluate its programs periodically to assess its progress toward achieving its goals and objectives, and use its evaluation results to refine, improve, and strengthen its program, and to refine its goals and objectives as appropriate. Source: US Department of Education. Safe and Drug-Free Schools Program: notice of final principles of effectiveness. Federal Register 1998;63:29902-29906, sample of school districts was identified. This process created a non-random sample of districts. Nevertheless, the probability of selection for each district was known, so the sample could be used to generate valid estimates for school districts nationwide. Of 1,984 eligible school districts, 1,593 (80.3 ) participated in data collection. Additional information about the sample designs was reported else- where.6 Data Collection Data were collected from February to September 1999. Questionnaires were sent by certified mail to the individual in each school, and in each district, considered most knowl- edgeable about the substance use prevention program. For nearly all schools, the appropriate respondent was identi- fied in advance through telephone contact either with some- one in the principal’s office or by the substance use prevention coordinator serving the school’s district. School district substance use prevention coordinators were identi- fied in advance by telephoning the district central office. Following the initial mailing, a reminder postcard was mailed to all respondents. A telephone prompt was made, and up to two additional questionnaires were mailed, as needed, to achieve the response rates indicated. Questionnaires contained questions about substance use prevention programs and other activities either in the school or the district. Most school respondents were teachers (63.0 ), and the remainder were substance use prevention coordinators, principals, counselors, and other school staff. Most district respondents were Safe and Drug-Free Schools Coordinators (74.5%) andlor substance use prevention coordinators (25.0%). Respondents received $10 for completing a 45-minute questionnaire. Measures Measures for assessing the Principles of Effectiveness were developed specifically for the study by the research team because previously developed measures did not exist for assessing this new policy. Awareness of the Principles of Effectiveness was measured by asking respondents if they received any written or verbal information about the principles, after first identifying the principles by name and listing each of the four. Respondents who answered “yes” were contrasted with those who answered either “no” or “not sure.” Plans for implementing were assessed by discrete ques- tions asking if the school or the district planned to take each of four actions reflecting the principles during the current school year: administering a student survey about drug use, writing measurable program goals and objectives, adopting a prevention program shown effective based on research or evaluation evidence, and evaluating a substance use prevention program. Questions were written in refer- ence to substance use prevention programs rather than to both substance use and violence prevention programs. With the exception of having evaluation plans, a single item measured the plans for each principle. Evaluation plans were assessed by two items: whether the school or the district had plans for evaluating its substance use preven- tion program, and whether it planned to participate in an evaluation conducted by a university or research organiza- tion. Respondents who answered yes to either item were coded as having plans for evaluation. Response codes for 182 Journal of School Health May 2003 ol 73 o. 5  each of the four measures of the plans for implementing the principles were the same as for the awareness measures. From the individual measures, a dichotomous summary measure was created that contrasted those having plans to implement any of the four actions with those having no plans. Other measures selected to describe which schools and districts were aware of the principles, and which had plans for implementation, were school and district demographic characteristics, including school or district size (total student enrollment), percent of White students in the school or district, region of the country (West, Northeast, Midwest, South), population density (urban, suburban, rural), and poverty. At the school level, poverty was measured by the percentage of students eligible for free or reduced-price lunch as part of a federal assistance program. At the district level, poverty was measured by the Orshansky index, which represents the number of students falling below the federal government poverty line as a percentage of all students within a district.' Table 1 contains descriptive statistics for the demographic characteristics of the school and district samples. Statistical Analyses Descriptive statistics (weighted and 95% confidence interval) for the measures of awareness of, and plans to implement, the Principles of Effectiveness are presented. At both the district and school levels, associations were exam- ined between demographic characteristics and awareness of the principles and plans to implement one or more of them. Table 1 Sample Demographic Characteristics for School Districts and Schools School (N = 1,593) Districts Schools (N = 1 6wb Region ( ) Northeast Midwest South West 17.5 15.9 40.0 31 O 24.9 34.3 17.6 18.9 Population Density ( ) Urban 4.2 20.8 Suburban 39.2 42.2 Rural 56.6 37.0 Mean White ( ) 81.8 72.9 Mean poverty' ( ) 25.0 43.4 Mean number of students 3,531.3 541.2 a Sample sizes ranged from 1,592 to 1,593 because of missing data. Sample sizes ranged from 1,459 to 1,656 because of missing data. Poverty is measured at the district level by the Orshanky index and at the school level by the percentage of students eligible for free or reduced-price lunch. Because the measures of awareness and implementation plans were dichotomous, logistic regression models were used. Finally, to assess the association between schools and districts in their awareness and implementation plans for the principles, the sample was limited to 1,369 schools and their associated districts from which respondents at both levels returned questionnaires. Associations between the district- and school-level measures of awareness and imple- mentation plans were assessed using chi-square tests for the bivariate relationships, and logistic regression for analyses in which the demographic characteristics were included as control variables. Analyses were conducted with Survey Data Analysis (SUDAAN) software, version 7.5, to provide variance esti- mates that account for the study's complex sampling design.' All analyses were weighted u priori to adjust for the sampling design and ex post fucto for nonresponse bias. Respondents in poor, urban, and largely minority schools were less likely to respond to the school-level survey. Respondents in urban and suburban districts, as well as those in the Northeast and Midwest, were less likely to respond to the district-level survey. Results may be general- ized to US public schools and their districts that include middle school grades. Table 2 Weighted Percentage of School Districts and Schools Aware of and Planning to Implement he Principles of Effectiveness POE) School Districts Schools N = 1,593) N = 1 656)b 95 CI 95%CI Aware of the POE 59.6 56.7, 62.5 22.3 20.2, 24.4 Plans for student survey 56.1 53.3, 59.0 30.9 28.7, 33.2 Plans for measurable goals and objectives 52.0 49.1, 54.9 21.9 19.8, 23.9 adopting an effective program 32.9 30.3, 35.6 18.5 16.5, 20.4 evaluation 53.5 50.6, 50.8 28.8 26.5, 31.0 Plans for Plans for No. of POE planned for implementation 0 21.1 18.7, 23.6 50.1 47.6, 52.6 1 19.1 16.8, 21.5 20.6 18.6, 22.6 2 21.9 19.5, 24.3 14.9 13.1, 16.6 3 20.8 18.5, 23.0 8.8 7.4, 10.3 4 17.1 15.0, 19.2 5.6 4.5, 6.7 a Sample sizes ranged from 1,532 to 1,593 because of missing data. Sample sizes ranged from 1,595 to 1,656 because of missing data. ~ ~ ~ Journal of School Health May 2003 ol 73 o. 5 183  RESULTS More school districts than schools knew about the Principles of Effectiveness and had plans to implement them (Table 2). At both levels, plans to survey students about substance use and to evaluate the substance use prevention program were the most common, while plans for adopting a substance use prevention program shown effec- tive were least common. Approximately 21% of districts had no plans for implementing the principles, while about 17 planned to implement all four. Among schools, analo- gous figures were approximately 50 and 6%, respectively. Results from the models examining the relationships of demographic variables with awareness of the principles and intentions to implement at least one of them showed that district size was a significant correlate of both. The larger the district, the greater the likelihood of being aware of the principles (Wald F = 39.8, p < .OOOO , and of having plans for implementation (Wald F = 16.9, p < .0000). None of the demographic characteristics at the school level was associ- ated with either awareness of the principles or plans for implementation. Schools in districts reporting awareness of the principles were significantly more likely themselves to report the same awareness (25.4%) compared with schools in districts not reporting awareness of the principles (14.5%) (x = 20.9, 1 df, p < .0000). This relationship remained statisti- cally significant (p < .OOOS after controlling for demo- graphic characteristics of the districts and schools. Similarly, schools in districts with plans to implement any of the principles also were more likely to have such plans (91.9%) when compared with schools in districts with no plans for implementation (79.6%) (x = 38.0, 1 df, p < .0000). The relationship remained significant (p < .OOOO after controlling for demographic characteristics. DISCUSSION In the year following promulgation, almost 60% of public school districts nationwide reported knowing about the Principles of Effectiveness, while the analogous figure among public schools with middle grades was only 20%. As the immediate target audience, the higher percent among districts relative to schools was expected. The find- ing that schools were more likely to know about the princi- ples when their associated school district also knew of them suggests that some districts were communicating informa- tion to schools. However, the very low level of knowledge at the school level suggests that more substantial communi- cation efforts are needed between district substance use prevention coordinators and school staff. Given that over 97% of public school districts receive funding through the SDFSCA, and that funding requires compliance with the principles,' further communication to district prevention coordinators evidently is also needed to ensure broader dissemination. Although knowledge of the principles logically precedes their implementation, the actions recommended represent sound educational practice, and therefore, one could argue that the principles already should be in place within the public school system.' * At the school level, the percentage of respondents reporting plans to establish measurable goals and objectives for the school's substance use preven- tion program, as well as plans to evaluate the program, exceeded the percentage that knew of the principles. In general, however, for both districts and schools a smaller percentage reported plans for implementation than reported awareness of the principles. These findings suggest a need for communication, train- ing, and technical assistance to assist district and school prevention staff in implementing and complying with the principles. As one example of such efforts, the Department of Education and the National Network of Safe and Drug- Free Schools Program Coordinators sponsor such training for new coordinators. The finding that school district size was the only district- or school-related demographic char- acteristic associated with measures of the principles suggests communication efforts should particularly target smaller districts. This goal likely will prove challenging in that such districts receive fewer SDFSCA resources and thus are less likely to support a drug prevention specialist. Among plans for implementing actions regarding each of the four principles, both districts and schools were least likely to have plans for adopting a substance use prevention program with evidence of effectiveness. Only approxi- mately one-third of districts and one-fifth of schools reported such plans. At both levels, the percentage is markedly lower than for other planned activities. Yet, a number of programs with evidence from research of effec- tiveness in preventing and reducing drug use by youth are now available, such as Life Skills Training, Project ALERT,I4 and Project Toward No Tobacco Use (TNT).Is To help identify such programs, schools can turn to any of several federal- or foundation-sponsored registries, includ- ing the Department of Education, that recommend such Implementing such programs rather than programs for which available data do not provide evidence of effectiveness (for example, D.A.R.E.) or which have not been scientifically evaluated at all s critical to school drug use prevention efforts. Results from this study should be considered in light of the following limitations. First, although the measures of awareness of the Principles of Effectiveness referred to both drug and violence prevention programs, measures of plans for implementation were asked in reference only to substance use. Hence, estimates of plans for implementing the principles may be lower than is the case and should thus be considered conservative. Moreover, plans in the current school year were queried. Given that the principles were adopted just prior to the current school year, many districts and schools might not have had time to put plans into place. A longer time frame might have shown that more districts and schools were making plans to implement each of the principles. Alternatively, some districts and schools might already have completed actions indicated by the principles and thus not had plans for the coming school year. This factor also could lead to underestimating the effects of the principles on school practice. CONCLUSION Future assessment of the Principles of Effectiveness is needed to gauge their dissemination and impact on school practice. Although awareness will undoubtedly increase over time from the baseline levels shown here, concomitant compliance cannot necessarily be expected. Communication is needed from federal and state policymakers, to district substance use prevention coordinators, to principals, and to 184 Journal of School Health May 2003 ol 73 o. 5  prevention teachers to ensure the principles are understood, endorsed, and appropriately implemented. Training and technical assistance will be key in this regard, as well as reminders of any consequences of non-compliance. Schools also may need more resources to implement the principles, particularly schools that require local data collection and analysis. The science-based approach of the Principles of Effectiveness represents best educational practices and the best hope for school efforts to influence youth drug use and violence. Their full implementation promises to be a potent component of the nation's drug use prevention agenda. H References 1. US Dept of Education. Safe and Drug-Free Schools Program: notice of final principles of effectiveness. Federal Register. 1998;63:29902- 29906. 2. Mayer 0 Substance abuse prevention takes to the classroom. State Legislatures. 1999;25:24-27. 3. Wenter DL, Ennett ST, Ribisl K, Vincus AA, Rohrbach LA, Jones SM. Comprehensiveness of substance use prevention programs in US middle schools. J Adolesc Health. 2001;30:455-462. 4. Silvia ES, Thorne J, Tashijian CA. School-Based Prevention Programs: A Longitudinal Study in Selected School Districts. Final Report of the United States Department of Education. Research Triangle Park, NC: Research Triangle Institute; 1997. 5. Quality Education Data I. Denver, Colo: Quality Education Data, Inc; 1998. 6. Jones SM, Sutton BC, Boyle KE. Survey methodology for studying substance use prevention programs in schools. Recent Adv Stat Methods. 2002: 157- 168. 7. Sawhill IV. Poverty in the US: why is it so persistent? J Econ Lit. 1988;25: 1075- 1077. 8. Shah BV, Barnwell GS, Bieler GS. SUDAAN User's Manual. Research Triangle Park, NC: Research Triangle Institute; 1996. 9. Tashjian MD, Silvia S, Thome J. Characteristics ofSDFSCA State and Local Programs: Summary of the 1991 93 State Biennial Performance Reports. Report to the US Department of Education. Research Triangle Park, NC: Research Triangle Institute; 1997. 10. Center for Substance Abuse Prevention. Decision support system. 2002. Available at: http://www.preventiondss.org/. 11, Northwest Regional Educational Laboratory. Catalog of school reform models: User's guide to selecting the right model. 2001. Available at: www.nwrel.org/scpd/catalogjguide/selecting.shtml. 12. Yen FB, Brinkerhoff CV, eds. Evaluation Resource Notebook for AOD Coordinotors. Portland, Ore: Western Regional Center for Drug- Free Schools and Communities; 1992. 13. Botvin GJ, Baker E, Dusenbury L, Botvin E, Diaz. T. Long-term follow-up of a randomized drug abuse prevention trial in a white middle- class population. JAMA. 1995;273:1106-I 112. 14. Ellickson PL, Bell RM. Drug prevention in junior high: a multi-site longitudinal test. Science. 1990247:199-1305. 15. Sussman S, Dent CW, Stacy AW, et al. Project Toward No Tobacco Use: I-year behavior outcomes. Am J Public Health. 1993;83: 1245-1250. 16. Center for Substance Abuse Prevention. CSAP's prevention portal: Model programs. 2001. Available at: http://www.samhsa.gov/centers/ csap/modelprograms/default.htm. 17. Centers for Disease Control and Prevention. Guidelines for school health programs to prevent tobacco use and addiction. MMWR. 1994;43:1-18. 18. Drug Strategies. Making the Grade: A Guide to School Drug Prevention Progranzs. Washington, DC: Drug Strategies; 1999. 19. Preventing Drug Use Among Children and Adolescents: A Research-Based Guide. Rockville, Md: National Institute on Drug Abuse; 1997. 20. US Dept of Education. Safe and Drug-Free Schools and Communities Program Model Programs. 2001. Available at: lncludes tips li ke Walk Talk' Instead of sitting at the table to do homework, take a walk with your child while practicing spelling words, multiplication tables, or U S Department of Health and Human Services CDC FOUNDATION Metlife oundation Journal of School Health May 2003 ol 73 o. 5 185
Search
Similar documents
View more...
Tags
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks