Overview

The Prevention Needs Assessment survey is sponsored by the Wyoming Department of Health and facilitated by the Wyoming Survey & Analysis Center at the University of Wyoming.

Since 2001-2002 school year, the PNA monitors the percent of students that have used or are currently using alcohol, tobacco, and other drugs. Additionally, the PNA has measured a wide variety of attitudes, beliefs, and perceptions of students in 6-12th grades that are related to alcohol, tobacco, and drug use.

Though primarily seeking information about substance use, the PNA has broadened its scope to include collecting information about mental health concerns, nutrition, and physical activity. The information from this survey can be used to choose and evaluate prevention interventions, strategies, and policies. Data from this survey is publicly available for the state and counties in Wyoming.

Most schools and school districts who participate in the survey can access information from the survey for their students. The information from the survey can be used by schools, school districts, and communities in Wyoming.

FAQ for Parents & Schools

Why is the survey being conducted?

The Wyoming Department of Health uses the Prevention Needs Assessment (PNA) survey results to measure youth substance use and student participation in anti-social behaviors. The survey results will be used by communities and the state to help understand the factors that influence students' decisions to use substances or participate in anti-social behavior. The survey also measures mental health concerns and information about health and nutrition.

How long does it take to fill out the survey?

The current survey is just over 100 questions long and takes about 45 minutes for most students. Typically the research teams asks that the school, where the survey is being administered, allot one full class period to administering the survey. In past years, most if not all the students were able to finish the survey in the provided time.

Does the survey include a physical exam?

The survey does not require a physical test or exam.

Is this going to be a lot of work for me to administer the survey?

No. We are trying to make sure that the survey administration is as easy and simple as possible. We will be doing the work of sorting, organizing, and preparing the surveys for all the classrooms and schools where it will be administered. We will be as accommodating as possible to make this process easy for you and will strive to come up with solutions to any concerns.

Who supports this survey?

The original core survey items were developed, tested, and normed using a grant from the National Institutes of Health. The Substance Abuse and Mental Health Services Administration, through the Center for Substance Abuse Prevention, encourages the use of this survey for community and state level planning as part of many grants sponsored by that agency. These federal grants include the Strategic Prevention Framework—Partnerships for Success Grants, the Federal Prevention Block Grants, and the Safe and Drug Free Communities Grants.

State level support for the PNA comes from the Wyoming Department of Health which has sponsored the survey every biennium since the 2001-2002 school year. Additional support comes from the Wyoming Department of Education. These state agencies have used information from the PNA to help plan and evaluate a variety of substance abuse prevention projects. Many local communities, school districts, and schools also support the PNA because it provides them with local level information that is not available from any other source to help them design and implement their own substance abuse and violence prevention systems.

Are sensitive questions asked?

Yes, some questions on the PNA are sensitive. The primary purpose of this survey is to measure substance use by our youth and related attitudes, beliefs, and behaviors that have been shown to be related to youth substance use. The survey also asks about mental health concerns, including suicide. These topics can be considered sensitive. As a result, the questions are written in a direct but non-threatening way. Additionally, the students are informed that their participation in the survey is voluntary and anonymous. The students may skip any questions that they do not feel comfortable answering and their responses are not individually identifiable. Consequently, the risk of responding to the survey is minimal, and is no greater than what the students might experience as they complete their typical school work.

Do students tell the truth?

Yes. (See “Do Students Tell the Truth on the PNA Survey?”) To summarize:

Reliability and validity. Based on published studies of the psychometric properties of the PNA survey items, the reliability and validity of these measures are well established. If you would like to read more, WYSAC can give you copies of these published papers.

Survey environment. The survey is conducted in a way that reassures the students feel they can answer the questions openly and honestly because their survey responses are anonymous. They are told to not write their name or any other personally identifiable information on the survey. The classroom is arranged so the student’s peers and the survey administrators cannot view the student’s answers to the questions. Students who complete the paper survey turn in their questionnaire into a common data collection envelope with all the rest of the surveys for the classroom. When the survey is administered online, all the students in the class use the same access code, and the survey instructions explain that only the students' answers to the survey are recorded. Other information, such as the computer’s IP address or other potentially personally identifiable information, is not collected.

Questionnaire design and content. The questionnaire was designed with some honesty checks to help identify those surveys in which the student may not have responded truthfully. These honesty checks typically only affect a small percentage of the surveys. If a survey is identified as not being honest, then that survey is removed from the dataset prior to analysis to minimize the effect of potential dishonesty.

Consistency across time and with other survey results. The PNA results in the past have generally been consistent with other national, state, and local surveys. The previous times that the PNA has been administered have produced similar results, both at the state, and local levels. Subgroup differences also remain consistent across time. Thus, the results seem to be consistent and comparable both across time and have similar results as other comparable surveys.

How do parents find out about the survey?

The primary means of informing parents about the survey will be through a letter sent to parents. The letter will be distributed approximately two weeks before the school’s survey administration date. WYSAC will provide the letter that will be sent to the parents at each school. If so desired, schools may also inform parents about the survey in other ways, such as newsletters, email lists or on school websites. WYSAC will work with the schools to ensure that all parents are informed of the survey prior to the survey administration date.

What if some parents do not want their children to complete the survey?

All parents have the option to request their child not take the survey. Parents can indicate this by returning the bottom portion of the survey information letter to the school, indicating they do not want to have their child participate in the survey. The school is then instructed to not administer the survey to those students whose parents withdrew their permission. The survey administrator's training explains this process in greater detail.

Will students’ names be used or linked to the surveys?

No. The survey has been designed to protect the students’ privacy. Students do not provide their name on the survey and are asked not to provide any other personally identifiable information.

The research team has designed the way in which to collect the data to be anonymous. Specifically, when the survey is administered using paper surveys, the students place their questionnaires in a collection box or envelope with all the other students' questionnaires. This helps ensure the surveys are not individually identifiable. Additionally, the survey materials for the paper questionnaire include pencils for the student to use when completing the survey so the marks on the survey are as similar as possible.

When the survey is administered online, then a common access code is used for the entire classroom. This is similar to the classroom collection envelope used in the paper survey. Other than the classroom access code, the survey administration web-application only collects the students’ responses to the survey. No other personally identifiable information, like IP addresses, are collected.

What survey results do we receive if we participate in the PNA?

Survey results for the state and the counties in Wyoming are available from the PNA website. The survey administration team at WYSAC can give many school district superintendents, school principals, and other authorized individuals access to their local school and district results on the PNA website. These local results are only available when the schools and districts have sufficient response rates and numbers of participating students to ensure valid and reliable results. These local results are only available when the anonymity of the students are protected.

How were our students chosen to be in the survey?

The goal of this project is to survey all 6^th, 8^th, 10^th, and 12^th grade students in Wyoming or, in other words, to conduct a complete statewide census of all students within those grade levels. Schools were chosen because they teach students who are in one or more of those grade levels.

To which of my students should I administer the survey?

The survey was designed to be administered to all students in the 6^th, 8^th, 10^th, or 12^th grades. Much of the PNA analysis depends upon having the correct grade levels represented in the dataset. If the survey is administered to other grade levels, that data will be removed from the dataset to prevent invalidating the results.

When will the survey be administered?

The survey can be administered from early February of the survey year until approximately mid-May in the same year. To minimize absences, the research team encourages the schools to administer the survey on a Tuesday, Wednesday, or Thursday. We will work with schools to choose the class period on the chosen day for survey administration.

Who can serve as survey administrators?

In most schools the classroom teachers will serve as survey administrators. They will be charged with sending the parental consent letters home two weeks prior to the survey administration date, along with administering the surveys to their own students using the prepared survey packets for each classroom. Each school principal will designate a school survey coordinator to aid the teachers and help assure that the survey process goes smoothly.

The school survey coordinator will be responsible for:

1. Distributing the parental consent letters to the classroom teachers so that the teachers can send the letters home with the students.
2. Distributing the classroom survey packets to the teachers prior to the survey administration date.
3. If applicable, collecting the completed classroom survey packets and returning them to WYSAC using the prepaid mailing label.

What is the survey administrator training and when will it take place?

The survey administrator training is a series of short web-video presentations about different aspects of the survey administration. The survey administrator training is available here. Survey administrators and coordinators are encouraged to view the web training at their convenience prior to the survey date in their school.

Where can I get access to the previous year’s results?

The PNA website provides the survey results for all the survey years that the survey has been administered. The survey has changed over time, and so the website provides the results only for the items that are on the most recently released survey. If a current survey item has been asked in past years, then the website allows you to view the results for all the past years where the item was present on the survey.

Local school district and school level results may be available as well for school district superintendents, school principals, and others authorized to view those results. Often these results are limited only to the most recent survey year’s results.

If you are a Wyoming school district superintendent, please contact Eric Canen at pna@uwyo.edu and he can arrange access to your local data results.

Do Students Tell the Truth on the PNA Survey?

The honesty of survey respondents is always a concern in survey research. The PNA uses a variety of procedures to encourage students to respond honestly. In addition there are procedures in place to help detect and correct for some dishonesty in survey responses. Based on these procedures as well as other evidence explained below, it appears that the vast majority of students answer the PNA openly and honestly. As a result, we are confident that the data from the PNA can be relied upon as an accurate picture of Wyoming’s students.

Procedures to Encourage Honesty

Survey Environment

The survey administration procedures are designed to foster an environment in which students feel that they can respond honestly. The procedures promote the anonymity of students’ responses and the confidentiality of the schools’ data. The instructions given to the students encourage open and honest responses.

The procedures include:

• Assigning the students to sit as far apart as possible throughout the classroom.
• Restricting the teachers and/or survey administrators to remain in a location in the classroom from which they cannot observe the students’ responses.
• For paper survey administration, designate a central collection box or envelope in the classroom where the students themselves place the surveys. Teachers inform the students that the collection box or envelope will be sealed immediately at the end of the surveying period so that no one in the school can gain access and associate any questionnaire with a specific person.
• For electronic survey administration, a common survey access code is used by the entire class that only is associated with the school where the survey is being administered. Additionally the survey only collects the students’ responses to the survey. Other information that might be personally identifiable, such as IP address is not collected.
• Telling the students about the importance of providing honest answers. Teachers will tell students that their answers will help improve programs and policies for students.

Questionnaire Design and Content

• The PNA questionnaire is designed to protect the anonymity of the students.
• No names or other types of personally identifying information are ever requested. Students are instructed not to put any identifying marks on the questionnaire.
• No skip patterns are used on the questionnaire (e.g., “if you answer ‘yes’ to question 7, skip to question 105”). Every student is asked to answer all the questions. This helps to ensure that all the students complete the questionnaire in approximately the same amount of time.
• To help students accurately understand and respond to the survey questions, the questionnaire is written for a 7th grade reading level. Questions are written in a straightforward and direct manner. For clarity, nearly all the survey questions require the student to choose only one response.
• The survey has been tested to make sure that it can be completed in a single classroom period.

Edit Checks

The original survey developers instituted a series of dishonesty criteria to identify surveys where the student’s responses appeared to be highly exaggerated or where the student admits to being completely dishonest. Surveys that meet these criteria are excluded from the analysis. In previous years, seven percent or less of the returned surveys met these honesty criteria. For more information about Wyoming PNA honesty measures, please contact WYSAC’s principal investigator for this project. The researchers also exclude from the analysis survey participants who have large numbers of logically inconsistent responses.

Evidence for Honest Responding

Logic within Groups of Questions

Questions with similar topics often have a logical relationship that would be expected if the students were responding honestly to the survey. For example, more students would be expected to admit to having tried cigarettes during their lifetime than the number of students who have used cigarettes during the past thirty days, and even fewer students would be expected to admit having smoked more than half a pack of cigarettes per day during the last 30 days. In almost every case, PNA results validate these logical patterns.

Comparison of PNA Results with Results from Other Surveys

When PNA results are compared to results from other national, state, and local surveys on the same topics, the results are generally quite similar. This is especially evident when differences in survey administration, sample selection, question wording and analysis methods are taken into consideration.

Psychometric Studies

In 2002, a set of researchers from the University of Washington published a scientific paper explaining the development and psychometric testing of the core survey questions used in the PNA (Arthur, Hawkins, Pollard, Catalano & Baglioni, 2002). The development included using and modifying established survey measures for key attitudes, beliefs, and behaviors that have been shown in scientific literature to increase or decrease the odds that youth will use alcohol, tobacco, and other drugs or exhibit other problem behaviors.

These survey questions were tested and refined until a survey instrument was developed that could be administered in a single 45 minute class period to 6^th, 8^th, 10^th, and 12^th grade students. The survey questions demonstrated adequate reliability and validity with regard to predicting youth substance use. Further studies (Glaser, Van Horn, Arthur, Hawkins & Catalano, 2005) have found that the core survey questions seem to measure similar attitudinal and behavioral constructs across five different race and ethnic groups and for both genders. Overall, these studies indicate that students respond to the PNA with enough honesty that it does not generally threaten the reliability and validity of the survey results.

References

Arthur, M. W., Hawkins, J. D., Pollard, J. A., Catalano, R. F., & Baglioni, A. J. (2002). Measuring risk and protective factors for substance use, delinquency, and other adolescent problem behaviors: The Communities That Care Youth Survey. Evaluation Review, 26(6), 575-601.

Glaser, R. R., Van Horn, M. L., Arthur, M. W., Hawkins, J. D., & Catalano, R. F. (2005). Measurement Properties of the Communities That Care Youth Survey Across Demographic Groups. Journal of Quantitative Psychology, 21(1), 73-102.

PNA & YRBS: A Comparison

Comparing the Wyoming Prevention Needs Assessment Survey and the Youth Risk Behavior Survey

There have been questions regarding two adolescent health surveys that have been done in Wyoming schools. These surveys are the Wyoming Prevention Needs Assessment (PNA) survey conducted by the Wyoming Department of Health, Public Health Division and the Center for Disease Control’s Youth Risk Behavior Survey (YRBS). The last time the YRBS was administered in Wyoming is 2015. The PNA is currently the only statewide student survey being done in Wyoming schools. For individuals familiar with the YRBS, this page clarifies the similarities and differences between the two surveys.

What is the Focus of the Surveys?

Who are the Participants in the Surveys?

Are the Participants Anonymous?

What Type of District-Level Participation is Desired for the Surveys?

What are the Costs of the Surveys?

What is the Reliability of the Surveys?

How Long are the Surveys?

What is the Yearly Timeframe of the Surveys?

When will the Results be Available?

How will the Survey Information be Used?