SELECT AN EVIDENCE-BASED PROGRAM
The second module (Needs Assessment: Getting To Know Your Audiences Better) of the Using What Works: Adapting Evidence-Based Programs to Fit Your Needs series of the National Cancer Institute features additional information on needs assessment: Website
The first step in selecting an EBP is to identify the needs of your audience. Knowing the needs of your audience will enable you to select an intervention that is relevant and has the potential to make a positive impact. “Needs” can be defined simply as the difference between present conditions and more desirable conditions. Needs are generally classified into two categories: actual (or true) and perceived (or reported). Actual needs can be objectively identified and are substantiated or demonstrated by measurements, such as death or disease rates. Perceived needs are reported by audience members as they draw from their experiences, ideas, feelings, and instincts.
Actual and perceived needs are not mutually exclusive. Often times the two will overlap because audience members will perceive needs that are substantiated by objective data. It would take enormous amounts of time and energy to classify all of an audience’s needs as perceived, actual, or both. A more efficient approach is to identify as many needs as possible and then determine which of these can be addressed through your organization’s programming.
Needs can be identified through primary and secondary sources of data. These data types are explained in the subsequent sections. Though it is ideal to gather data from numerous sources, your organization may not have the resources to use all of the sources that are provided as examples. Do not let this discourage you from assessing your audience’s needs. Simply select the sources that are realistic given your organization’s resources. If at all possible, use at least one primary and one secondary source. For information on what to do with needs assessment data once it is collected, see the content on Prioritizing Needs.
FIGURE 3. EXISTING INSTRUMENTS AVAILABLE FOR USE
Example Sources of Instruments
Sources of Primary Data
Primary data are data you collect yourself. Generally, primary data answer questions that are tailored to your audience and situation, which is a major advantage of this data type. A major disadvantage of primary data is the need to assure the validity and reliability of the tools and techniques used to collect the data. Validity is the degree to which a tool (e.g., survey) or technique (e.g., observation) measures what it is designed to measure. Reliability is the degree to which a tool or technique yields results that are consistent. While the abovementioned definitions are succinct, there are several types of validity and reliability. Experts with backgrounds in research methods and statistics are best able to determine which tools and techniques are sound. If you do not feel anyone at your organization is well suited to determine if tools and techniques are valid and reliable, you might benefit from existing instruments that can be adapted and used by your organization (see Figure 3). The advantages and disadvantages of various primary data collection methods are summarized in Table 2.
Single-step or cross-sectional surveys
In single-step or cross-sectional surveys, participants are asked to self-report answers to questions during a single contact. Surveys can be used to gather information about a variety of topics including subjective experiences (e.g., health concerns, health behaviors, feelings about a health program) and preferences (e.g., health programs of interest, the best location or time to offer a program). Though self-reported measures can evaluate a number of topics, they rely on participants’ abilities to accurately recall information and to answer honestly. Several survey approaches are discussed below.
Written questionnaires — Surveys in the form of a written questionnaire are frequently used to collect self-reported data. Written questionnaires can be distributed to participants in a number of ways, including mail. It is best to use simple and clear language on a questionnaire to avoid confusion. It is also beneficial to place questions in a logical order. Seniors often have trouble with their vision, so it is best to use a 14 point (or larger) font. To encourage completion, surveys should be concise and to-the-point. A mix of open-ended questions, which prompt narrative answers (e.g., What sports do you play?), and close-ended questions, which offer response choices (e.g., Please place a check mark by each sport you play: __Football __Basketball __ Soccer __Tennis), can be used. Remember that open-ended questions generally take longer to answer and require more effort than close-ended questions. Personal or sensitive questions (e.g., income, ethnicity, drug or alcohol use) should be limited to the extent possible and placed towards the end of the survey. Provide clear instructions, keep the survey pages uncluttered, use an easy-to-read font, and provide plenty of room for responses. Comprehensive information on questionnaire design can be found in Module 4 — Questionnaire Design of the Centers for Disease Control and Prevention’s Reproductive Health Epidemiology Series
Electronic questionnaires — Electronic questionnaires are written questionnaires that are in an electronic format. They may be distributed by email or linked to webpages. These questionnaires may be the most affordable and quickest to administer. Special attention must be paid to privacy when using electronic questionnaires. Additionally, some seniors may be uncomfortable with computer technology, so be sure to assess your audience to see if they are at ease with an electronic format.
Telephone interviews — Trained interviewers, along with a questionnaire that includes a script, are required to conduct telephone interviews. The interviewers must be careful to ask the questions in an objective manner, so they do not “lead” participants to answer in a given way. The organization conducting the interviews must have access to a directory of audience members’ phone numbers. The Wallace Foundation has created a useful guide for organizations interested in conducting telephone surveys
Face-to-face interviews — Questionnaires can also be administered through face-to-face interviews. Trained interviewers who can build rapport with participants are needed for this approach. Like in telephone interviews, the interviewers must be careful to ask the questions in an objective manner, so participants are not led to answer in a particular way. This is generally the most expensive and most time consuming method of questionnaire administration. Additional information about face-to-face interviews is available in the previously mentioned questionnaire design document produced by the Centers for Disease Control and Prevention (see section on written questionnaires).
The Delphi technique is a form of multi-step survey where the administrators contact participants more than once. Generally, participants are contacted via mail or email with one or two broad questions. The responses to these questions are analyzed and a second questionnaire, with more specific questions, is returned to participants. Several iterations or cycles of this process (usually five or fewer) will occur until consensus is reached. The Delphi technique is often used with groups of experts on a given topic. During a needs assessment, key informants (individuals with the knowledge and ability to report the needs of the audience) serve as experts. Additional details on the Delphi method, including sample invitation letters, are available on a tip sheet developed by the University of Wisconsin Cooperative Extension.
Community forums, also termed town hall meetings, bring together the audience and needs assessors. The forums or meetings are often held in easily accessible buildings, such as libraries or schools, and are scheduled at convenient times. Once participants have gathered, a moderator will explain the meeting purpose and will ask participants to share their ideas and concerns. Generally, participants will stand up before the group or even stand before a centrally located microphone when sharing. The moderator should be skilled in maintaining order, keeping the meeting on track, and eliciting responses from participants who are reluctant to share. Recorders should take notes or audio record the session so responses can be analyzed after the meeting. Tips on planning a town hall meeting are available from the Centers for Disease Control and Prevention.
A group of 8-12 individuals are guided through an open discussion during a focus group. Ideally, focus groups are comprised of similar individuals who do not know one another. Both the similarity among individuals and the fact that they will not see one another again lower inhibitions and encourage sharing. The moderator stimulates free-flowing conversation by providing approximately 10 open-ended questions as prompts. While free-flowing conversation is the goal, the moderator helps participants focus on the questions at hand in order for needs assessors to obtain the information they need. A 45-90 minute discussion is long enough to gather data without fatiguing participants. It is best for an audio recording to be made so the conversation can be transcribed and analyzed for themes. Always be sure to ask permission from participants before recording them. Also, it is beneficial for an observer to take notes during the focus group in case the recording fails. The Community Tool Box of the University of Kansas provides guidelines for planning and conducting focus groups:
Nominal groups are comprised of approximately five to ten participants. A facilitator will present the participants with a question. Each participant then documents his or her answer silently and independently. Once each participant has documented an answer, each shares his or her response in a round-robin fashion. The facilitator documents each answer in a way that all participants can see (e.g., chalk board, white board, flipchart). Once all of the responses have been documented, the participants discuss the responses for clarification and then rank-order the responses by importance. In some cases, participants vote privately to rank-order the responses. In other cases, participants will have a preliminary vote that is followed by discussion before conducting a final vote. A brief on nominal groups is available from the Centers for Disease Control and Prevention.
Members of the audience and the environment surrounding them can be observed as part of a needs assessment. For example, needs assessors may stand on a street corner and observe drivers to determine how many are wearing seatbelts. Additional behaviors and environmental conditions that can be observed include the use of hard hats by workers in a construction site, cigarette smoking by employees during work breaks, or vegetable consumption by individuals in a cafeteria. A popular form of observation is a windshield survey or walking tour, during which assessors drive and/or walk through a designated area, viewing as much of it as possible. While moving through the area, they watch for various predetermined elements (e.g., public transportation availability, housing type and quality, existence and quality of parks). The assessors document notes about what they see, hear, and smell regarding the elements. The notes are later analyzed for themes. A brief guide to windshield surveys is available from The Community Tool Box of the University of Kansas.
TABLE 2 - PRIMARY DATA COLLECTION METHODS
Sources of Secondary Data
Secondary data are data that have already been collected by someone else. Secondary data are advantageous because they are often available for immediate use and are inexpensive, if not free, to use. However, because secondary data were collected previously, it is difficult to determine if the data are still applicable to the audience or if appropriate methodologies were used during data collection. These are significant disadvantages. Obtaining secondary data from reputable sources, several of which are identified below, can help you capitalize on the advantages of secondary data and guard against the disadvantages.
International — Abundant data on global health can be found at the World Health Organization (WHO) website and the websites of its regional offices (European, South-East Asia, Americas, etc.). The WHO is the directing and coordinating authority for health within the United Nations system. Data and statistics are available by health topic or by nation. Additionally, the WHO website contains a number of publications on evidence-based practices and program.
National — Numerous governmental organizations provide useful information for program planners. Demographic data (e.g., median household income, percent of population over age 65) can be found on the U. S. Census Bureau website. The Centers for Disease Control and Prevention (CDC), a component of the U. S. Department of Health and Human Services, provides health data in a variety of formats (see Table 3) on its website.
State and local — Each state has a respective state-level health agency and local (e.g., regional, county, city) health agencies. Various terms are used to denote these agencies, including health departments, health services, and health districts. These agencies often conduct periodic needs assessments, which may contain important information regarding your audience. They may also provide information regarding funding opportunities for EBPs. In Texas, the Department of State Health Services is the state-level health agency. Useful information available on the Department of State Health Services website includes disease and death rates for the state, details on available health services, and contact information for local health departments.
Non-governmental agencies and organizations
Voluntary health organizations — Voluntary health agencies are nonprofit organizations comprised of individuals (lay and professional) who seek to improve and protect the health of a particular group of people or combat a particular disease, disability, or type of disease or disability. These agencies receive funding from voluntary contributions. Their activities often include education, advocacy, research funding, patient support groups, and other services to affected individuals and communities. As part of their educational efforts, voluntary health organizations will often provide data related to their respective disease or disability emphases. Well-known voluntary health organizations include the American Cancer Society and the American Heart Association. There are voluntary health organizations for almost all diseases and disabilities. Speaking with individuals with these conditions, contacting local healthcare specialists, and searching the internet can help you identify these organizations.
Foundations — Organizations established by endowment with provision for future maintenance are considered to be foundations. Several health-related foundations provide useful data for determining the needs of an audience. For example, the Robert Wood Johnson Foundation, the nation's largest philanthropy devoted solely to the public's health, partners with the University of Wisconsin Population Health Institute and Community Catalyst to maintain the County Health Rankings & Roadmaps program . The Rankings portion of the program provides information on death and disease rates, as well as data on health behaviors, healthcare access, and socioeconomic factors for each county in the U. S. The Roadmaps portion of the program provides grants to local coalitions and partnerships among the policy, business, education, public health, healthcare, and community sectors; grants to national organizations working to improve health; recognition of communities whose promising efforts have led to better health; and customized technical assistance on strategies to improve health. Other foundations that provide useful data include the Henry J. Kaiser Family Foundation, which focuses on healthcare, statistics, and the John A. Hartford Foundation, which focuses on improving healthcare for older Americans.
Academic institutions — Many universities and colleges have needs assessment data on the communities surrounding their institutions. For example, the Center for Community Health Development of the Texas A&M Health Science Center School of Rural Public Health, in conjunction with the Brazos Valley Health Partnership, generated the Brazos Valley Health Assessment, a comprehensive, valid, and reliable source of needs assessment information for the Brazos Valley region of Texas. Another example is The Lower Rio Grande Valley Community Health Assessment generated by The University of Texas Health Science Center at Houston School of Public Health. Contact local institutions to determine if they have any relevant needs assessment data. Colleges and universities can also be a source for interns, technical assistance, and funding for EBPs.
Healthcare organizations — Hospitals and other healthcare institutions often collect needs assessment data on the communities in their service areas. Though any healthcare organization may provide this sort of data, it is especially beneficial to check with non-profit hospitals, which are now required by provisions in the Patient Protection and Affordable Care Act to conduct community health needs assessments and then implement plans to meet identified needs. An example resource is the community health needs assessment conducted by Scott & White Healthcare, a non-profit hospital system in Central Texas. On this page, users can find information on demographics, community health indicators, health disparities, and promising practices.