Development and implementation of a website assessment tool to evaluate policies and resources for sexual assault and alcohol use on US college campuses

Abstract

Background

Sexual assault on college campuses is a major public health concern. Over half of incidents of sexual violence on college campuses occur in the context of victim or perpetrator alcohol use. Students who have experienced sexual violence, including in the context of alcohol use, are likely to seek information about sexual violence resources and services online.

Methods

We developed a Website Assessment Tool to capture a student’s perspective when navigating campus sexual violence and alcohol websites to find resources and services. The tool evaluates (1) accessibility, (2) comprehensibility, and (3) inclusivity of sexual violence and alcohol prevention information and resources. Alcohol policies and alcohol-related campus resources are also evaluated for linkage to sexual violence information. We assessed websites from 36 college campuses across the USA participating in a sexual violence prevention study.

Results

The accessibility, comprehensibility, and inclusivity of educational content, services, and supports related to sexual violence and alcohol use varied widely. Websites with Title IX regulations, outlining protections against sex-based discrimination, were easily accessible for each school. Identifying whether services were confidential was inconsistent. Sexual violence resources were difficult to locate or embedded in lengthy text and difficult to comprehend, terms were not always defined, and descriptions of services were not explicitly inclusive of diverse populations. The alcohol policies for each school were accessible. Most campuses’ (97%) alcohol policies and resources did not include sexual violence-related information or link alcohol-related information to their student counseling or health center website.

Conclusions

This Website Assessment Tool may be useful for college campuses seeking to optimize their students’ receipt of information related to sexual violence and alcohol to increase awareness of sexual violence support services and to connect survivors with resources. Both sexual violence and alcohol use prevention and intervention strategies should include the prevalence and increased risk of alcohol-involved sexual violence on college campuses to help reduce alcohol-involved sexual violence and ensure survivors are connected to supports and services with few to no barriers.

Background

Sexual assault on college campuses is a major public health concern. Over a quarter (25.9%) of female-identifying and 6.9% of male-identifying undergraduate students report experiencing nonconsensual sexual contact since entering college [1]. For students with a disability [1,2,3,4] and students who identify as a gender and sexual minority, [5,6,7] rates of sexual violence (SV) are even higher [1]. Alcohol-involved SV is common on college campuses, with 50–70% of sexual assault cases occurring in the context of victim or perpetrator alcohol use [1, 8,9,10,11,12,13,14,15,16,17]. SV, especially alcohol-involved SV, is underreported, [18] with few students seeking SV services. SV is often underreported due to stigma, social norms that blame survivors, fear of consequences related to underage drinking, and complex reporting processes that may prevent survivors from being in control of the situation [19,20,21,22,23]. Students experiencing marginalization and oppression including students who identify as LGBTQ + , who are from minoritized backgrounds, and who have disabilities experience higher rates of SV victimization and are less likely to seek formal services, [24,25,26,27,28,29] underscoring the need for inclusive and accessible resources. Students may be unsure whether an incident was an assault [30] or may struggle with acknowledging victimization especially when alcohol was involved, [31] underscoring the need for education and consistent messaging related to drinking and consent on campuses. Survivor support services can reduce mental health symptoms, self-blame, and continued victimization; however, these services are undersubscribed [32,33,34,35,36,37,38]. Low levels of awareness and use of SV support services [39,40,41,42,43,44,45,46] highlight the need to better connect victims with such resources.

Given how common seeking health-related information online has become, [47] student survivors and their peers are likely to turn to their college campus website to find support and resources after an experience of SV. Online resources are accessible 24/7 and confidential, increasing the likelihood that young adults will seek information by going online after experiencing SV. Title IX is a US federal civil rights law that provides protection against discrimination within educational institutions based on sex, sexual orientation, and gender identity [48] and ensures every college creates policies, procedures, and prevention and intervention strategies for SV [49]. The Clery Act requires colleges to annually report crime statistics to students including sexual assault, rape, fondling, incest, and statutory rape. Under the Clery Act, colleges must provide students with prevention programs and trainings, an explanation of their rights, and access to counseling services and legal services [50]. In the context of trauma, survivors may experience information processing and decision-making difficulties and have trouble concentrating, [51,52,53] demonstrating the need for clear, convenient, and understandable SV-related resources and information on where and how to seek help [54,55,56].

Alcohol policies and resources are also relevant for SV prevention and response, as SV and drinking co-occur and students who are underage may delay seeking care for SV when alcohol was involved [46, 47]. Evidence suggests that information about such policies is difficult to find and may use language that is challenging for students to comprehend [48]. One study found that college campuses with liberal or nonrestrictive campus alcohol polices reported higher rates of SV than college campuses with more restrictive alcohol polices, [49, 50] suggesting that campus alcohol polices may influence risk of SV on a campus [50]. Similarly, more restrictive state alcohol policies are associated with fewer alcohol-related disciplinary actions and rape offenses on college campuses [51].

To evaluate information about resources and services available on campus websites, as part of a larger study on alcohol-related SV on college campuses, our team developed and implemented an assessment tool to examine access to educational content, services, and supports related to SV and alcohol use. The purpose of this tool is to assess, from the perspective of an undergraduate student, how accessible, comprehensible, and inclusive school SV and alcohol-related websites are with the ultimate goal of offering campuses a guide to improve how they present information online. We explored three questions related to SV and alcohol-related websites as we implemented this tool across campuses: (1) how difficult is it to find information about SV and alcohol-related resources, (2) how understandable is the available information for an undergraduate student, and (3) how are information and resources tailored to be inclusive for lesbian, gay, bisexual, transgender, questioning, or queer (LGBTQ +), trans/nonbinary, Black, Indigenous, and people of color (BIPOC), and students with disabilities. This article reports on findings from implementing this tool with a range of diverse campuses, highlighting areas where accessibility, comprehensibility, and inclusivity could be optimized. We also assessed how alcohol policies and resources related to alcohol use were evaluated for their linkage to SV-related information.

Methods

In preparation for a larger clinical trial (NCT05185440), we conducted a website scan to assess prevention and intervention strategies and programs at college campuses to address SV, including alcohol-related SV. The current manuscript focuses on data from the website scans to illustrate the utility of assessing characteristics of online resources on each campus. The project team consisted of researchers from public health, nursing, social work, adolescent medicine, and campus-based SV prevention advocates, high school and undergraduate students, and survivors. Participating campuses (n = 36) were located across the USA, with institutions ranging from small (< 3000 students) (n = 13) to large (> 3000 students) (n = 23) in student population. Campuses were public (n = 19), private (n = 17), religious (n = 6), and secular (n = 30) institutions.

We developed a Website Assessment Tool (Additional File 1: Website Assessment Tool) to capture a student’s perspective when navigating campus SV and alcohol websites to find resources and services. We incorporated elements from a previous digital checklist developed by members of our team [57] as well as a qualitative environmental scan [58, 59] developed for a prior college campus SV prevention study (NCT02355470) [60]. The previous digital checklist [57] focused on a school’s online presence in SV prevention, assessing accessibility of SV resources and services, website visualization, website reading level, and rank of Google Search results for SV phrases like “rape” and “sexual assault” to evaluate how visible SV prevention and services are on a campus. Other website reviews [55, 57, 61] and a campus toolkit designed for prevention specialists [62] have focused more generally on assessing availability and location of SV information and resources and how alcohol’s role in campus sexual assault is represented. The tool we describe in this manuscript was designed to evaluate a website from a student’s perspective searching for information including ease of finding SV information as well as alcohol policy and resources to support students who are at elevated risk for SV victimization in the context of hazardous drinking. Additional questions assessed inclusivity of resources for students experiencing marginalization and oppression. The tool is comprised of multiple-choice, dichotomous, and open-ended questions and responses and is divided to assess online resources for SV and alcohol-involved SV. For SV resources, team members used the tool to examine definitions of forms of sexual- and gender-based violence, types of resources provided, linkages to SV prevention programs, and the school’s nondiscrimination policy. The tool assesses whether and where policies for alcohol use include connections to SV. The tool begins with a Google Search webpage (as it is the most used search engine) and then navigates first to the Title IX page for the college or university to complete SV-related questions and then to alcohol-related pages. As having information about Title IX is part of federal legislation required for college campuses, the search for SV-related information started with Title IX followed by questions about links to the main page with SV prevention and response information (indicated by having a heading such as “Sexual Misconduct Response”).

Measures

The tool examined SV prevention information and resources listed on college websites to evaluate the following: (1) accessibility, (2) comprehensibility, and (3) inclusivity. Additionally, alcohol policies and available resources were evaluated for their linkage to SV-related information.

We defined “accessibility” as the ease of finding specific information by recording the number of clicks required to reach each topic of interest, starting from the Title IX landing page for SV information and the Alcohol Policy page for alcohol information. We used the following criteria to measure accessibility: very easy (0 click), easy (1–2 clicks), difficult (3–6 clicks), very difficult (7 + clicks), unable to find/link broken, or located behind a password wall. Although there are no empirical studies on number of clicks and disengagement, we denoted three clicks or more as being difficult based on online discussions about optimal website design. We calculated means and standard deviations (SD) for each question, ranging from 1 (very easy) to 4 (very difficult).

“Comprehensibility,” meaning how easy the information available is to understand, was operationalized as whether resources were clearly described and labeled as “confidential” or “non-confidential,” whether SV terms (such as rape and dating violence) had definitions, and whether a summary of relevant resources was available for download or sharing (such as a flyer, one page sheet, or infographic).

“Inclusivity” was defined as having SV-related information and resources for people who are traditionally excluded or marginalized. We assessed whether the websites specifically identified resources relevant for people with transgender and gender diverse identities, with diverse sexual orientations, with disabilities, and from diverse racial and ethnic backgrounds, as well as highlighted nondiscrimination policies which are especially relevant for students experiencing marginalization.

Tool implementation

Four team members conducted scans from November 2023 to July 2024; all reviewers were current or pre-college students. The senior research coordinator trained each reviewer on conducting the website scan using the tool. Reviewers initially piloted this tool with eight schools. After reviewing preliminary findings with the team, we modified questions and response options for clarity. Reviewers then reapplied the revised tool with the first eight campuses. The senior research coordinator conducted quality checks for each campus website assessment consisting of scanning for missing data and spot-checking by randomly selecting questions to review for each school. The senior research coordinator shared any discrepancies and errors found with the team in a continuous quality checking process. Reviewers then completed the remaining website scans with the revised tool. With the data from 36 schools compiled, the study team met together to conduct a final quality check and resolve any remaining uncertainties about how to categorize website content. Each website review was completed in approximately 1 h, although navigation of some websites took up to 2 h (see supplement for copy of the tool).

Data analysis

To evaluate (1) accessibility, (2) comprehensibility, and (3) inclusivity of SV and alcohol prevention information and resources on college websites, descriptive statistics (i.e., counts, percentages, means (SD)) summarized the data across all 36 college campuses. Data were collected via a REDCap survey and stored within REDCap (Research Electronic Data Capture tools hosted at the University of Pittsburgh) [63, 64].

Results

Sexual violence-related information

Accessibility

The Title IX page was often the first result from the Google Search when using the prompt “[SCHOOL] Title IX.” Every school’s Title IX page was accessible from a Google Search as either the first (n = 33), second (n = 1), or third (n = 2) result (Table 1). On their respective Title IX page, 91% of schools provided a list of confidential vs. nonconfidential resources. The level of difficulty (by clicks) to find this information ranged from very easy (n = 2), easy (n = 20), to difficult (n = 5).

Table 1 Sexual violence information and resources for accessibility

Almost three-quarters (72%) of schools provided resources and flow of reporting processes on their Title IX page. The level of difficulty finding this information was either very easy (n = 2), easy (n = 19), or difficult (n = 4). One school’s reporting processes and information were listed behind a password-protected wall.

From their Title IX pages, 86% of schools provided information on local and community-based victim advocacy programs that was very easy (n = 2), easy (n = 25), or difficult (n = 4) to find. Most (n = 32) schools provided 24/7 accessible confidential resources that were very easy (n = 5), easy (n = 23), or difficult (n = 4) to find. Almost three-quarters (72%) of schools provided links to their main sexual violence prevention and response page making it either easy (n = 22) or difficult (n = 4) to find. Additionally, six schools included prevention and response information on their Title IX page, which made it very easy to find (0 click).

On their SV prevention and response pages, 58% (n = 21) included resources for groups disproportionately impacted by SV. These connections to resources were easy (n = 16), difficult (n = 4), or inaccessible due to a broken link (n = 1).

The nondiscrimination policies were on average “easy” to navigate from the SV prevention response page, very easy (n = 2), easy (n = 18), difficult (n = 7), or very difficult (n = 1).

The reviewers noted challenges with website formatting in their open-ended comments, even though the SV-related information was accessible per number of clicks. Many sites were text heavy (large paragraphs), not in plain language, missing information, and disorganized (e.g., resources were not grouped or labeled by type of services like medical vs emotional). Reviewers noted challenges with finding relevant information about resources and services when such information was located in difficult-to-navigate drop-down menus; behind a password-protected wall, contained in a lengthy PDF of a student handbook or school’s safety report; or found through searching external webpages linked from another office [31]. Reviewers frequently encountered broken links to supplemental materials (n = 10 campuses). For schools that had multiple campuses, resources were often not tailored for the satellite campus but linked back to the school’s main campus page making it more difficult to locate local services and linkages to support on the satellite campus. Three schools provided a list of names and contact information for resources without including information about what the resource offers or how it could be used.

Comprehensibility

Almost all of the schools (92%) provided a description of what confidential vs. nonconfidential resources were, although only half (51%) defined “confidential” and “non-confidential” in plain language (Table 2). The remaining 18% of schools either did not provide a clear distinction between confidential vs. nonconfidential resources, or this information could not be found listed on the Title IX page. Most (89%) of schools provided 24/7 clearly identified confidential resources for students to access.

Table 2 Sexual violence information and resources for comprehensibility

Twenty-six schools provided a flow and resources overview of the reporting processes on their Title IX page that included information such as who to talk to, what the reporting processes might look like, and potential outcomes of reporting SV.

About three-quarters of schools (72%) linked their Title IX page to a main SV prevention and response page (noted by a title such as “Sexual Misconduct Prevention and Response”), while the Title IX page and SV prevention and response page were the same page for 11% of schools (n = 6).

The prevention and response page for 33 schools listed and defined multiple forms of sexual- and gender-based violence. Sexual assault, rape, sexual harassment, and dating violence were among the most frequently defined terms. Additional forms of sexual -and gender-based violence listed by various schools included the following: fondling, statutory rape, incest, retaliation, and more. Several schools (14%) either did not list separate forms of violence (7%), or this information could not be found at all on their website (6%).

Each school’s prevention and response page listed multiple short- and long-term options for care (Table 2). The most frequent included counseling resources (n = 35), a crisis hotline (n = 34), 911 or other campus emergency number (n = 33) and Title IX reporting (n = 32). Less frequent options included housing accommodation and assistance, pregnancy testing, emergency contraception, campus escorts, peer support, legal consultations, and more. Most (86%) of schools displayed local community-based victim advocacy programs.

One school stated explicitly that they were in compliance with laws such as Title IX, the Clery Act, and the Violence Against Women Act (VAWA) but did not define what that meant or include information on how they were compliant. Very few schools (6%) had a step-by-step description of what it means to report to Title IX, with an emphasis on a victim’s choice in the process. Three schools provided resources for abusers/perpetrators/respondents for sexual and gender-based violence.

Inclusivity

Almost half (44%) of schools provided resources tailored for LGBTQ + students’ experiences of SV, 22% for trans/nonbinary students, 19% for BIPOC students, and 6% for international students (Table 3). Some schools had additional resources specific to diverse populations like trans women of color, Asian American/Pacific Islander (AAPI) individuals, Indigenous women, and employees.

Table 3 Sexual violence information and resources for inclusivity

Of the 30 schools whose nondiscrimination policies were accessible online, several schools did not enumerate sexual orientation (n = 2), gender identity (n = 4), gender expression (n = 9), race (n = 2), or disability (n = 2) on their website.

SV resources for students with disabilities were scant, with only five sites noting resources tailored for this group of students. When locating resources specific to students with multiple identities, only one school listed resources for trans women of color, AAPI, and Indigenous women to name a few. Additionally, one school provided guidance for pregnant students. Considering language access, only one school provided resources in various languages including Arabic, French, Korean, Mandarin, and Spanish.

Alcohol and SV connections

Starting from Google and searching for "[SCHOOL] alcohol", the first webpage for the campus on the Google Search list was selected as the school’s alcohol page. The accessibility of alcohol policy information as well as linkages to student health, student counseling, and Title IX varied widely (ranging from very easy to difficult). All 36 schools did not have a webpage dedicated solely to alcohol information but included “alcohol” as a topic on other webpages (Table 4). Half of schools (n = 18) listed alcohol information on their alcohol and drug policy page, while one school listed their alcohol information on their student conduct page. For the remaining 17 schools, information was retrieved from various webpages or directly from the student handbook.

Table 4 Alcohol and sexual violence connections

Each school’s main alcohol page linked to their respective alcohol policies for most (86%) schools, to Title IX website (61%), to student counseling (67%), and student health (42%). However, almost all (n = 35) of the alcohol policies and resources on college websites were not connected to SV-related information, and none linked alcohol-related information to the student counseling website. One school provided a thorough explanation on the negative impacts of alcohol within their policy and included how SV is increased when alcohol is involved.

Discussion

Researchers have conducted various website assessments including a similar digital checklist, [57] a comprehensive auditing tool, [59] and a toolkit [60] addressing SV resources and the role of alcohol on college campuses, intended primarily to guide administrators and prevention specialists on improving visibility of available resources. The website assessment tool described here builds on this prior work by focusing on the student’s experience navigating these websites to find resources after a sexual assault and campus policies regarding alcohol use. We also fill a critical gap in prior research by evaluating whether resources were comprehensible and inclusive of diverse populations. Using this tool, we found multiple challenges that students may face accessing information and resources for SV and alcohol on college websites.

We completed 36 college website assessments to ascertain whether their SV information and resources are accessible, comprehensible, and inclusive as well as how alcohol policies and resources are linked to SV-related information. Overall, we found that most school websites need significant improvements. Although the Clery Act is a useful starting point for making Title IX policies and resources more visible to college students, as evidenced with the ease of finding the Title IX page for each campus, more structural improvements to these websites are needed to help survivors on college campuses with identifying and receiving relevant information, services, and supports. All campuses need to increase visibility of resources tailored for students from diverse populations especially those who are trans/gender diverse, have disabilities, are international, or identify with multiple identities.

Although SV-related information was relatively easy to navigate to from a campus Title IX page (or found on that page), the information was difficult to comprehend, with resources and terms not always defined and often not inclusive of diverse populations, particularly students with disabilities. As information was often embedded in lengthy text and challenging to find, if students are not familiar with how to utilize browser-embedded tools, like searching for specific words or phrases, this could delay or prevent students from finding the right information for resources, care, and support. This poses many challenges and barriers for students when seeking options for both immediate and long-term care as some resources proved challenging to locate. Strategic improvements are needed to update information and resources, and navigability of, college websites.

As part of Title IX, many schools listed forms of sexual and gender-based violence but did not provide definitions. For example, definitions for intimate partner violence (IPV), sexual misconduct, and sexual exploitation were missing and can contribute to misunderstanding [54]. A clear distinction between terms is critical to reduce confusion among survivors when trying to make sense of what they have experienced and what resources to seek [54].

A third (30%) of schools in this sample did not provide a clear description or definitions for the differences between confidential and nonconfidential resources. If resources were listed, it was not always clear if these were confidential or non-confidential. This is one modifiable barrier for students and survivors to access resources, information, and care that aligns with their preferences, especially in a time of crisis.

Only one school listed alcohol-related policies and supports linked to SV-related information, and none linked alcohol-related information to the student counseling website. Alcohol was also not mentioned in SV resources. Given the prevalence of alcohol misuse and alcohol-related SV on college campuses as well as the literature on prior trauma exposure that increases the likelihood of heavy drinking, [65] linking alcohol use, mental health supports, and SV explicitly may be a critical aspect of SV prevention in addition to providing information for survivors [55]. Training about alcohol policies and prevention efforts would benefit from including how alcohol increases the risk of alcohol-related SV.

When developing online resources for SV and alcohol-related supports, colleges may consider (Table 5) ensuring that their website is (1) accessible, with resources like step-by-step instructions for accessing services and reporting; (2) comprehendible, with information that is clearly organized and developmentally appropriate; and (3) inclusive, with specific resources disproportionately impacted diverse populations and non-native English speakers.

Table 5 Recommendations for developing online resources for sexual violence and alcohol-related supports

The number of schools assessed limits the generalizability of findings, although the sample did include smaller and larger, public and private schools with geographic diversity. By operationalizing ease of finding information by combining one and two clicks, we may have missed a substantive difference between one and two clicks that could be a practical barrier for an undergraduate student (i.e., a student looking for information could give up after one click). Revising the assessment tool to measure actual number of clicks instead of grouping clicks together on a Likert-type scale might help to better distinguish how easy or difficult each website is to navigate. In addition, the number of clicks alone may not tell a complete story, as the information may be a click away, yet students still need to search through a lengthy document or use search features on the website to find information they are seeking.

Four students conducted and compiled the data from the Website Assessment Tool. Even with the quality checks, interpretation of results is subjective, and website navigation experiences may vary for students from diverse populations, different abilities, and those experiencing language barriers [54]. Campuses could consider inviting students from diverse backgrounds to use the tool to evaluate their campus website, including measuring the length of time it takes to find resources to optimize how such vital information is received by various student populations. Implementing these changes may help reduce student confusion and navigation challenges, increase uptake of relevant services by survivors, and increase safety for all students [55].

Scans for each school website were conducted using a computer or laptop and did not include the use of a smartphone which could be the preferred device for going online; we did not assess how legible webpages containing information, resources, and support were for mobile devices. For example, if a student is required to click on a link and then rerouted to search for information in their student handbook, they may not be able to find the information. This tool could be updated to include accessibility on smartphones as well as screen readers and other assistive technology. We also did not assess the accuracy of the information as a part of this assessment, and information listed may be incomplete, outdated, or contain misinformation about resources.

This Website Assessment Tool may be useful for college campuses seeking to optimize their students’ receipt of information and services related to SV and alcohol use. In the future, a standard approach or national guideline could be established to provide standardized information on college websites for SV and alcohol use prevention information [55].

Conclusions

As college students are likely to seek information on their campus website, this Website Assessment Tool may be a strategy to assist campuses in making information about SV and alcohol use accessible, comprehensible, and inclusive by enhancing design, content, and navigability of their websites. Our data highlights the absence of linkages between alcohol polices and resources linked to SV-related information.