Introduction

The domestic US plan to accelerate SARS-CoV-2 vaccine development aimed to advance multiple stages of clinical trials concurrently. The US government-funded COVID-19 Prevention Network (CoVPN) partnered with the White House and the Department of Health and Human Services (HHS) to evaluate at least two vaccine candidates from each of the three most promising vaccine platforms in tandem1,2. In addition to aligning and synchronizing clinical trial activities, CoVPN worked to coordinate activities and harmonize data and resources across clinical trials to minimize competitive practices and to increase the value of clinical and immunologic data obtained from these clinical trials. Here, we describe one such coordinated resource: a centralized clinical trial volunteer screening registry (VSR) used to support enrollment in CoVPN clinical trials.

Historically, large clinical trial networks, individual recruiting sites, and clinical research organizations (CROs) have developed and maintained independent, localized volunteer registries. With a national effort spanning several concurrent clinical trials involving over 100,000 participants, a centralized registry was envisioned to prevent redundancy and competition for volunteers. Thus, in May of 2020, we started the development and promotion of a national centralized registry.

The CoVPN VSR was designed to support multi-pronged and concurrent clinical trials, to meet requirements of private vaccine developers, and to interface with public/private clinical research networks. Additional constraints were imposed by the rapid advancement and scaling of vaccine clinical trials in the first months of the COVID-19 pandemic. Furthermore, rapidly evolving scientific evidence influenced the scope and content of screening questions, user queries, and risk estimates needed to support forthcoming clinical trials.

Existing software platforms were initially considered to support a volunteer registry. Several platforms, including customer engagement, electronic data capture (EDC), and clinical trial management systems (CTMS) were evaluated for their capacity to meet functional requirements such as connecting volunteers with clinical trial recruiters and screeners3,4. Candidate systems such as REDCap5,6 were customizable, enabled web-based intake, and facilitated volunteer engagement. However, these systems were not designed to scale to handle millions of records and hundreds of simultaneous users. Other commercial software options included patient engagement and recruitment functions but did not meet needs of CoVPN to equitably distribute volunteers, incorporate risk scoring, or perform needed oversight functions. Building a system under an accelerated timeline allowed timely implementation of CoVPN-specific requirements including utilization metrics, scoring functions, and collaboration functions while also scaling to support national and international clinical trials. An existing COVID-19 donation agreement from Oracle Corporation (Oracle) to support HHS-related pandemic efforts enabled us to efficiently collaborate and leverage Oracle technology and development resources to create and deploy the system rapidly.

Results

The implementation of the CoVPN VSR system consisted of distinct user interfaces: the Volunteer Questionnaire; the clinical trial Site Portal for screening volunteers; and an Administrative Portal for oversight of volunteer records, analytics, and site monitoring. The latter two utilized role-based access. An overview of the flow of information from the Volunteer Questionnaire to the VSR database and the two portals is shown in Fig. 1.

Fig. 1: CoVPN VSR Overview flow chart.
figure 1

This graphic shows how prospective volunteers submit information through the intake survey and are subsequently viewable through the Administrative Portal and the Site Portal for recruitment and enrollment into clinical trials.

Full size image

The Volunteer Questionnaire was designed and tested in May-June of 2020 to support varied, projected needs of clinical trials that might hinge on volunteer risk factors, exposures, symptoms, as well as evolving inclusion/exclusion criteria of clinical trials still under active development. This was a challenging task due to the unclear etiology of COVID-19 disease, the contributions of intrinsic versus behavioral risk factors, and the unfolding requirements of each clinical trial. The final components of the questionnaire included Contact info, Demographics, Occupation, Residence, Community interactions, COVID-19 risks and testing, and Symptoms and health risks. These records supported volunteer screening and prioritization for multiple clinical trials (the full survey/protocol is provided in Supplementary Materials).

The Site Portal was developed as an interface for clinical trial site staff and principal investigators to characterize, query, and access volunteer records for the purpose of volunteer contact. A total of 1094 site staff and principal investigators were trained to use the system. The Site Portal, accessible by approved staff and CoVPN administrators, provided six ways of interacting with volunteer records:

  • Query volunteers - Selection of tailored subsets of volunteer records

  • Manage volunteers - In-browser tools to set volunteer record statuses or return records

  • Upload volunteer results - De-identified record status updates via file upload

  • Site demographics - Descriptive statistics of volunteers within a site’s catchment area

  • Site map - Interactive map and selection tools to identify volunteers within target areas

  • Enrolled volunteers - Overview of volunteers marked as enrolled by a site

The Administrative Portal for CoVPN administrators utilized an activity dashboard and provided standardized reports that facilitated survey response monitoring, site administration, and risk algorithm administration (Fig. 2). Several reports were customized to provide survey statistics and site-specific and trial-specific monitoring. The system also supported all functions from the Site Portal at the site level, as well as the ability to create ad hoc queries, reports, and data visualizations. Access to full survey responses was further restricted based on user role.

Fig. 2: Administrative portal.
figure 2

A variety of live reports from the database supported periodic monitoring of volunteering, site utilization, and enrollment according to time, geography, or clinical trial.

Full size image

The registry was launched to the public on July 8, 2020, seven weeks after the announcement of federal vaccine development plans7. With 657,750 volunteers in total, the VSR was highly effective; 50% of these volunteers signed up within the first 36 days. Volunteer registrations also increased in concert with public service announcements during August and September (see Fig. 3). Participation was also increased using site-specific registration codes, which enabled recruiters to utilize the VSR in concert with volunteer campaigns and events.

Fig. 3: Volunteer registrations over time and geography.
figure 3

a Over 300,000 volunteers registered in the system within the first month. b The map shows the volunteering rate per 1000 adults in each state (mean = 2 × 23/1000 US adults).

Full size image

The median time to complete a survey was 5 min. The completion rate of the VSR questionnaire was 95.3%. Among completed surveys, 79.3% of respondents were White, 7% Hispanic or Latino, 4.4% Asian or Pacific Islander, 3.2% Black or African American, 0.3% Native American, 4.6% Mixed and 1.1% Other race/ethnicity. Additional demographic details and volunteer rates over time are shared in Supplementary Figs. 1–3 and Supplementary Table 1. Furthermore, 8557 Spanish surveys were submitted (1.3% of total surveys submitted). No clear trend was observed in survey completion rates by zip code population density or urban/rural classification (see Supplementary Table 2 and Supplementary Fig. 4).

The Site Portal was subsequently launched on August 4, 2020. In the US, 455 unique clinical research sites accessed a total of 168,015 volunteer records for recruitment and enrollment into six phase 3 COVID-19 vaccine clinical trials. These volunteers were drawn from catchment areas that included 22,201 US zip codes. Supplementary Table 3 shows the number of sites involved per clinical trial.

The demographic details captured in the VSR illustrate the national effort of the CoVPN and recruitment strategies of the various clinical trials. As shown in Fig. 4 and Supplementary Table 4, the registry enabled significant enrichment of enrollment of diverse respondents. Although socioeconomic status was not directly elicited by the survey, volunteers having either occupational or behavioral/household risks based on survey responses were enrolled at 3.5× and 2.5× higher rates, respectively. Overall, approximately 20% of the Moderna Phase 3 study population were Black, Indigenous and People of Color (BIPOC)8, whereas 55% of those enrolled at CoVPN sites 9, and approximately 80% of those enrolled from the VSR were from BIPOC communities.

Fig. 4: Enrollment of registry volunteers, by clinical trial and volunteer race/ethnicity.
figure 4

The VSR enabled a high proportion of study participant enrollment from BIPOC communities, with approximately 80% and 60% from the VSR into Moderna and AstraZeneca, respectively.

Full size image

The rapid development and deployment of the system limited intensive user-centered design or system evaluations. However, the VSR was publicly credited with advancing the clinical trials. Although the emergency circumstances prevented a formal evaluation, we detail below steps taken to respond to volunteer and user system requirements.

The VSR survey and Site Portal both allowed submission of comments to the development team. Volunteer feedback on the survey was reviewed weekly to identify any logical errors, omissions, usability concerns, or confusing questions. For a summary of user feedback, see Supplementary Table 5. Site Portal users also requested some functions including support for site- and campaign-specific recruitment codes for volunteers, additional query filters, and the ability to save and retrieve query results. These features were incorporated into updated versions of the system. Thrice-weekly development team meetings and task management software were used to prioritize any bugs or feature requests.

Stage 2 and 3 clinical trials for each vaccine launched on different dates. Therefore, some clinical trials were able to access more diverse and recent volunteers than other clinical trials.

Discussion

Due to COVID-19 lockdowns and limitations on in-person collaboration, clinical and research teams responding to the pandemic faced unprecedented challenges. In our case, the CoVPN VSR development and administrative teams assembled and collaborated remotely during every phase of the project, encompassing system design, development, documentation, training, technical support, and evaluation. In addition to remote collaboration tools such as Zoom and Microsoft Teams, the team utilized telephone, email, Slack, and web-based office productivity, scheduling, project management, and survey applications to support information sharing and coordination. Team members’ experience with these platforms varied according to institutional licensing for specific technologies, which required compromise as the team formed new work processes.

The network of clinical trial sites engaged for each study was selected through a collaborative process that involved stakeholders from HHS, vaccine developers, CoVPN, and CROs. Because each trial included sites from pre-existing vaccine trial networks as well as newly appointed CRO sites, it was necessary to position the VSR for accessibility and generality instead of specializing for a more uniform user base or the needs of a single developer. To achieve adoption, dozens of remote training sessions were held with hundreds of recruiters and investigators from different clinical trial sites. Individual sites often employed specialized recruitment procedures or utilized additional digital platforms mandated by vaccine developers or CROs. Future efforts to coordinate clinical trial networks may benefit from establishing plans for secure, standards-based, bidirectional information sharing akin to electronic medical record exchange.

The CoVPN VSR quickly accrued volunteers upon its introduction, reaching over 300,000 within the first month. However, despite ongoing advertising, volunteering rates slowed after the initial few months of availability. Vaccine misinformation and fears were already circulating during this time before the clinical trials were even complete. It is possible that this misinformation affected volunteering rates as well as subsequent recruitment efforts. To combat these efforts, the CoVPN website included information about vaccine science, individual clinical trials, the study process, and frequently asked questions in both English and Spanish.

One chief concern raised from VSR data was the inclusion of potentially sensitive identifying information and health conditions among survey responses. Such data were required to identify volunteers potentially at risk of severe COVID-19 outcomes, to ensure participants could be reached, and to ensure equitable recruitment. The need to collect and store this data might have been obviated by a national health identifier or linkage to electronic health records. Having collected this information, our user interface, training, and data use policy were crafted to promote retention of such data within the VSR system. Collection, use, and management of the data were approved through an Institutional Review Board.

Several alternative COVID-19 clinical trial volunteer registries were developed in the initial months of the pandemic by both commercial and non-profit entities10,11,12,13. For example, some registries leveraged electronic health records to identify patients for COVID-19 therapeutic clinical trials, following the clinical cohort-driven model adopted by Clinical and Translational Science Award sites through the Accrual to Clinical Trials initiative (CTSA ACT)14,15. This recruitment model was not appropriate for prospective vaccine clinical trials that required healthy volunteers.

Other approaches included vaccine developer-specific, site-specific, region-specific13,16,17, or comorbidity-specific12,18,19,20 volunteer registries. Though beneficial for specific recruiting organizations, this approach had the drawback of locking volunteers into one or a small subset of clinical trials for which they might be eligible. Two NIH initiatives provide prior examples of attempts to recruit a broader population of healthy volunteers. ResearchMatch.org is a general volunteer registry to connect eligible volunteers with clinical trials21. In its 12-year history, this CTSA project has accrued 166,289 volunteers. Another NIH program, the Clinical Research Volunteer Program, is designed to support local clinical trials run by the NIH22.

The rapid design and implementation of the CoVPN VSR was facilitated by Oracle Application Express (APEX, a low-code application development platform), application programming interfaces, web-based visualization tools, and the Oracle Database (all hosted in Oracle’s Government Cloud). Use of an Entity-Attribute-Value23 data definition for questionnaire intake data allowed for mid-flight schema updates which could be quickly passed through to updated user interfaces. This prevented the need for major architecture changes as new COVID-19 risk data and clinical trial priorities became available in the last months of 2020. Web-based project management, video conferencing, chat, file-sharing, and email facilitated rapid collaboration and establishment of new cross-organization teams.

Although the full impact of the registry on final clinical trial enrollment is unknown, based on the recruiting sites that consistently reported enrollment status, we estimate that up to 20% of enrollment was accounted for by VSR usage in some clinical trials. Given the difficulties commercial CROs had in recruiting diverse participants, adoption of the VSR unquestionably accelerated accrual into clinical trials and increased participation of more diverse populations (see Supplementary Figs. 1 and 2)9,24,25.

In addition to expediting enrollment, the VSR thus served as a crucial recruiting platform to ensure representation of diverse populations in study findings. Moreover, it improved recruitment efficiency by enabling sites to reach volunteers who met preliminary eligibility criteria. Though initially a strictly US-based system, we were able to quickly clone the VSR for a separate application in Peru to support clinical trials in that country, demonstrating its scalability and potential for ongoing trials. The system was later adapted to create the Red Ribbon registry for HIV clinical trials, and several aspects of the VSR can inform design of more general systems for ongoing and emergent use cases.

By achieving an unprecedented geographic coverage in record time, the VSR also mitigated the risk of alternative epidemic scenarios or unfavorable clinical trial outcomes, which could have required highly targeted or continuous enrollment.

Developing and deploying a national volunteer registry for clinical trials during an emerging pandemic was extremely challenging, and this effort required extensive resources and a devoted team focused exclusively on this endeavor. Open communication and flexibility were key to rapidly organizing a functional development team and meeting the unique needs of these accelerated clinical trials. The use of specialized teams to focus on system requirements, architecture, rapid iterative design-and-build process, administration, community engagement, and training allowed the efficient division of labor.

In an emergency situation, competing priorities among stakeholders require compromises in the design and implementation of digital health systems. During implementation of the VSR, we faced challenges like a lack of interoperability with clinical trial management systems (CTMS) and localized registries, the inability to require recruiters to track or report volunteer enrollment/status, and the lack of scalability of existing research systems such as RedCap.

Public health experts have widely discussed the need for increased surveillance capacity, sustained investments in the public health infrastructure, and investments in data standardization and the health care system to better prepare and respond to pandemics. Advance preparation can mitigate conflict or competition between stakeholder needs and maximize the benefit of common resources such as the VSR. To support this, such investments must encompass the infrastructure for clinical trials and research IT. Additional facilitators of efficient and coordinated clinical trials include:

  • Access to updated data standards, survey instruments, and other artifacts26

  • Surge capacity among clinical and IT experts

  • Harmonized record sharing across EMR, CTMS, laboratory, and other clinical sources

  • Flexible funding streams to support partnerships between domain experts and private enterprise

  • Mandates for multi-way sharing of information between infrastructure centers, researchers, government agencies, and vaccine developers

Through a public-private partnership, remote teamwork, and nimble development, we overcame barriers to collaboration imposed by the pandemic to connect hundreds of thousands of study volunteers to vaccine and antibody clinical trials. Emergency circumstances required balancing different priorities of the US government, sponsors, vaccine and antibody developers, research partners, and the public. Preparedness for future public health emergencies will benefit from continued investments in cross-institutional collaboration tools, data standardization, digital health data linkage, and platforms for public health communication.

Methods

The initial scope, design, and system requirements were determined during the initial wave of the pandemic through collaboration between the COVID-19 Prevention Network, HHS/NIH, Oracle, and the overall United States government (USG) operation bridging public and private sector expertise and infrastructure. We solicited additional feedback from vaccine developers, the CDC, and clinical trial sites. The effort encompassed a protocol (Supplementary Appendix A), Informed Consent Form (Supplementary Appendix B), design of a volunteer intake questionnaire (Supplementary Appendix C), database/user interface (UI) development, user accommodations, and development of risk and targeting algorithms. The protocol, questionnaire (English and Spanish versions), relevant skip patterns, and the consent form were reviewed for regulatory compliance and approved by Advarra IRB (Pro00044444). All participants provided informed consent.

The VSR system design requirements encompassed the need for data scale, concurrent users, multiple client interfaces, schema flexibility, adaptability, resilience, compliance, and security. A key requirement was the ability to efficiently store, query, and/retrieve millions of associated records to scale to hundreds of thousands of volunteers. Given concurrent recruitment and enrollment around the country, the system was also designed to respond to hundreds of simultaneous push/pull requests. (A summary of system requirements and functions is in Supplementary Table 6.)

Due to the speed of execution, the system was planned and developed in parallel with the volunteer questionnaire and several other aspects of the clinical trial network, even as the clinical trial protocols were still being finalized. To ensure adaptability to changes of the volunteer questionnaire and downstream database use cases, the underlying database architecture was designed to allow for customizable volunteer attributes and value sets.

Security and privacy of volunteer data was a key consideration in design and implementation of the system, while access to the system was required across diverse recruiting partners. CoVPN and Contract Research Organizations (CROs) clinical trial sites (selection previously described)2 engaged by vaccine developers were granted access contingent on 1) oversight by local principal investigators or administrators, 2) individual recruiter acceptance of a comprehensive data use agreement (DUA), and 3) completion of system training that included information security, cybersecurity, and DUA review, and 4) existence of local IRB and privacy/security controls. System access was controlled via two-factor authentication. All VSR user activity was automatically logged, providing an audit trail.

Given the purpose of the registry to facilitate volunteer outreach, deidentification was not consistent with the intended design. However, users were presented with a reduced set of volunteer demographics necessary for contact and pre-screening and risk scores based on survey responses that allowed targeting those likely to become endpoints without disclosing most survey details. Furthermore, users were provided with non-identifying unique IDs for each volunteer to enable completely anonymized status updates for each volunteer record.

To further protect volunteer privacy and minimize repeated contacts, access was granted only to volunteer records within the pre-defined catchment area of each site, and volunteers’ details were only shared after meeting the selection criteria of user queries. This limited visibility of non-targeted records to summary statistics. Only clinical trial sites and CROs contracted by vaccine developers accessed volunteer data; vaccine developers did not access volunteer data, however enrolled participants provided their data again for enrollment into specific clinical trials. The DUA required adhering to HIPAA, local laws/policies, and other local IRB controls. It further prohibited sharing of data, limited access to authorized personnel, required device security, limited data retention to the clinical trial period, and mandated sole use of novel unique identifiers for transmission of status updates to prevent unintentional exposure of volunteer data.

The application user interface designs for clinical trial sites and administrative support were created using rapid prototyping to support recruiters and CoVPN management, respectively. The primary clinical trial site UI consisted of six functions identified in consultation with HVTN and CoVPN leadership, investigators, site Principal Investigators (PIs), and informaticists. The landing page of the Site Portal was designed with simplicity in mind for ease of training, adoption, and use during recruitment (Fig. 5).

Fig. 5: Site portal.
figure 5

a Landing page showing six core functions; b Query volunteers function showing interactive filtering, descriptive statistics of volunteer demographics, and risk scores. c Interactive map of a site catchment area and volunteer density that enabled selection of volunteers from target regions.

Full size image

The underlying relational schema contained 26 tables to support the multi-trial, multi-user nature of the system. The table structure (Supplementary Fig. 5) supports various functions: Recording volunteer information (Surveys, Questions, Survey_responses, Question_values, Volunteers, Consent_forms, Risk_scores); Trial information (Trials, Trial_sites, Catchment_areas, Trial_site_zips, Site_users); Survey management (Checkout_sets, Status_codes); and System behavior (Languages, Risk_algorithms, Users, and other event logging). Survey questionnaire responses utilized an Entity-Attribute-Value model to support evolving questions/values from the survey. This approach enabled rapid and relatively seamless changes of the survey without impacting queries that supported system behavior.

Administrative functions evolved more progressively over time as additional management and analytics functions were identified. These included monitoring of VSR usage statistics by site, trial, and geographic area; rates and demographics of volunteer accrual; and ad hoc analytics and risk score development.

Several accommodations were made for users and volunteers. Training materials were developed to help users navigate through the system, understand approaches to screening volunteers, and to share features and requirements of the system. Dozens of virtual training sessions were held by the developer and administrative support teams.

To ensure equitable outreach to a broader population, Spanish language versions of the VSR recruitment website, the VSR questionnaire, and other study materials were provided. A helpline was also established to facilitate registry participation for the elderly or others who faced challenges accessing the registry website.

Accelerating these clinical trials required rapidly screening volunteers and enrolling hundreds of thousands of participants to meet different study criteria. The VSR questionnaire was designed by a panel of epidemiologists, clinical trial staff, investigators, and community engagement experts to help meet these needs. Registry questions were added to represent demographics and comprehensive clinical, environmental, and behavioral risk factors identified by epidemiologic studies. When possible, questionnaire items were aligned with validated survey instruments or controlled vocabularies, clinical trial enrollment criteria, and emerging literature. The questionnaire and UI were also pilot tested with community members and existing application review panels from different demographic groups to ensure survey comprehensibility and ease of completion. Feedback from these assessments were incorporated into the final questionnaire design.

To optimize the accrual of study endpoints, we calculated risk assessments for individual volunteers. These assessments included published COVID-19 risk models as well as additional clinical and behavioral risk flags and algorithms to estimate the level of exposure, symptomatic disease, hospitalization, intensive care admission, and death. Risk predictions were used to evaluate user queries.

An evidence review was conducted to identify common variables used as inputs for available risk models early in the pandemic. (See Wynants et al. for an earlier review of COVID-19 prediction models27.) When possible, these variables such as age, demographics, and other emerging risk factors were included in the volunteer survey.

At the time of initial deployment, given scant published evidence, the VSR supported analytics and volunteer filtering with the COVER-H, COVER-I, and COVER-F risk scores28 (estimating risk of hospitalization, ICU admission, and fatality respectively). Support for additional risk models was added during the subsequent four months of VSR deployment. As additional published research on COVID-19 risk accumulated, we synthesized this evidence into additional models to predict the risk of symptomatic disease and SARS-CoV-2 exposure.

The Site Portal provided tools to support volunteer record allotment, management, and status reporting. Signed data use and confidentiality agreements were required for system access, which was further safeguarded by two-factor authentication. A non-identifying volunteer key (distinct from other patient identifiers) was provided to clinical trial sites to enable secure status updates after volunteer contact. Each clinical trial site in the continental United States was limited to screening a fixed number of records within a local catchment area, adjusted for number of competing sites and enrollment targets of the trial. Volunteer records could be “checked out” to prevent competition for volunteers within zip codes targeted by the clinical trials. In September of 2020, queries of the VSR returned volunteer records in order of descending risk to facilitate accrual of study endpoints.

Once a site reached the record checkout limit, we required status code updates on previously accessed records (indicating enrollment, removal, or return to the registry) before new records could be accessed. The system supported a limited set of screening outcomes to simplify use of the system (Contact Not Attempted, Unable to Contact, Reconsider Later, Remove from Registry, Duplicate Record, Contacted/Not Enrolled, and Enrolled). The status codes are shown in Supplementary Table 7. Automated reminders were provided to users if the status of checked out records was not updated within two weeks.