ClinicalTrials.gg
CompletedNCT03896776Updated Oct 28, 2026Results posted

A Pragmatic Trial of Two Strategies for Implementing an Effective eHealth HIV Prevention Program (Keep It Up! 3.0)

An interventional study of Keep It Up! 3.0 in HIV/AIDS, Gonorrhea and Chlamydia, sponsored by Northwestern University. Completed at 1 site in United States. Open to male participants aged 18 Years to 34 Years, including healthy volunteers. Per ClinicalTrials.gov, last updated 2026-10-28.

Sponsored by Northwestern University · Not applicable, Interventional, and Prevention

How we know

Updated Oct 28, 2026Results postedStudy documents posted+2 moreGo to Updates ↓

Help build ClinicalTrials.gg — short surveys, remote interviews or testing, in your language. Take part →

Phase
Not applicable
Study type
Interventional
Enrollment
2,249
Allocation
Randomized
Ages
18 Years to 34 Years
Sex
Male
01

Where this study stands

Counted across the records on this site · 2026-11-22

Among 1,671 studies in Acquired Immunodeficiency Syndrome with a site in United States — 112 open to participants now, 154 under way, 9 of them phase 3. Acquired Immunodeficiency Syndrome · United States.

Enrollment
21st of 1,291
2,249 · median 60
Started
2019
with 76 others
No phase
642 of 1,671
59 open now
Registered
1,331st of 1,671
by date

Among the 1,671 studies in Acquired Immunodeficiency Syndrome with a site in United States, this one is the 21st largest by enrollment of the 1,291 interventional ones that give a count, is one of 77 that started in 2019, is one of 642 with no phase, and was the 1,331st registered.

Acquired Immunodeficiency Syndrome

3,039 studies on the registry are indexed under Acquired Immunodeficiency Syndrome; 308 are open to participants now.

Browse Acquired Immunodeficiency Syndrome studies →

Open to participants now

112 of the 1,671 studies in Acquired Immunodeficiency Syndrome with a site in United States are open now. Recruiting first, the newest first:

107 more — sign in →

All 112 open studies →

A study can be open while its sites in United States have not opened yet or have stopped. Only those recruiting at a site in United States →

Who these studies take

Of the 197 Acquired Immunodeficiency Syndrome studies recruiting now, 117 (59%) set no upper age limit and 67 (34%) stop at 75 or younger; 43 take children under 18; 68 take healthy volunteers.

This study: 18 Years to 34 Years · male · healthy volunteers too.

The full table on the hub →

This tab as a report to print or save as PDF → With an account.

Was this useful?
Get alerts for this landscapeDownload CSVNarrow it in search →With an account: a weekly mail of what changed among 1,671 studies in Acquired Immunodeficiency Syndrome with a site in United States, or all of them as a file.
02

Study summary

This study evaluates and compares two implementation strategies of an online HIV prevention intervention: Strategy 1 in which community-based organizations apply, and are selected, for funding to deliver Keep It Up! through current HIV testing programs; and Strategy 2 which is a "direct-to-consumer" model where centralized staff at Northwestern University recruit participants nationally through online advertising campaigns and manage engagement.

Read the detailed description

Substantial effort has gone into developing and testing the efficacy of eHealth HIV prevention interventions, but these interventions will not affect the spread of the epidemic unless implemented effectively to reach the most at-risk populations. Despite emerging evidence that eHealth approaches can be highly effective in reducing HIV risk or improving care engagement, implementation requirements do not fit within established strategies for public health scale up, leaving it an open question as to how to bring these programs into practice. To date, little to no implementation science has examined strategies to effectively scale up eHealth HIV prevention programs.

The overarching goal of this study is to address this need by developing and evaluating novel strategies for implementing eHealth HIV prevention programs. Keep It Up! (KIU!) is an online program proven to reduce HIV risk among diverse young men who have sex with men (YMSM). KIU! was designed for YMSM who just tested HIV negative to keep it up and maintain negative status by reducing risk and enacting protections. Strong evidence of effectiveness on behavioral (decreased condomless sex) and biomedical outcomes (reduced STI incidence) makes KIU! an ideal eHealth intervention for implementation research. Lessons learned from KIU! implementation will set the pathway for implementing the many HIV eHealth programs currently undergoing efficacy testing.

Implementation strategies need to be evaluated using rigorous research designs. Through formative research with community-based organizations (CBOs), health departments, the CDC, and researchers, the investigators have identified two pragmatic implementation strategies that will be evaluated and compared:

In Strategy 1, CBOs apply for, and are selected for, funding to deliver KIU! through current HIV testing programs. Because CBOs do not have the capacity to host the technology, KIU! is hosted centrally at Northwestern and deployed by local CBO staff. Training and coaching of CBO staff on how to use the technology and integrate it into routine HIV testing is provided through capacity building assistance.

Strategy 2 is a "direct-to-consumer" (DTC) model where centralized staff at Northwestern primarily recruit participants nationally through online advertising campaigns and manage participants' engagement. HIV/STI testing kits are shipped directly to YMSM, and upon documentation of an HIV negative test result, participants are delivered KIU! This DTC model has been widely adopted in technology dissemination but it is innovative in terms of implementation of HIV prevention programs.

The investigators will accomplish the goal to inform eHealth intervention implementation with two specific aims:

Aim 1: Compare two implementation strategies using a quasi-experimental trial.

The investigators selected a type III hybrid implementation-effectiveness design, which prioritizes empirical comparison of implementation strategies while also collecting evidence of effectiveness. KIU will be delivered using Strategy 1 (CBOs) in 22 counties for two years; simultaneously, the investigators will deliver KIU nationally using Strategy 2 (DTC). YMSM participants in the DTC strategy will be matched to CBO strategy participants based on post-hoc propensity scores. Implementation and effectiveness outcome data will be collected from YMSM participants, CBOs, and the technology vendor. The primary outcomes are public health impact (defined as reach X effectiveness) and cost per infection averted, a key component of CDC decision making about which HIV prevention programs will be supported. There continues to be equipoise for the trial (i.e., either strategy could be superior). Because implementation success can be defined differently for different stakeholders, the investigators will also continue to collect and report as secondary outcomes multiple metrics of reach, effectiveness, and implementation.

Aim 2: Examine adoption characteristics that explain variability in implementation outcomes.

There is remarkable variability in implementation success when CBOs adopt evidence-based interventions. The investigators will seek to explain variability in implementation success across counties by conducting mixed-methods research on the domains from the consolidated framework for implementation research (CFIR), such as county characteristics, adaptations, support from organization leadership, and approach to planning adoption. Data on CFIR characteristics will be collected through administrative data, surveys, and telephone interviews with key stakeholders.

In addition to these 2 specific aims, the investigators will explore sustainment of KIU! at the completion of the study. CBOs will be provided with materials to facilitate applying for external funding to continue to provide KIU! after study completion, including an Impact Tool to estimate local impact and costs. The investigators will also examine factors that predict applying for funding and ongoing sustainment. For Strategy 2, through consultation with the CDC, capacity building assistance providers, health departments, and small businesses involved in Health 2.0 initiatives, the investigators will examine models for ongoing sustainment of the DTC model.

03

Conditions studied

04

Who can participate

Ages eligible
18 Years to 34 Years
Sexes eligible
Male
Accepts healthy volunteers
Yes

Go through the criteria one by one — yes, no or not sure — and take the list to your doctor or the study team. Kept in this browser only.

Inclusion criteria

  • recently received an negative test result OR self-report negative or unknown HIV status
  • speaks/reads English
  • has an active email address

Exclusion criteria

Exclusion Criteria:

  • has not been on PrEP and adherent for 6 months prior
Terms in these criteria, in plain language(1)

Underlined terms in the criteria open these too. Explained by clinicaltrials.gg, not part of the registry record. The study team decides who can take part.

HIV

The human immunodeficiency virus, which weakens the immune system.

Many studies now accept people whose HIV is controlled on treatment, judged by the viral load and the CD4 count. Check the limits this study sets.

05

Study design

Phase
Not applicable
Primary purpose
Prevention
Allocation
Randomized
Intervention model
Parallel assignment
Masking
None (open label)
Enrollment
2,249 participants (actual)

Study arms

  • Experimental
    Community Based Organization (CBO) Delivery

    CBOs will be selected through a Request for Proposal (RFP) process. The RFP will describe research and KIU! delivery activities to be conducted as part of the study and allowable budget to carry out the activities. The RFP will ask CBOs to describe past experience providing HIV services for YMSM. Selected CBOs will recruit participants into the intervention and encourage participants to complete each session of intervention content at baseline, 6 Week Follow-up, and 12 Week Follow-up. Participants in the CBO delivery arm will receive baseline HIV and STI testing at a CBO. Participants will receive an at-home STI test kit at 12 Week Follow-up if testing is not provided at their CBO site.

    Behavioral: Keep It Up! 3.0

  • Experimental
    Direct to Consumer (DTC) Delivery

    In the DTC arm, participants will be recruited online via paid social media advertising (e.g., Facebook, Instagram). Advertisements will target placements by age, gender, sexual orientation, racial background, "likes" that are relevant to YMSM (e.g., local LGBT organizations, "out" celebrities), and location (i.e., target county). Dating/sex-seeking apps (e.g., Grindr) will also be used to recruit YMSM, using a similar advertising approach as social media. These online recruitment strategies will be supplemented by referrals from local organizations and participant registries, and snowball recruitment. Study staff at Northwestern University will manage this recruitment process and encourage participants to complete each session of intervention content at baseline, 6 Week Follow-up, and 12 Week Follow-up. Participants in the DTC delivery arm will receive at-home HIV and STI test kits at baseline. Participants will receive an at-home STI test kit at 12 Week Follow-up.

    Behavioral: Keep It Up! 3.0

Interventions

  • BehavioralKeep It Up! 3.0

    KIU! is an online HIV prevention intervention developed for high-risk young men who have sex with men (YMSM) who recently tested HIV negative. Content was developed in collaboration with YMSM-serving CBOs and subjected to usability testing with diverse YMSM. The Information-Motivation-Behavioral Skills model and eLearning principles guided development of highly interactive, engaging, and culturally relevant health messages. KIU! involves 7 modules completed across 3 sessions, totaling \~1 hour of main content, plus 2 booster sessions at 6- and 12-week follow-ups. Each module is based on a setting or situation relevant to YMSM (e.g., connecting to the gay community and meeting guys through apps), with developmentally appropriate behavior change content embedded. KIU! uses diverse delivery methods (e.g. videos, animation, games) to address HIV knowledge gaps, motivate safer behaviors, teach behavioral skills, and instill self-efficacy for preventive behaviors.

    Also known as: KIU! 3.0

06

What researchers measure

Primary outcomes

  1. Public Health Impact

    Public Health Impact (PHI) is defined as the number of HIV infections averted. HIV infections averted were calculated as the difference between baseline and 12-week extrapolated HIV incidence rates (IRs) per 100 person-years. Given the incidence of HIV among YMSM in the US, very large sample sizes would be required to directly estimate HIV infections averted, so HIV IRs were extrapolated for each study arm-timepoint using a model predictive of HIV incidence published by Mulick and Murray (2020) that uses rectal gonorrhea infection as a determinant.

    Time frame: 12 Weeks

07

Results

Posted Oct 28, 2026

Participant flow

Originally 66 counties with largest YMSM populations were randomized 2:1 into CBO \& DTC arms. 22 CBO counties selected from the 44 based on most competitive CBO applications. DTC arm revised to recruit nationally but could not recruit from CBO counties. No DTC participants assigned at local level, only nationally. Outcomes reported at participant level for more meaningful interpretations of study effects. Primary study outcome, public health impact, is wholly reliant on YMSM participant data.

Participant flow — Overall Study
MilestoneCommunity Based Organization (CBO) DeliveryDirect to Consumer (DTC) Delivery
Started7811468
Community based organization (cbo) staff survey1250
Community based organization (cbo) ymsm participants6560
Completed3541158
Not completed427310

Outcome measures

PrimaryPublic Health Impact

Public Health Impact (PHI) is defined as the number of HIV infections averted. HIV infections averted were calculated as the difference between baseline and 12-week extrapolated HIV incidence rates (IRs) per 100 person-years. Given the incidence of HIV among YMSM in the US, very large sample sizes would be required to directly estimate HIV infections averted, so HIV IRs were extrapolated for each study arm-timepoint using a model predictive of HIV incidence published by Mulick and Murray (2020) that uses rectal gonorrhea infection as a determinant.

Time frame:
12 Weeks
Reported as:
Number · HIV infections averted per 100 person-yr
Public Health Impact
HIV infections averted per 100 person-yrCommunity Based Organization (CBO) DeliveryDirect to Consumer (DTC) Delivery
Public Health Impact1.65 (1.29 to 2.02)0.62 (0.49 to 0.74)

Adverse events

Collected over 12 weeks. Non-serious events are listed at a 0% frequency threshold.

Adverse event summary by group
GroupDeathsSeriousOther
Community Based Organization (CBO) Delivery0/656 (0%)0/656 (0%)0/656 (0%)
Direct to Consumer (DTC) Delivery0/1,468 (0%)0/1,468 (0%)0/1,468 (0%)
Community Based Organization (CBO) Staff0/125 (0%)0/125 (0%)0/125 (0%)

Baseline characteristics

Age, Categorical
Age, Categorical(Participants)Community Based Organization (CBO) DeliveryDirect to Consumer (DTC) DeliveryCommunity Based Organization (CBO) StaffTotal
<=18 years0000
Between 18 and 65 years65614681252249
>=65 years0000
Age, Continuous
Age, Continuous(years)Community Based Organization (CBO) DeliveryDirect to Consumer (DTC) DeliveryCommunity Based Organization (CBO) StaffTotal
Mean26.34 ± 4.0826.04 ± 4.5132.71 ± 9.3326.13 ± 4.38
Sex: Female, Male
Sex: Female, Male(Participants)Community Based Organization (CBO) DeliveryDirect to Consumer (DTC) DeliveryCommunity Based Organization (CBO) StaffTotal
Female003535
Male6561468902214
Race (NIH/OMB)
Race (NIH/OMB)(Participants)Community Based Organization (CBO) DeliveryDirect to Consumer (DTC) DeliveryCommunity Based Organization (CBO) StaffTotal
American Indian or Alaska Native1140455
Asian501664220
Native Hawaiian or Other Pacific Islander511117
Black or African American17319944416
White322830571209
More than one race4113011182
Unknown or Not Reported54924150
Ethnicity (NIH/OMB)
Ethnicity (NIH/OMB)(Participants)Community Based Organization (CBO) DeliveryDirect to Consumer (DTC) DeliveryCommunity Based Organization (CBO) StaffTotal
Hispanic or Latino21640137654
Not Hispanic or Latino4401067881595
Unknown or Not Reported0000
Region of Enrollment
Region of Enrollment(participants)Community Based Organization (CBO) DeliveryDirect to Consumer (DTC) DeliveryCommunity Based Organization (CBO) StaffTotal
United States65614681252249
08

Study locations

1 site
  • Northwestern University
    Chicago, Illinois 60611, United States
09

References and documents

Publications

  • Zamantakis A, Zapata JP, Smith JD, Danielson EC, Macapagal K, Saber R, Li D, Benbow N, Mustanski B. Incentivizing Prevention: Community Based Organizations' Perceptions on Financial Incentives for Recruitment and Retention of Participants in Keep it Up! (KIU!), an eHealth HIV Prevention Intervention. AIDS Behav. 2026 Apr;30(4):1239-1251. doi: 10.1007/s10461-025-04963-w. Epub 2025 Nov 21. PubMed 41266922 ↗
  • Zamantakis A, Danielson E, Rudd E, Zapata JP, Benbow N, Saber R, Knapp AA, Mustanski B. Community-Based Organizations' Approaches to Recruitment and Retention for a Digital HIV Prevention Intervention for Young Men Who Have Sex With Men: A Mixed Methods Study. J Med Internet Res. 2025 Nov 3;27:e63199. doi: 10.2196/63199. PubMed 41183372 ↗
  • Mustanski B, Benbow N, Macapagal K, Li D, Madkins K, Saber R, Linas B, Smith JD, Brown CH, Munroe S, Reddy S, Schackman BR, Swann G, Janulis P, Zamantakis A, Zapata JP. Comparing Implementation and Effectiveness Outcomes for Two Implementation Strategies of the Keep It Up! Digital HIV Prevention Program: A Type 3 Hybrid Effectiveness-Implementation Trial. AIDS Behav. 2025 Dec;29(12):4030-4042. doi: 10.1007/s10461-025-04838-0. Epub 2025 Aug 19. PubMed 40826259 ↗
  • Zamantakis A, Zapata JP, Greenawalt I, Knapp AA, Benbow N, Mustanski B. Barriers and Facilitators to Implementing Keep It Up!, A Digital Health Intervention, in Community-Based Organizations. AIDS Behav. 2024 Dec;28(12):3944-3955. doi: 10.1007/s10461-024-04525-6. Epub 2024 Oct 10. PubMed 39387999 ↗
  • Li DH, Zamantakis A, Zapata JP, Danielson EC, Saber R, Benbow N, Smith JD, Swann G, Macapagal K, Mustanski B. A mixed-methods approach to assessing implementers' readiness to adopt digital health interventions (RADHI). Implement Sci Commun. 2024 Aug 27;5(1):91. doi: 10.1186/s43058-024-00628-2. PubMed 39192345 ↗

Study documents

PDFs hosted by the registry, ClinicalTrials.gov.

Individual participant data

Plan to share: Yes — The PI agrees to develop a de-identified database, codebook, and mechanism by which IPD could be shared with other investigators upon approval of the PI. Data will be available for request approximately 6 months after completion of the project. Interested investigators will be asked to complete a standardized request form stating the specific aims of the analysis, the analytic plans, resources the requestors have to carry out the project, the proposed timeline, and distribution goals (manuscripts and/or grant application). The PI will review these requests to determine whether the proposed analyses constitute an innovative and significant exploration of the data, whether the proposed team has sufficient resources to undertake the request, how data will be protected/managed, and whether there are sufficient resources to honor the request. If any of these issues are problematic, the PI will attempt to negotiate a fair resolution with the interested parties and/or with NIH program staff.

10

Timeline

1 registry update since Sep 25, 2026
ElapsedEstimatedEstimated, not yet reachedSimilar studies: half by the dot; the band is the middle halfHalf of similar studies reached it by the dot; the band is the middle half, from a quarter to three quarters

Ran from Oct 2019 to primary completion in Mar 2023, 41 months.

Results were first posted in Oct 2026, 3 years 6 months after primary completion.

Of 642 non-industry interventional studies with no phase under Acquired Immunodeficiency Syndrome, half reached primary completion within 29 months of starting (the median), counting those still running; a quarter within 17 and three quarters within 45. This study's 41 months is above that.

Counting every sponsor, half of 648 studies reached it within 29 months; a quarter within 17 and three quarters within 45.

How these figures are worked out, and which studies are left out of them.

Registered
Mar 28, 2019
First posted
Apr 1, 2019
StartActual
Oct 9, 2019
Primary completionActual
Mar 31, 2023
CompletionActual
Mar 31, 2023
Results first posted
Oct 28, 2026
Last update
Oct 28, 2026
What these dates mean
Start.
When the first participant enrolled; while estimated, when the study expects to open to recruitment.
Primary completion.
When the last participant was examined or treated to collect data for the study's main question, its primary outcome. A study that stopped early gives the date its data collection stopped.
Completion.
When the last participant's last visit took place, for every outcome, secondary ones and side effects included: the end of the study. Often later than primary completion, sometimes the same day.
Actual and Est.
Actual means the registry says the date has happened. Est. (estimated) is a date the sponsor plans or expects. It can still move, and the sponsor is expected to change it to actual once it has passed.
Results
Results posted
posted Oct 28, 2026
Enrollment
2125→2249
Oct 28, 2026
Primary outcomes
1 edited
What changed in the outcomes
  • EditedPublic Health Impact (reach X effectiveness)Time frame: 12 Weeks
Oct 28, 2026
Publications
5 publications citing this study linked from PubMed, e.g. PubMed 41266922 ↗
Oct 28, 2026
Show all 1 update
  1. Oct 28, 2026
    Results posted
    Publications 5 publications citing this study linked from PubMed, e.g. PubMed 41266922 ↗
    Enrollment 2125→2249
    Primary outcomes 1 edited
    What changed in the outcomes
    • EditedPublic Health Impact (reach X effectiveness)Time frame: 12 Weeks
    + 2 other changes: verification date and index terms

From the registry record's own update history. This site started tracking changes on Sep 25, 2026; for anything earlier, see the record history on ClinicalTrials.gov ↗

11

Registry details

Key details

Study ID
NCT03896776
Responsible party
Brian Mustanski (Director, Institute for Sexual and Gender Minority Health and Wellbeing, Northwestern University) — Principal investigator
First posted
Apr 1, 2019
Start date
Oct 9, 2019
Primary completion
Mar 31, 2023
Completion
Mar 31, 2023
Results posted
Oct 28, 2026
Last update
Oct 28, 2026

Study contacts

Brian Mustanski, PhD
principal investigator · Northwestern University

Oversight

Data monitoring committee
Yes
FDA-regulated drug
No
FDA-regulated device
No
View the source record on ClinicalTrials.gov ↗

Not currently enrolling

This study is completed, as verified in Jun 2026. You cannot join it, but the record below documents what was studied.

Follow this study

Get an email when the registry record changes — status, dates, results — or when someone posts here.

Sign in to follow

Discussion

Questions and observations about this study, from anyone following it. Not medical advice, and not a channel to the study team — their contact details are on the registry record.

Sign in to join the discussion. Reading takes no account; posting does. You choose a display name, and a pseudonym is the default.

Nothing here yet. If you are running this trial, taking part in it, or weighing whether to, this is the place to say so.

Start the discussion