Using behavioral outreach to counteract administrative burden and encourage take-up of simplified disability payment rules

olicymakers who are interested in facilitating use of public programs can find it challenging to recruit participants, but tailored outreach can help address some bottlenecks that inhibit take-up. Substantive challenges can arise from administrative burdens facing potential participants who must make complicated determinations about whether enrollment is in their best interests (Currie, 2006; Herd, DeLeire, Harvey, & Moynihan, 2013; Riphahn, 2003; Remler, Rachlin, & Glied, 2001). People may need to weigh the benefits of participating against the time and monetary costs of applying, the effort of learning about complex program rules, and any social stigma associated with participation. Other factors, such as procrastination and confusion, might further limit people’s capacity to “compare the expected costs and benefits” of participation (Bhargava & Manoli, 2015). Using behavioral insights to identify and address these burdens can lead to more effective outreach and streamlined application procedures, resulting in greater take-up of public programs (Herd et al., 2013; Richburg-Hayes, Anzelone, Dechausay, & Landers, 2017; Summer & Thompson, 2008; Wright, GarciaP V V Abstract: Take-up of employment programs among people with disabilities can be limited by the administrative burdens of decision-making, which must factor in the complexities of how work affects disability cash assistance payments. This study presents evidence on using outreach motivated by behavioral research to encourage enrollment in a pilot initiative with the Social Security Administration that simplified Social Security Disability Insurance payment rules. Because enrolling would leave some beneficiaries worse off, informed enrollment decisions required understanding both the complexities of current rules and potential effects of the new demonstration rules. We sought to counteract bottlenecks stemming from decision-making burdens through increased outreach with tailored messaging. A randomized controlled trial was used to test two features of a reminder postcard. First, we compared fold-over postcards containing information about the demonstration to open postcards with more generic information, finding that fold-over postcards increased enrollment by around 25 percent (or 0.12 percentage points). Second, we compared an urgent message framing with no stated enrollment end-date to a deadline framing with an explicit enrollment cutoff date. Although the final enrollment rate was similar across timeline framing options, the urgent framing appears to have resulted in faster enrollment.

1. Structure and language, using either a fold-over postcard containing specific information about POD or an open postcard containing more generic information. 1 2. Timeline framing, using either an urgent framing with an "act now" message and no stated end-date for enrollment or a deadline framing with a "time left" message and an explicit enrollment cutoff date.
These message design choices were informed by a review of the behavioral insights literature, as well as research on prior SSA employment programs-as discussed in the next section. Crossing the two variants of each component resulted in four different versions of the final reminder postcards. The randomized design allowed us to answer primary research questions about the relative effectiveness of each messaging component and variant. In addition, we developed a quasi-experimental test to explore a secondary research question on the likely average effects of being sent any final reminder postcard.
We found that the fold-over postcard structure with specific information about POD increased the enrollment rate compared to the open postcard structure with more generic language. Although the timeline framing of the postcard did not affect the final enrollment rate, the urgent message framing appears to have resulted in earlier enrollments compared to the deadline framing. In the rest of this study, we provide additional details about how the final reminder postcards fit into the broader context of POD recruitment efforts, the experimental methodology used to test the postcard's components, findings from the experiment, and a discussion of implications.

POD Recruitment Efforts
POD simplifies SSDI "work-incentive" rules and may allow some beneficiaries to keep more of their cash benefits while working, but others either do not stand to gain from the new POD rules or could retain more of their cash benefits under current SSDI rules. The POD outreach strategy sought to encourage beneficiaries to enroll after making an informed choice based on whether they might fare better under POD than current rules. After some initial refinements, the core recruitment outreach strategy consisted of a primary mailing of informational and enrollment materials and two postcards. We sought to enhance these efforts by adding a final reminder postcard.

Overview of POD
POD incorporates a new benefit offset formula for SSDI beneficiaries, as required by Congress under the Bipartisan Budget Act of 2015 (Public Law 114-74, Section 823). Existing SSDI work-incentive rules are complex and change depending on a beneficiary's earnings patterns over time (Appendix A). For example, beneficiaries continue to receive all SSDI benefits after initially returning to work, but may eventually lose their cash benefits completely after engaging in substantial work activity (with earnings above a threshold level set by SSA) for a sustained amount of time-a phenomenon commonly called the "cash cliff". This complete loss of benefits may inhibit some beneficiaries from engaging in substantial work (Ruh & Staubli, 2019;Schimmel, Stapleton, & Song, 2011;Stapleton, O'Day, Livermore, & Imparato, 2006). Qualitative evidence suggests that both the fear of losing benefits and confusion related to the complexity of SSDI rules can inhibit work (O'Day, Martin, Burak, Freeman, Feeney, Lim, Kelley, & Morrison, 2016).
The goal of POD is to simplify work-incentive rules and facilitate work by reducing benefits gradually as earnings increase above an earnings threshold, but the rules also leave some beneficiaries financially worse off. The new rules are favorable for beneficiaries whose earnings regularly exceed the threshold for the cash cliff, potentially reducing the disincentive to work under current SSDI rules. Simplified rules could also reduce other, ongoing administrative burdens of the SSDI program in ways that both increase work and improve wellbeing. However, because the POD threshold is below the cash-cliff threshold, the new rules result in lower SSDI cash benefits, and thus lower total income from earnings and benefits, for workers with earnings between those two thresholds.
Take-up was an important concern for POD, which is being evaluated using a randomized controlled trial to assess impacts on work and benefit receipt outcomes. The demonstration needed to include a sufficient number of enrollees to reliably measure these impacts, but we expected that a relatively small share of eligible beneficiaries would enroll based on prior SSA work-incentive and employment support demonstration projects (Gubits et al., 2013;Stapleton, Mamun, & Page, 2014). Qualitative findings from one of these demonstrations also highlighted decision-making burdens stemming from understanding SSDI work-incentive rules, changes to these rules, and social stigmas, as well the potential for delays and disengagement to limit enrollment (Gubits et al., 2013).

POD Recruitment Efforts
With this context in mind, recruitment efforts for POD sought to leverage principles from past research. A range of studies have shown how providing targeted information to potential program participants can improve take-up (Armour, 2018;Bhargava & Manoli, 2015;Engström, Forsell, Hagen, & Stefánsson, 2019;Mastrobuoni, 2011). Even in situations where only a subset of potential participants stands to gain from a program and others could be worse off, targeted information and assistance can increase participation (Bettinger, Long, Oreopoulos, & San-bonmatsu, 2012, Duflo & Saez, 2003. Additionally, we incorporated a principle of repeated contact used to improve response rates to potentially burdensome surveys (Dillman, 1991).
The core recruitment effort centered on mailings containing primary study enrollment materials intended to provide information about POD and support informed consent among those interested in participating. We sent primary mailings to working-age SSDI beneficiaries who were eligible for POD in eight sites where SSA was conducting the demonstration from January to October 2018, with enrollment closing at the start of 2019. 2 The primary mailing included a letter, supplemental information describing the current and POD work-incentive rules, a consent form, and a short survey. Because it was critical for potential enrollees to make an informed decision about participating, the informational materials needed to provide an accurate depiction of the complex rules currently governing SSDI benefits and work and the changes introduced by POD, along with a sense of the circumstances under which someone might benefit or be worse off under POD rules.
The primary mailing was part of a broader direct outreach strategy that included multiple contacts with potential enrollees both before and after the primary mailing. 3 About two weeks before the primary mailing, we sent an advance notification postcard intended to promote awareness of POD among interested beneficiaries. About two weeks after the primary mailing, we sent a reminder postcard encouraging beneficiaries to consider enrolling in the study. Results from a pilot test conducted during the first two months of POD recruitment indicated that this reminder postcard resulted in an enrollment rate that was 1.5 times as large as the rate achieved if only sending a primary mailing. This pilot also tested the effectiveness of reminder phone calls, which were similarly effective as postcards but cost substantially more, leading us to focus on additional postcards in this final reminder effort. (Text messaging was not an option in this setting.) A separate report from the POD evaluation includes more details about outreach and recruitment, as well as information about how self-selected enrollees differ from SSDI beneficiaries who did not enroll in POD (Hock, Wittenburg, Levere, Denny-Brown, & Gordon, 2020a).

Final Reminder Postcards
We sought to conduct additional outreach because it was not clear that ongoing recruitment efforts would attract enough enrollees to reliably evaluate POD. Together with SSA, we developed a second, final reminder postcard based on the success of the initial reminder postcard and the principle of repeated contact. We sent final reminder postcards to beneficiaries who had been included in primary mailings from July to September 2018. We focused on beneficiaries who had not yet enrolled and for whom we had valid contact information as of late October and early November 2018-that is, 5 to 14 weeks after the primary mailing. We also limited the sample to exclude those with the highest expected likelihood of enrollment (who were targeted for other outreach initiatives in late 2018) and those requiring special options for SSA notices (such as large print). 4 We designed four versions of the postcard that could potentially address the enrollment decision-making burdens and bottlenecks described above in different ways. The four versions were the result of varying each of two messaging components: 1. Structure and language (fold-over versus open card). The fold-over postcard structure contained information specifically about POD. Beneficiaries had to take an active step to open this type of postcard, which displayed the POD logo, noted that POD might allow beneficiaries to keep SSDI benefits while working, and pointed them to the toll-free line and website for more information. In contrast, the open postcard structure contained less-specific information and displayed the SSA logo. Beneficiaries could immediately read this type of postcard, which described "an important SSA study" but did not mention POD directly (which would have revealed private information about receipt of SSDI benefits). 2. Timeline framing (urgent versus deadline). The urgent framing included an "act now" message indicating that time was running out to sign up for POD, but it did not specify a cutoff date for enrollment. The deadline framing stated the deadline for POD enrollment and used a "time left" message indicating that there was still a chance to sign up.
We did not include a "pure" control group (receiving no final reminder postcard) because evidence from the pilot test indicated that reminder postcards were effective at increasing enrollment, which was the proximal goal of enhanced outreach. Appendix B displays the four postcard versions.
We identified these messaging variations based on the behavioral insights literature. Making information more salient, as in the fold-over version of the postcard, can increase the effectiveness of messaging (Richburg-Hayes et al., 2017). Folded designs also have the potential to help distinguish outreach mailings from advertisements (Dillman, 1991). In addition, both highlighting urgency and including deadlines have been shown to increase program participation (Amin, Chojnacki, Moorthy, Perez-Johnson, Darling, & Lefkowitz, 2017;Darling, O'Leary, Perez-Johnson, Lefkowitz, Kline, Damerow, Eberts, Amin, & Chojnacki, 2017;Wright et al., 2017;Richburg-Hayes et al., 2017). Knowles et al. (2017) conducted a field experiment on charitable giving showing that providing either a short deadline or no deadline increases giving relative to a farther off deadline, suggesting that the longer deadline might lead to inaction due to procrastination. Taubinsky (2014) also discusses deadlines and reminders in the context of a model of inattention and evidence from completion of online surveys over a brief study period. Implications from this study suggest that longer deadlines could lead to lower response rates than shorter deadlines, though many responses might still occur both shortly after a reminder or at or near the deadline. Our design was intended to complement the existing empirical evidence by measuring the overall effect of a non-urgent deadline on enrollment rates at the time of the deadline in the potentially distinctive context of taking up a program that may involve multiple years of prospective engagement. Additionally, as discussed below, we also conducted an exploratory assessment of whether this framing might have altered the timing of enrollment, which could be an important consideration for future recruitment efforts with time limits or other calendar constraints.

Methodology
We randomly assigned beneficiaries from the July-September primary mailings who met the criteria described earlier to be assigned to one of the four postcard versions (Figure 1). We stratified beneficiaries into groups by the month of the primary mailing and demonstration site. Within each stratum, we then assigned approximately 25 percent of the beneficiaries to each of the four postcard versions. The analysis sample consisted of 146,548 beneficiaries who met the criteria for being sent a final reminder postcard. 5 Beneficiaries in each of the four groups had comparable initial characteristics such as age, gender, SSDI duration, and primary diagnosis; we found no measurable differences in these characteristics across random assignment groups (see Appendix C).
To measure the impact of each postcard messaging component, we made comparisons that leveraged the random assignment design based on an intent-to-treat principle. For example, we measured the relative effectiveness of postcard structure based on the difference in enrollment rates between beneficiaries assigned to fold-over postcards and those assigned to open postcards. Similarly, we measured the relative effectiveness of

Figure 1 Randomly Assigning Beneficiaries for the Final Reminder Postcard Experiment
Notes: Beneficiaries were selected for the experiment if they were sent a primary mailing, were not targeted for other concurrent outreach, and did not require special options for notices from SSA. Beneficiaries who had an invalid address or who responded to the primary mailing as of the date the postcards were printed were excluded from the sample. timeline framing by comparing enrollment rates between groups of beneficiaries assigned to urgent framing versus those assigned to deadline framing. In practice, we used a regression model that used fixed effects to account for the stratified random assignment by month of primary mailing and demonstration site. The model also accounted for potential heteroscedasticity using robust standard errors. The main model included binary indicators for each of the two messaging components, which allowed us to test the relative effectiveness of structure/language and timeline framing, and we also estimated alternative specifications that included interaction terms. Our primary results are for enrollment rates at the end of the recruitment period, but we also conducted exploratory tests examining enrollment at earlier dates to assess whether relative effectiveness changed over time.
In addition to the analysis of the experiment described in this study, we also developed a quasi-experimental test to assess the likely average effects of being sent any final reminder postcard based on benchmark enrollment rates of similar beneficiaries from earlier primary mailings (who were not sent this postcard). This comparison group included beneficiaries who were not sent the postcard because they were included in a primary mailing in June 2018 but otherwise met similar criteria described earlier to be eligible for the final reminder postcard. For groups of beneficiaries sent the final reminder postcard, we matched beneficiaries from the June 2018 primary mailing with similar characteristics and who had also not responded a similar number of days after the primary mailing, to ensure the greatest comparability. Beneficiaries in the June 2018 primary mailing could contribute to the comparison group benchmark for none, some, or all of the primary weekly mailings in the postcard group. To measure the impacts of this postcard, we used a regression model to assess whether the enrollment rate was higher in the postcard group than in the comparison group. Additional details about the methodology from the quasi-experimental test are contained in a POD evaluation report (Hock, Levere, & Wittenburg, 2020b).

Findings
Among those sent a final reminder postcard, the share who enrolled in POD differed across the four versions ( Figure 2 and the first panel of Table 1). Observed enrollment rates were highest for beneficiaries who were sent the fold-over postcard structure with an urgent framing and lowest for those sent the open postcard structure with an urgent framing. On average, about 0.57 percent of people sent a final reminder postcard enrolled. This corresponds to one quarter of the final enrollment rate (2.1%) among beneficiaries who were sent primary mailings and who might have been eligible for the postcard had they not responded (excluding those who would

Figure 2 Enrollment Rates Across Versions of the Final Reminder Postcard
Notes: The figure shows enrollment rates among beneficiaries who were sent each of the four versions of the final reminder postcard. Enrollment rates are estimated from a regression model that includes fixed effects to account for the stratified random assignment design. The 95 percent confidence interval uses heteroskedasticity robust standard errors. have been targeted for other outreach). 6 As discussed below, our exploratory quasi-experimental analysis suggests that a slight majority of these enrollments would likely have occurred even without the final postcard.
The fold-over postcard structure led to a higher rate of enrollment than the open postcard. Combining information across the postcard versions using the regression model described above, we found that those who were sent fold-over postcards with more specific information about POD were 0.12 percentage points more likely to enroll than those sent an open postcard with more generic language (Table 1, second panel). This statistically significant difference represents a nearly one-quarter increase relative to the enrollment rate of 0.51 percent for the open postcard.
Final enrollment rates did not differ measurably across postcards using the urgent and deadline framings. By the close of enrollment, a little less than 0.6 percent of beneficiaries who were sent each type of timeline framing had enrolled in POD, and the difference in enrollment rates was statistically insignificant (Table 1, third panel). Additionally, no synergies were evident between the two messaging components; the fold-over postcard structure was equally effective when paired with either the urgent framing or the deadline framing (results not shown).
Although the urgent framing and deadline framing were similarly effective at the end of the enrollment period, the urgent framing was more effective early on. Results from secondary analyses examining effects at different points in time indicate that the urgent framing led people to respond faster, though this effect faded closer to the stated deadline (Figure 3). For example, as of the first week in December, the enrollment rate was 0.09 percentage points higher with the urgent framing than with the deadline framing. However, from that point on, the gap in enrollment between the two timeline framings began to narrow and continued to do so until the close of recruitment. This suggests that the urgent framing might have become less salient over time, the deadline framing might have become more salient as the deadline approached, or both. In contrast, this secondary analysis showed that the measured effectiveness of the fold-over postcard structure relative to the open postcard structure grew consistently over time. The final reminder postcards themselves likely increased enrollment. Exploratory results suggest that the postcard group was more likely to enroll than the comparison group that was not sent the postcard (Appendix D). We estimated that 0.57 percent of people in the postcard group and 0.31 percent of those in the comparison group enrolled in POD. Hence, the final reminder postcard might have almost doubled the enrollment rate among people who were sent this card. Additionally, though some versions of the postcard were relatively less effective (such as the open postcard variants), each version of the postcard still likely increased enrollment relative to the counterfactual of no postcard.

Discussion and Conclusion
The final reminder postcard experiment yielded a better understanding of how to design program outreach in ways that increased enrollment for POD. We suspected that outreach could help some SSDI beneficiaries overcome the hurdles of learning about both POD and current work-incentive rules, fill out forms, or overcome other behavioral bottlenecks, thereby increasing take-up among those who concluded that the new rules might be more desirable (and less burdensome) for the period of the demonstration. When testing the effectiveness of each component of the postcard design, we determined that a fold-over postcard structure was particularly effective. However, while the fold-over postcard produced a large gain in enrollment (almost 25%) relative to the open postcard, the absolute increase in enrollment was only 0.1 percentage points, which potentially limits the external validity of these findings.
Several features of the fold-over postcard's structure and language could have influenced behaviors related to take-up. First, the recipient had to open the postcard to read it. Second, the fold-over postcard contained

Figure 3 Relative Effectiveness of Each Postcard Messaging Component Over Time
Notes: The figure shows estimated impacts on the enrollment rate over time using circles for the fold-over postcards compared to open postcards and diamonds for the urgent framing compared to the deadline framing. A circle or diamond marker that is solid (hollow) indicates that the corresponding impact estimate on that date is significantly (insignificantly) different from zero at the 5 percent level. For each messaging component, the difference between the maximum and minimum impact estimates is statistically significantly at the 5 percent level, indicating that the effectiveness changed over time. All postcards were mailed by November 2, 2018, so the figure only considers time points after which final reminder postcards were likely received. more specific information about POD, whereas the open postcard referenced "an important study." Additionally, the fold-over postcard displayed the POD logo instead of the SSA logo and used slightly different language indicating the voluntary nature of the study. Taken as a whole, these differences might have made the fold-over postcard appear more personalized and engaging, or facilitated sustained action, thereby leading more to enroll in POD.
Although the timeline framing of messaging did not affect final enrollment rates for POD, our exploratory analysis indicated that the urgent framing appeared to result in faster enrollment compared to the deadline framing. These findings are consistent with previous literature on deadlines and reminders (e.g., Knowles et al., 2017;Taubinsky, 2014), although we consider a potentially distinct behavior (program take-up versus charitable giving or completing online surveys). When the postcards were initially sent, the deadline was about two months away, leading to potential inattention due to a lack of urgency. The urgent message might have temporarily reduced this type of inattention through an "interruption" that re-focused recipients on POD enrollment, given that this message generated significantly greater enrollment early on. In contrast, as the deadline approached, those sent the deadline framing became relatively more likely to enroll than those sent the urgent framing, leading the positive impact of the urgent framing to disappear. Thus, the approaching deadline might have separately re-focused potential enrollees by triggering their memories. However, it may be important to verify these patterns in future research because they did not follow from this study's primary experimental test; interpretation of these exploratory findings might be limited by factors not accounted for in our design and analysis.
Additional results from a separate quasi-experimental test of the overall effectiveness of the final reminder postcard also suggested that, on average, this postcard likely produced substantial increases in take-up. Together with experimental estimates of the impact of the first set of reminder postcards, these results suggest a general pattern of more contact with beneficiaries leading to greater enrollment, which aligns both with past research on program take-up (Richburg-Hayes et al., 2017) and similar research in the context of survey response rates (Dillman, 1991). However, this study emphasizes the potential importance of how such contact occurs, similar to recent research about types of contact in the context of enrollment in social programs (Engström et al., 2019;Richburg-Hayes et al., 2017), particularly given declining incremental gains from additional efforts. That said, the study can only speak to the effects of different forms of contact on enrollment; it cannot provide insights about whether low overall enrollment was due to unaddressed decision-making bottlenecks versus an explicit understanding that POD could adversely affect total income (or other rational reasons).
The findings of this study also highlight the potential for refining outreach to increase take-up of workrelated initiatives for people with disabilities, which could be of critical importance for them to achieve selfsufficiency. Although 40 to 45 percent of SSDI beneficiaries want or expect to work, only one-third of these work-oriented beneficiaries are employed or searching for jobs (Livermore, Shenk, & Sevak, 2020). Less than half of work-oriented beneficiaries are aware of key SSA work supports (Livermore et al., 2020). Limited awareness along with other decision-making bottlenecks could explain the low take up of other work-incentive and employment support demonstrations discussed previously (Gubits et al., 2013;Stapleton et al., 2014). However, our findings indicate that it was possible during the course of a few months to increase voluntary enrollment in POD by enhancing outreach for the demonstration. Given the range of mechanisms whereby outreach could have addressed the burden of decision-making and take-up, future work could provide deeper insights about the mechanism(s) underlying changes in enrollment by testing additional variations concurrently-for example, varying a postcard's physical design independently from the information it contains. Nonetheless, our results further reinforce the broad value of tailored, strategic messaging as a way to address bottlenecks that might inhibit take-up of employment programs among people with disabilities.

Notes
1. In addition to the design of the postcard (fold-over versus open) and the language (specific versus general), several other aspects of the two types of postcard structure differed. For example, the fold-over postcard structure indicated that it was the beneficiary's choice to enroll, while the open postcard structure indicated that a beneficiary only needed to respond if they wanted to sign up for the study. Additionally, the foldover postcard structure had a POD logo while the open postcard structure had the SSA logo. Hence, differences in enrollment between the fold-over and open postcard structures reflect all of these differences. 2. SSA identified this pool of beneficiaries as those who, at the time of recruitment, were living in a site where POD was being tested, were at least age 20 and under age 62 for the duration of the project, were entitled to SSDI based on their own past earnings (and not entitled to SSDI as a dependent), were either receiving SSDI benefits or had their benefits suspended due to work, and were not participating in any other SSA demonstration project, among other criteria. 3. Additional, indirect outreach for POD included a toll-free telephone line and website, as well as engagement with community organizations that help SSDI beneficiaries make enrollment decisions. 4. At the same time we sent the postcard, we sent a letter highlighting the benefits of POD for people who regularly had high earnings. We targeted this letter toward people with a recent history of earnings above SSA's Trial Work Period level ($850 per month in 2018), since the enrollment rate through the summer of 2018 for this group had been twice as high as for others. In total, 8.2 percent of people potentially eligible to enroll in POD had such recent earnings. To avoid duplicated effort or confusion, the final reminder postcard tested in the experiment was reserved for beneficiaries without such an earnings history. We also excluded from the experiment blind or visually impaired SSDI beneficiaries who selected alternative options for receiving SSA notices (for example Braille, large print, or audio recordings). 5. The analysis sample excluded a small set of people (195 beneficiaries) who enrolled between when we identified the experimental sample and when the final reminder postcard was mailed. 6. As noted previously, those eligible for the postcard exclude those with high earnings (because they were targeted for other concurrent outreach), those who had already responded to previous forms of outreach (because the subsequent outreach would not affect them), and those who require special options for SSA notices.

Appendix C. Characteristics of Beneficiaries Assigned to the Four Final Reminder Postcard Versions at the time of Random Assignment
Final reminder postcard version