Encouraging firms to adopt beneficial new behaviors: Lessons from a large-scale cluster-randomized field experiment

Policymakers are interested in how to encourage firms to adopt beneficial new behaviors. In this study, we report on the results of an experiment to encourage firms to file their annual accounts electronically and on time. Our intervention involved UK firms filing their annual company accounts to an official registrar of companies. In a cluster-randomized controlled trial, we found behaviorally informed letters had no detectable effect on encouraging firms to file electronically. A letter using a social norm had a small (2.4%, p=0.053) effect on encouraging firms to file on time. The trial indicates behavioral science in this context has limited use in influencing firms to adopt new behaviors. We conclude more evidence is required to understand which behavioral interventions will have the most impact on influencing firm behavior in different contexts. 

A large number of firms do not adopt new behaviors even when they are beneficial. Research has found better management practices are associated with higher profitability and productivity (Bloom al., 2012;Bloom at al., 2013). Similarly, the introduction of new technologies can lead to significant benefits to firms (Atkin et al., 2017). For instance, the Office for National Statistics in the United Kingdom estimates the adoption of business management technologies is associated with a productivity gain of 25% (ONS, 2018). Despite these benefits, the diffusion of leading practices and technologies among firms remains surprisingly slow (Atkin et al., 2017). As one example, in 2016 only 25% of companies in the OECD area reported using cloud computing services despite potential efficiencies (OECD, 2017). Because of productivity benefits to the economy, policymakers are interested in how to increase the adoption of leading practices and technologies by firms (Haldane, 2017).
As its central question, this article considers how policymakers can influence firms to adopt beneficial new behaviors. Whilst firms may fail to adopt a new behavior for multiple reasons, including access to credit, infrastructure, and skills, we focus on an area highlighted as important by empirical work: the salience and influence of information (Bloom et al., 2013).
We present a large-scale cluster-randomized controlled trial that uses a behavioral intervention to highlight the benefits of using an electronic service to file annual accounts and compliance with filing annual accounts on time. In our context, adoption of electronic filing (e-filing) is already high (some eight in ten filers already use e-filing) and, for the population that receives the intervention, compliance is at 85%.
The key contribution of this paper is to add empirical evidence to findings that show promising, but still emerging, evidence on the use of behavioral interventions to influence firm behavior.
Behavioral interventions in policy have generally been targeted at individuals (Sanders, Snijders & Hallsworth, 2018). However, there is growing interest in how policymakers can influence firms (Leets et al., 2020;Holz et al., 2020). In contrast to recent positive findings focused on tax compliance (Leets et al., 2020;Holz et al., 2020), we find no impact of a behavioral intervention to increase the adoption of e-filing among firms and a positive but marginal impact on compliance. Our results provide, at best, muted encouragement for the use of behavioral interventions to influence firms to take-up beneficial new behaviors. In summary, this study expands firm-level interventions into areas beyond tax compliance but finds more evidence is required to understand the most consistent and effective behavioral interventions for firms.

Prompting firms to adopt new behaviors
This article builds on literature related to influencing firms through behavioral interventions. We focus on two areas of this literature: compliance and the adoption of new management practices or technologies.
In the area of tax, Hallsworth et al. (2017) present a model that suggests compliance with timely payments is a function of liquidity constraints, the penalty for paying late, and moral costs.
Whilst this model has generally applied to individuals, a recent large-scale trial found an increase in the salience of negative consequences led to higher tax payments among firms (Holz et al., 2020).
Firm-level adoption of new technology has a wide literature. Workplace conflict between manager and employee (Atkin et al., 2017), risk regarding return on investment (Atkin et al., 2017), competition (Bloom & Van Reenen, 2007), size (Chen & Srinivasan, 2019), age (Chen & Srinivasan, 2019), and access to information (Bloom et al., 2013) are just some areas presented as predictors of firm-level technology adoption. Ultimately, these factors are summarized in the Technology-Organization-Environment framework that argues firm decisions to adopt new technologies are influenced by the feasibility of the technology, internal organizational dynamics, and external environmental factors such as competition (Tornatzky, Fleischer & Chakrabarti, 1990).
Empirical studies of targeted interventions to influence specific firm behaviors, especially by governments, remain rare (Atkin et al., 2017;Bandiera, Barankay & Rasul, 2011). Similar to the application of behavioral science to increase individual tax compliance the relative effect of different approaches remains unclear (John & Blume, 2018). The lack of empirical studies is particularly the case for behavioral interventions that attempt to influence firms by leveraging external environmental factors.
From the available behavioral studies, there are indications that interventions that use simplification (Leets et al., 2020;Broughton et al., 2019, p.18;Rosenkranz et al., 2017), personalization (Algate, 2015), salience of potential penalties (Holz et al., 2020), social norms (Brahm, Lafortune & Tessada, 2016;Kettle et al., 2016), formality , and a 'kitchen-sink' approach that combines multiple behavioral prompts (Brockmeyer et al., 2019;Vainre et al., 2020) can succeed in influencing the behavior of firms. These techniques are applied in areas such as improving tax compliance or increasing the take-up of government subsidies and programmes. For the impact of messengers, current findings are generally limited to qualitative studies and correlational work (Bartholomew & Smith, 2006). In general, there is insufficient evidence in the existing literature for policymakers to be confident in consistent results.
Finally, there is also a relevant literature on timeliness of financial reporting and adoption of internet-based financial reporting among firms. Generally, the literature suggests a positive association between timeliness and firm size, firm profitability, whether the firm has low leverage, and the size of a firm's auditor (Alkhatib & Marji, 2012). In a similar finding, use of internet-based financial reporting is found to be associated with firm size, whether a firm is listed, and the level of technology sophistication in the industry (Debreceny, Gray & Rahman, 2002).
This trial offered the opportunity to test whether qualitative and correlational findings on the impact of messengers translate into a causal environment at scale. The size and scale of the trial also meant we could add further evidence to the role of social norms in influencing firm behavior in a developed economy. Finally, dynamic social norms that highlight how a norm is changing have successfully influenced the behavior of individuals in small-scale experiments (Sparkman & Walton, 2017). This trial offered an opportunity to test whether dynamic social norms are effective with firms at scale. Therefore, before launching the trial we generated four hypotheses.

H1:
A simplified and salient letter will out-perform a status-quo letter in encouraging firms to file on time and via e-filing.

H2:
The use of a relevant external messenger will out-perform a status-quo letter in encouraging firms to file on time and via e-filing.

H3:
The use of a social norm will out-perform a status-quo letter in encouraging firms to file on time and via e-filing.

H4:
The use of a dynamic social norm will out-perform a status-quo letter in encouraging firms to file on time and via e-filing.
In exploratory analysis, we consider how available firm characteristics such as age of firm, location, sector, and whether they use an accountant predict likelihood to file by paper.

Study context
Companies House is the registry for all UK limited companies. There are more than four million companies on its register and 600,000 new companies incorporated each year (Companies House, 2018a). Sole traders are not required to register with Companies House. Only those who would like to operate as a limited company are required to register. Being a limited company creates a distinct legal entity separate from its owners providing financial and legal protections.
As part of its goal to reduce costs, save time, and increase data integrity, Companies House aims to be an entirely digital organization (Companies House, 2018b). Companies House wants companies to switch to e-filing their annual accounts rather than by paper and post. At the same time, it wants to maintain or increase the number of companies filing on time. Higher e-filing of annual accounts increases efficiency and reduces the risk of error for Companies House. Firms and accountants are likely to experience cost savings and efficiencies by moving to e-filing.
The proportion of firms using e-filing methods with Companies House increased from one in ten filers in the financial year 2008-2009 to eight in ten by 2017-2018. However, the adoption of e-filing has slowed in recent years. Of the firms that file by paper, Companies House research suggests between 15-20% are filing their own company accounts (self-filers). The remainder are filed by accountants acting on behalf of the firm. Internal research by Companies House found inertia as the primary barrier to shifting to e-filing. For instance, in surveys run by Companies House both accountants and small and medium-sized enterprises were aware of the benefits of e-filing. They report advantages such as speed, ease, immediate confirmation of submission, security, reduced paper usage, and the lower cost of e-filing. However, knowledge of these benefits apparently fails to outweigh the familiarity and status-quo of paper.
Finally, reminder letters are sent 35 to 42 days before a company's annual accounts are due if Companies House has not yet received their accounts. Companies House identified reminder letters as the easiest existing communication channel with companies that they were able to change. Therefore, reminder letters were chosen as a pragmatic communications moment to help prompt change.

The intervention
Reminder letters are sent to the 58% of companies on the register who have not signed up to receive email reminders. Letters include an extra leaflet that reminds companies of penalties associated with not fulfilling their responsibilities to file accounts (see 'control letter leaflet' in the appendix). All companies must provide Companies House with an official contact address for the delivery of statutory mail. This address can represent the company's own office address or that of the firm who provides accounting services on their behalf.
The status-quo reminder letter used by Companies House, whilst containing appropriate information and frequently asked questions, lacked a direct call to action for the use of e-filing (see Figure 1).

Figure 1
Status-quo Reminder Letter Sent to Firms 9 The trial tested three new versions of the reminder letter. The new letters were simpler than the original reminder letter and increased the salience of the request to e-file. A prominent 'stamp' with the text "Your accounts are now due" was added to ensure the filing date was salient. In addition, each of the three new letters included a headline message. Letter A used a static social norm "8 out of 10 companies file online". Letter B tested a dynamic social norm "Over the past 5 years, online filing has increased from 50% to 80%. Consider filing online this year.". Letter C used a messenger effect "I file my accounts online every year. It was quick and easy. I would recommend it for everyone. Louise -company director". See Figure 2 for the front page of the social norm letter.

Method
This was a four-arm cluster-randomized controlled trial, with stratification at the cluster level, and outcomes measured at the individual company level. Companies due to file between 12th September and 31st December 2018 were clustered by postcode then randomly allocated to receive the control letter or one of the three treatment letters.
We randomized at the level of postcodes rather than individual companies because multiple companies often share the same postcode. This means that companies could end up receiving (or being aware of) different versions of the reminder letter. Companies were grouped into clusters according to their postcode. Postcodes were then randomized to receive either the control letter or one of our treatment letters. All companies within the same postcode cluster were sent an identical letter. We further stratified the randomization by age of the company in years, company's standard industrial classification, region in which the company is registered, whether the company is a private limited company and the month that the accounts were due.
We excluded companies that shared their address with more than 100 other companies and additionally stratified the randomization according to the cluster size (the number of businesses that shared an address). The cluster size grouping were: no duplicates, 1 duplicate, 2-9 duplicates, 10-25 duplicates, 26-49 duplicates, 50-74 duplicates, and 75-99 duplicates. Clusters of more than 100 companies were excluded due to lack of clarity in defining what these clusters represent making it difficult to know how to interpret their filing behavior. Excluding larger clusters also prevents the issue of having highly unbalanced cluster sizes which may threaten the trial's internal validity.
The sample was selected to only include active companies from the Companies House Database who could conceivably file online. The full trial sample included a total of 1,291,232 active companies. This number reduced to 672,758 eligible companies once exclusion criteria were applied (see appendix for criteria). This equates to 290,284 postcode clusters in the trial sample.
Letters were sent between 7th August 2018 and 12th December 2018. Of the 672,758 eligible companies in the trial, 481,888 were sent a reminder letter. The 190,870 companies who were not sent a reminder letter were mainly those that had filed before the letters were sent. For a small group (4,287 companies) this was due to entering insolvency proceedings during the trial period.
Given this is still an emerging area we assumed power of .8 and an alpha of .05 and we varied the baseline proportion from 0.79 to 0.84. For all our analysis we use an ordinary least squares regression model, estimating the Intention to Treat (ITT). For our primary and secondary analyses (which differ based on the sample analyzed) the model is as follows: ! is the outcome measure; a binary variable taking the value of one (1) if the company files electronically and (0) otherwise for the population that did file. In our secondary, specification ! is a binary variable taking the value of (1) if the company filed on-time and (0) otherwise.
is the constant. company's standard industrial classification, region in which the company is registered, whether the company is a private limited company, month the accounts were due, and a categorical measure of the number of companies they share an address with. As a robustness check, an indicator for whether the previous filing type (where available) was paper-based is included.
Standard errors are corrected for clustering at the unique postcode level, as randomization is conducted at the level of unique postcode. When we investigate the level of heterogeneity in the estimated treatment effect we consider: company age, sector (by the UK's standard industrial classification code), region, previous filing type, and the month that the accounts were due. Table   2 in the appendix compares the average value of the key variables used in this analysis across each of the treatment arms.

Results
When looking at those companies who filed, we found no difference in likelihood to file by paper between companies who received the control letter and those who received one of the three treatment letters. See Figure 3. Error bars correspond to the 95% confidence interval associated with the estimated treatment effect. We also observe a null result when treatment groups are combined. Therefore, we reject all hypotheses that the reminder letter had an impact on e-filing. 2

Figure 3
Likelihood of Paper Filing  Figure 4). This is equivalent to a 2.4% reduction in the probability of non-compliance. In addition, Letter A is found to reduce the probability of just filing late by 0.3 percentage points (p<0.05, 95% CI = [-0.50, -0.10]), relative to the control Letter D. The proportion of companies that file late amongst the control group is 8.3% implying that the decrease of 0.3 percentage points is equivalent to a 3.6% reduction. In both compliance and filing late, we observe a null result when treatment groups are combined. In conclusion, we reject Hypotheses 1 and 4 that simplification or dynamic social norms would have an impact on compliance. For Hypotheses 2 and 3, whilst the intervention did not have a statistically significant impact at the 5% level of significance for our primary measure of interest (late or no filing) it did at the 10% level (p=0.053) and for solely filing late. Given the large sample size, the small effect size (2.4%) and failure to (just) reject the null at the 5% level indicates the treatment letters had, at best, a marginal impact. However, the low cost of the intervention, effectively zero as it is a letter variation, means even a small effect such as this should be of interest.

Exploratory analysis
Our third set of analyses investigated whether the impact of the letters differed by company age, sector or region. We regressed each of the outcomes on a binary indicator for any behaviorally informed treatment and the interaction between this treatment indicator and each of company age, sector, and region. Once we applied the Hochberg correction to account for carrying out multiple tests, there was no difference in the effect of letter type across firms that differed according to age, sector or region (Hochberg, 1988). In other words, the treatment letters were not more or less effective for certain firms in our sample.

Impact of recipient
We hypothesized that a reminder letter may have a stronger impact on the behavior of self-filers compared to accountants. Self-filers may be more likely to open the letter themselves and act on its contents compared to an accountant who receives multiple letters.
The data does not allow us to distinguish between self-filers and accountants. We instead use a proxy measure of whether a company has the same registered address as another company. A company that does not have the same address as any other company is more likely to represent a company filing its own accounts, whilst companies that have the same address as other companies are more likely to represent those using the same accountant (as the registered address provided is that of the accounting firm, which files on behalf of multiple companies with that same address). There were 261,626 companies who do not share an address.
We repeated the analysis restricting our focus to the sub-group of companies who did not share an address (our proxy for self-filers). We found a stronger effect on the likelihood of general non-compliance (filing late or not at all). Both letters A and C decrease non-compliance by around 0.5 percentage points for those who do not share an address from a baseline of 17.8% of the control group companies who are non-compliant. Table 3 in the appendix shows output from this regression analysis.

Predictors of filing by paper
The size of the trial dataset allowed us to perform predictive analysis on the likelihood of a company filing by paper using gradient boosted decision-tree analysis. Gradient-boosted decision tree analysis is a machine learning technique which algorithmically partitions the data based on the variance in the estimated probability of filing by paper. 3 Nearly two-thirds (65.3%) of companies who filed by paper in the previous year filed again by paper in the current year, in comparison to just 2.3% of those who had previously e-filed. This suggests that companies consistently file by paper rather than switch between methods. The other factors that were identified as having explanatory power were:

Link between findings and current literature
For our hypotheses, our results provide only muted encouragement. We reject Hypotheses 1 and 4. Simplification, as tested by combining all treatment arms, produces a null result. The success of dynamic social norms for individuals does not translate to our setting of employees within firms. For Hypotheses 2 and 3, we find Letters A (messenger) and C (social norm) were effective, though with a marginal effect size, when prompting what was likely to be an intended behavior (filing annual accounts) that has a known and clear sanction for non-compliance.
The letters were not successful at increasing the adoption of e-filing through a message increasing the salience of external environmental factors. This was the case even though the behavioral prompts specifically targeted this action. Both results indicate a stronger intervention than a letter may be required to shift larger numbers of firms to take-up new behaviors. Our results show a clear null when the behavior was voluntary and technical feasibility and organizational readiness for the targeted firms were unclear to the public administrator. This provides new empirical data for understanding when external environmental factors, such as the use of social norms, may be less effective at influencing firms to adopt new technologies (Tornatzky, Fleischer & Chakrabarti, 1990).
Our findings stand in contrast to previous work in developing economies that found a significant impact from messages directed at firms that make salient the costs of non-compliance or apply social norms (Holz et al., 2020;Kettle et al., 2016). In our context, the behavioral messages were not sufficient to cause large changes. Similar to findings applying social norms to individual tax compliance, our results show further research is required to understand the nuances of how behavioral messages can influence firm-level decision makers in developed economies (John & Blume, 2018). Further, they reinforce recent findings that observed effect sizes are often lower for behavioral interventions in the presence of large sample sizes and institutional constraints in trial implementation, such as limiting the trial to a letter-based intervention (Dellavigna & Linos, 2020). In one encouraging sign, our proxy for the recipient (self-filer or accountant) revealed some impact -giving encouragement to correlational and qualitative findings related to the importance of targeted messages for firms (Bartholomew & Smith, 2006).

Impact for Companies House and individual companies
We estimate that had all firms in our sample received the social norm Letter A (the letter with the strongest effect) there would be an extra 5,927 compliant companies across the total Companies House sample eligible for our trial. Our minimum estimate is Letter A would save Companies House £19,609 each year. In 2019/20, Companies House received a total £95,700,000 from late filing penalties. Using the best performing letter may result in at least £890,000 of lost penalties for Companies House (and the equivalent in gains back to firms who do not have to pay a penalty). Despite the revenue implications Companies House has an explicit aim to reduce late filing and shift to e-filing. Their measure of success for the trial was an increase in both e-filing and compliance. Finally, our saving estimate does not include the overall efficiency gains for both Companies House and firms from filing accounts on time.

Limitations
The treatment letters introduce several behavioral interventions in that all featured simplification, a clear call to action, and a unique headline message. Given the lack of meaningful difference between the effect sizes of the treatment letters it is difficult to fully disentangle mechanisms.
Further, the majority of the sample had already adopted e-filing, meaning that the remaining 'hold-outs' may have been particularly entrenched in their behavior. It is possible behavioral techniques may be effective at accelerating adoption of new technologies when they are first introduced rather than at this late stage.

Broader Significance of Findings
Our findings are directly relevant to company registrars around the world. In both developed and developing markets, registrars are seeking to become more digital and are seeking new ways to improve compliance (Debreceny, Gray & Rahman, 2002;Mokhtar, 2017). Whilst governments around the world are using behavioral science to inform policies targeted at individual citizens, much less is known about how behavioral insights can influence the actions of employees within firms.
The findings have four implications for wider work by policymakers to prompt firms to take-up new behaviors. First, they present new evidence that suggests letters that make key information salient and include behavioral messages (in particular using social norms) produce real but marginal changes to firm behavior. Second, the predictive analysis adds to the existing literature on internet-based financial reporting and indicates there may be timely moments, such as incorporation, when firms should be provided with additional assistance to achieve leading practice from year one. Third, it matters who receives the letter -letters sent to postcodes with only one company registered are marginally more likely to change behavior. Finally, entrenched behaviors, such as filing by paper, may be difficult to change through small interventions such as those used in this trial. Ultimately, this trial provides muted encouragement for the application of behavioral science to influence firm behavior in regards to compliance. However, the results are not as promising as recent studies related to tax compliance (Leets et al., 2020;Holz et al., 2020).
Therefore, it is clear more research is required to understand which behavioral interventions will have the most impact on influencing firm behavior in different contexts of public administration.

A.4 Messenger treatment letter (front)
Certain companies (618,474 in total) were excluded from the trial, as detailed in Table 2 below. 1,291,232 active companies were due to file between 12th September and 31st December 2018 before exclusions applied. Clusters of more than 100 companies were excluded, due to lack of clarity in defining what these clusters represent (e.g. independent companies located in the same building or area, or companies using the same accounting firm etc.), making it difficult to know how to interpret their filing behavior. Excluding larger clusters also prevents the issue of having highly unbalanced cluster sizes which threatens the trial's internal validity.

168,167
Limited Liability Partnerships Limited liability partnerships were excluded due to differences in the required information within the reminder letters.

Interest Companies and Charities
Community interest companies and charities are unable to use e-filing, due to having to file a specific type of accounts.

2,882
Public Limited Companies Public limited companies have to file full audited accounts, which cannot be done via e-filing methods.

839
Welsh companies Welsh companies that have registered for a bilingual letter were excluded as it was beyond the scope of the trial to produce new treatment letters in the Welsh language.

216
Companies with missing data Companies missing either a valid postcode information or a corporate ID number. There are two rejections of the null hypothesis of balance across the four trial arms. The probability that the company is based in Northern Ireland (p<0.01) or operates in the Wholesale & Retail Trade; Repair of Motor Vehicles sector (p<0.10) is not found to be statistically random across the four treatment arms. While the orthogonality test fails, the differences in the probability of falling into these two categories between the treatment arms is small in magnitude.

3,071
The statistical significance is likely a byproduct of the large sample size and number of hypotheses being tested. As such, we make no change to our analysis strategy on the basis of these tests, and we consider the randomization to be unbiased on key observables. The treatment effect estimates are obtained by applying the base empirical specification to the subset of companies which do not share their address with another company. Each row presents the coefficient estimate with standard errors presented in parentheses. Statistical significance is indicated as follows: + statistically significant at the 10% level, * statistically significant at the 5% level and ** statistically significant at the 1% level.