Additional intervention evidence on the relationship between public service motivation and ethical behavior

hile a substantial amount of research has investigated the existence and importance of public service motivation (PSM), much of this research has focused on two main themes. The first theme involves identifying the link between PSM and important behavioral outcomes such as job attraction, retention, and performance. The second theme builds on the first by identifying the mechanisms by which PSM can be used by organizations and supervisors to increase desired outcomes. In this latter theme, three general strategies have emerged. An organization can increase the positive outcomes associated with PSM by 1) recruiting employees with higher PSM levels or taking steps to either 2) cultivate (re increase), or 3) activate existing employee public service motivations. Researchers have made important progress in our understanding of the questions reflected in these two themes but there is still a lot we do not know. Much of the PSM research, for example, has produced inconsistent findings and relies on weaker designs that limit our confidence (Vandenabeele, Brewer, & Ritz, 2014). As a result, questions remain about the connections between individual public service motives and desired behavioral outcomes. In addition, researchers have yet to fully engage, let alone resolve, whether PSM can change and be increased, or is more stable (like a trait) but can be activated. Consequently, many have called for the use of experimental and quasi-experimental designs that can produce stronger evidence on whether PSM can change over time and how public managers can use PSM to influence organizational behavior (Bozeman & Su, 2015; Prebble, 2016; Wright & Grant, 2010). Our study addresses both of these calls by testing the effectiveness of a prosocial intervention in increasing W V V Abstract: The nascent scholarship on public service motivation (PSM) and ethics exhibits mixed findings. This research article aims to describe and relate the current landscape of findings in this arena and to conduct an experiment that addresses design weaknesses that may explain some past null findings. Using a national sample of college-age respondents, we found that although self-reported PSM was positively correlated with ethical intentions, prosocial priming did not increase ethical intentions or behavior. We contextualize these findings in terms of previous studies, to inform our understanding of the efficacy of prosocial interventions. While our research suggests that self-reported PSM can predict, if not influence, ethical intention, we are unable to make conclusions about PSM’s effects on ethical behavior. Second, similar to past studies, we are not able to confirm specific mechanisms or interventions that might be used to increase ethical behavior or intentions.


ethical behavior by altering or activating an individual's public service motivation under experimental conditions.
We recognize a growing number of experimental studies investigating prosocial motivation interventions but observe that evidence is still mixed as to their effectiveness. Some studies, for example, have used PSM questions to increase self-reported prosocial behavioral intentions (Meyer-Sahling, Mikkelsen, & Schuster, 2018;Pedersen, 2015). Other studies, using simple public service messages, found no effect on public service behavioral outcomes (Linos, 2018). Experiments using more intense interventions, e.g., reflection and self-persuasion, to prime prosocial motivation found an effect on behavioral intentions (Arieli, Grant, & Sagiv, 2014) and outcomes (Bellé, 2013(Bellé, , 2014, while other studies have either found no effect on behavioral intent or behavior (Christensen & Wright, 2018) or an effect on behavioral intent but not on behavior (Awan, Esteve, & van Witteloosuijn, in press). Our study adds to this literature by providing another test of the effectiveness of public service motivation interventions and using a more intensive theory-driven priming intervention (Arieli et al., 2014). We investigate a prime's effect on both behavioral intentions and outcomes in a national student sample.
In addition to adding to our understanding of prosocial motivation interventions, our study also contributes to the growing literature investigating the relationship between PSM and ethical behavior. Current literature investigating this relationship is limited and the evidence is inconclusive. Studies indicating the strongest support for the relationship between PSM and ethical behavior tend to rely on weaker cross-sectional data and measures of behavioral intention or values (Caillier, 2017;Wright, Hassan, & Park, 2016;Stazyk & Davis, 2015), or stronger quasi-experimental studies that find self-reported measures of PSM can predict unethical behavior Gans-Morse, Kalgin, Klimenko, Vorobyev, & Yakovlev, 2019). The findings from recent studies relying on stronger designs attempting to manipulate PSM are more mixed. While two quasi-experimental studies using PSM questions to prime ethical behavior have found no effect on unethical behavior Peng & Li, 2019), a third experiment randomly assigning the PSM prime found that it had a small but statistically significant influence on self-reported ethical behavioral intentions (Meyer-Sahling et al., 2018). Another study conducting several experiments with a more intensive and theory-driven priming intervention found no effect on either ethical intentions or behavior (Christensen & Wright, 2018). Although this was the only study that used priming to test the relationship of PSM on both ethical intentions and behavior, its failure to provide evidence of these relationships may be due to its reliance on student samples from a religiously affiliated university, incentive structures, or measures that may reduce the likelihood of unethical behavior. This current study adds to the existing literature studying the relationship between PSM and ethics by replicating Christensen and Wright (2018) with a more diverse student population and a different measure of ethical behavior that increases the likelihood of cheating by providing a stronger economic incentive for dishonesty and reducing the risk of detecting an individual's dishonesty.

Method
As noted above, this study attempts to strengthen and replicate Christensen and Wright (2018) to more precisely illuminate the connections between prosocial interventions and ethical behavior.

Sample
Consistent with this previous study relying on 2016 data, our present study consists of undergraduate students in the United States and differs in that it uses a national sample of college student respondents who were not necessarily attending religiously affiliated schools. We contracted with Qualtrics to provide a more representative sample of traditional, i.e., not online, college students in the Fall of 2017 (n=309). To be clear, our aim was to collect a sample of students, from across the nation, with demographic proportions reflective of national demographic proportions. We make no claim that our sample is representative of either the institutions-the universities sampled--or the student populations within them. Our intent was not to identify a representative sample of workers but to address specific weaknesses in Christensen and Wright's (2018) earlier work. As such we make no claims to external validity beyond our chosen sample and future work should investigate related questions in different samples, including full-time public servants.
Regarding internal validity, comparing two equivalent groups does not require an unbiased or representative sample (Mook, 1983). To the extent that the Qualtrics sample suffers from selection or response bias, random assignment allows us to evenly spread those biases across the two groups and make the claim that any differences in the ethical outcome measures are most likely due to the treatment and less attributable to any initial differences in the two groups. Given that we randomly assigned participants to experimental conditions and our randomization check failed to find differences across the two groups, we feel confident in making causal claims with important caveats. First, as previously noted, our findings may not be the same in other settings or samples (e.g., a sample of full-time employees). Second, claims based on null findings can be problematic.
Following the procedures of Christensen and Wright (2018), participants were told that they would complete several exercises and provide some basic demographic information (including aspects of their personality and beliefs) to help the research team study the relationship between personality and decision-making. We excluded responses from duplicate IP addresses and those who did not sufficiently attend to the study tasksdescribed below-either by spending too little time (less than 9 minutes), writing too few words (less than 50), or filling their responses with copied/pasted nonsense text. Our usable sample was n=254 1 . Although the attrition rate was high (17.8%), unusable cases were evenly distributed by condition both in terms of frequency with which it occurred (28 in the change condition, 27 in the prosocial condition) and reported demographic differences (gender or age). Respondents in the usable sample were 56% female and averaged 20.6 years of age. Additional descriptive statistics for our sample are reported in Table 1.

Experimental Priming
As in previous studies using the same priming intervention (Christensen & Wright, 2018;Arieli et al., 2014), respondents were randomly assigned to either a prosocial treatment group or a control group. In the former, respondents completed four exercises that required the respondent to reflect in different ways on the value of benevolence--kindness, helpfulness, other-orientation--in their lives. First, the participants in the treatment condition read a short (420 words excluding citations) summary of scientific evidence highlighting how individuals are significantly more cooperative, compassionate, and helpful than most people realize and the benefits of that behavior. Participants were then asked to complete a checklist identifying ways in which they have helped others in the past month. In the third exercise, participants were asked to spend five minutes writing a story about how they (or someone they knew) made a significant difference in the lives of others. In the fourth and final exercise, participants were asked to take another 5 minutes to write two persuasive paragraphs to convince others why it is important to be benevolent, generous, and helpful. In the control condition, participants performed the same four exercises but with a focus on the importance and frequency with which individuals are capable of changing their personality and abilities. There were no statistically significant differences between the treatment and control group in gender, race, age, self-reported social economic status, political ideology, or religiosity (see Table 2).

Ethical Behavior
To address another shortcoming in Christensen and Wright's (2018) work, the present current study attempts to increase the likelihood of participant unethical behavior. To do this we first replaced the dice game with a widely used coin-flip game (Abeler, Becker, & Falk, 2014;Abeler, Nosenzo, & Raymond, 2019;Hugh-Jones, 2016;Pascual-Ezama et al., 2015). Christensen and Wright (2018) relied on computer generated but predetermined dice scores to determine that each participant lied to increase their reward and that participants may have been less likely to cheat if they felt that the scores were secretly known by the researchers. Our study asks respondents to flip their own coin and report whether the result is heads or tails. However, on the page where they report the results of their flip, we told them that flipping a "heads" will earn them a $2 bonus as part of survey completion. Thus, there are two main changes between this study and Christensen and Wright (2018).
First, it provides a stronger economic incentive for cheating by guaranteeing a $2 payout instead of offering entries for a prize drawing for a $100 gift card or the number of lab credits students received for participating in the study. Second, this study reduces the study participants' risk of being caught as our procedures only allow to see whether one group has a higher aggregate probability of cheating (heads being reported at greater than 50 percent in each group) and we are not able to detect whether any individual is dishonestly reporting flipping "heads" to receive the additional $2 payment. This measure was taken immediately after the participants completed the treatment or control exercises.

Ethical Intentions
We measured ethical intentions using the identical two measures used by Christensen and Wright (2018). In the first, students read eight ethical scenarios and reported the percentage chance that they would engage in the unethical behavior described (the scenarios follow those used by Detert, Trevino, & Sweitzer, 2008;Piff, Stancato, Côté, Mendoza-Denton, & Keltner, 2012). Consistent with previous use of this measure, respondents' answers to all eight scenarios were averaged into a single score such that a higher number represented a greater reported intention to behave unethically. In the second measure, students were asked to imagine being tasked with negotiating a salary with a job candidate seeking long-term employment and were asked the percentage chance that they would tell the candidate that the job would be eliminated (Aquino, Freeman, Reed, Lim, & Felps, 2009;Piff et al., 2012). For this measure, a higher number represents a greater self-reported intent to behave ethically. These measures were taken immediately after the participants reported their coin flip results.

Demographic Information
In addition to the measures of ethical intentions and behavior noted above, participants were also asked to provide some basic demographic information. Some of these measures (age, gender, and race) were measured in the beginning of the survey (prior to the treatment) while the others (public service motivation, religiosity, political orientation, and socioeconomic status) were measured at the end of the study (after completing the four exercises as well as the measures of ethical behavior and intention). Public Service Motivation (PSM) was measured with a commonly used five-item global measure of PSM (Wright, Christensen, & Pandey, 2013) with respondents indicated their agreement with each item on a sevenpoint scale (where strongly disagree = 1 and strongly agree = 7). Political orientation was measured using a single item asking the respondents, "Politically speaking, how liberal do you consider yourself compared to the average member of society". Religiosity was measured with a single item asking, "How religious do you consider yourself compared to the average member of society?". Social Economic Status was measured with a single item asking, "How "well off" are you compared to the average member of society in terms of money, education, and opportunities.". For all three of the latter measures, respondents indicated their answers on a seven-point scale (much more = 1 and much less = 7). Table 3 provides the means and standard deviations by condition for different measures of ethical behavior, intentions and prosocial values. To test the effectiveness of the prosocial intervention, we first look at its effects on our measure of ethical behavior. Although the group exposed to the change prime serving as our control condition were more likely to report flipping a Heads (59.4%) than the group exposed to the prosocial prime (54%), the difference is not statistically significant (p < 0.05). Given our relatively small sample size and the use of a coin flip reduces the amount of variation in cheating (approximately 50% of the participants would have flipped Heads so only half of the sample would befit by cheating), our study does not have sufficient power to detect the significance of small differences. Future efforts might increase the number of study participants or the number of coin flips per participant to increase statistical power.

Findings
Unlike previous studies (Christensen & Wright, 2018), there is evidence that cheating did occur and in a pattern that might suggest expected differences might be found in larger samples. Although our design does not allow us to identify which individuals were dishonest in reporting the coin flip results, we can follow the analytical approache used by other studies (Bucciol & Piovesam, 2001;Hugh-Jones, 2016;Pascual-Ezama et al., 2015), using this measure of unethical behavior and small sample sizes to assess dishonesty by determining whether the percentage of members in each group that reported heads is significantly higher than the percentage of heads we would expect by chance alone (50%). While 54.8% of the treatment group reported flipping "heads" is slightly larger than expected, it is not outside normal expectations (95% CI [46.1, 63.5]) and we cannot reject the null hypothesis that the proportion reporting heads is 50%. In contrast, 59.4% of the control group reported flipping "heads." This is outside of normal expectations (95% CI [50.9, 67.9]) and we can reject the null hypothesis that the proportion reporting heads is 50%. The higher than expected reporting of "heads" in the control group but not the treatment group suggests that group differences may be significant in much larger samples.
To test the impact of the prosocial intervention on ethical behavioral intentions, we report the means and standard deviations, by condition, in Table 3. Similar to previous findings using the same treatments and measure (Christensen & Wright, 2018), we observe no difference (p > .05) in the prosocial and control conditions on either the eight ethical scenarios commonly faced by college students or the hypothetical job negotiation. Given that values are often used as a proxy for behavioral intentions and prosocial interventions may strengthen prosocial values, we also tested for, but failed to find, evidence of any differences between the two groups (p > 0.05) on self-reported PSM.
While we found no evidence that the prosocial condition increased ethical intentions or prosocial values--often thought to be associated with ethical behavior--it is worth noting that self-reported PSM is significantly correlated (p < 0.05) with measures of ethical intentions ( Table 1). Given that previous cross-sectional studies have found a consistent relationship between PSM and ethical intentions, we conducted a series of OLS regression analyses to test whether the relationship holds after controlling for other factors that may influence ethical intentions. The results are reported in Table 4. Although the findings support expectations of a statistically significant relationship between PSM and both measures of ethical intention, the models explain only a small amount of the variation. Self-reported PSM only explains 4% of the variation in the intention to behave unethically in the college student scenarios and 1% of the variation in the intention to behave ethically in the job negotiation scenario.

Discussion and Conclusion
Our study largely rests on null findings. Our failure to find strong relationships between our intervention and ethical behavior or intention could be driven by weaknesses in the intervention, measures, or sample size. Alt- hough we noted a number of steps taken to minimize these weaknesses, we cannot say that prosocial interventions do not influence ethical behavior or intentions, only that we failed to find evidence of a relationship across any of our outcome measures. By placing our results in the context of previous studies, however, our findings underscore an emerging pattern and inform our understanding of the efficacy of prosocial interventions. Consistent with extant scholarship, our study reports mixed findings regarding the effectiveness of prosocial interventions. On the one hand, our findings are consistent with other studies that find a relationship between self-reported PSM and ethical intentions (Caillier, 2017;Christensen & Wright, 2018;Wright et al., 2016). On the other hand, our findings are also consistent with other past experimental and quasi-experimental studies in that it failed to find evidence that prosocial interventions can influence ethical intentions or behavior (Christensen & Wright, 2018;Olsen et al., 2019;Peng & Li, 2019).
These findings do, however, contradict those of another recent study (Meyer-Sahling et al., 2018) that used a different prosocial intervention and a different measure of ethical intention. While differences in the prosocial interventions and ethical intention measures may explain these variations, we think that it is unlikely given our use of a more theory-driven and intensive four-stage intervention and multiple-item measures of ethical intention. A more likely explanation for these inconsistent findings is that the current study's sample (n = 254) is only a fraction of the sample size (n = 4,763) used in that previous study. Thus, it is possible that a relationship exists but that the effect size is small and could only be detected if the current study were replicated with a much larger sample. Another possible explanation is that studies are more likely to find a relationship between prosocial interventions and ethical intentions than between prosocial interventions and ethical behavior. Such a pattern would be consistent with research showing that changing behavioral intentions does not always produce corresponding changes in behavior (Webb & Sheeran, 2006). Some evidence even raises the possibility that ethical intentions are not always strong predictors of ethical behaviors such as whistleblowing (Mesmer-Magnus & Viswesvaran, 2005). The intention-behavior connection certainly warrants further attention in future work (Hassan & Wright, 2020) especially the role that social desirability may play in inflating responses about sensitive behaviors and the ways in which its effect can be minimized (Jensen, 2020). Considering these dynamics all together, one very plausible practical conclusion is that even if prosocial interventions can be successful, such interventions may not be a very easy or a very potent strategy to change ethical behavior.
Our findings also provide additional information regarding important questions on whether PSM can change (e.g., be increased) or whether PSM is more stable (like a trait) but can be activated. Our current study failed to find evidence that the prosocial intervention can be used to increase participant self-reported levels of PSM. Using the same prosocial four-stage intervention, Christensen and Wright (2018) found evidence that it enhanced self-reported PSM in only one of three samples. Admittedly, that sample was substantially larger than the other two samples in that study; it was also substantially larger than the sample used in the present study. Similar to the mixed findings on ethical intentions, one plausible explanation for this may be that any effect of the intervention in activating PSM is so small that it can only be seen in large samples. Even if that were true, however, the evidence suggests that prosocial interventions may not be a very easy or a very potent strategyparticularly as they may only trigger social desirability responses or have very temporary effect. In this way, our findings seem to support previous research suggesting that an individual's PSM is a fairly stable property that is only likely to change slowly over time (Vogel & Kroll, 2016) or after intense experiences (Braender & Andersen, 2013).
In conclusion, our study provides additional information that can help inform two streams of PSM research. First, consistent with past research, our results suggest that self-reported PSM can predict, if not influence, ethical intention. This may, in turn, increase ethical behavior in public sector organization but only when supported by incentive systems and culture Hanna & Wang, 2017). Unfortunately, our study was not able to test for a relationship between self-reported PSM and ethical behavior but previous findings using similar measures of dishonesty have been mixed. While one study found self-reported PSM predicted cheating on a dice-based measure of cheating , another found such a relationship in all three samples on a bribery simulation game but only one of three study samples on a dicebased measure of cheating (Gans-Morse et al., 2019). Second, similar to past studies, we were not able to identify or confirm a specific mechanism by which PSM can be used by organizations and supervisors to increase ethical behavior. Using a theory-driven and multistage prosocial priming intervention, we were unable to increase ethical behavior or intentions. This suggests that PSM may be difficult to cultivate or activate. In fact, we found no evidence that the prosocial intervention increased individual PSM levels. One possible interpretation is that PSM is a relatively stable concept that is both difficult (or slow) to change and yet still related to ethical behavior. Even so, future research is certainly warranted to identify any conditions under which priming may be more or less effective (Cesario, 2014).

Notes
1. 32 responses took less than 9 minutes; these were excluded because we deemed the time too low to have adequately participated in the treatment/control interventions. Another 23 were dropped for nonsense responses or responses of less than 50 words. Excluding these did not change results. Even with the full sample of n = 309 the cheating was always in the 54-55% for the treatment group and 59-62% for control; PSM also consistently predicted the intention measures in a regression.