A closer look at the relationship between justice perceptions and feedback reactions: The role of the quality of the relationship with the supervisor

Citation FEYS, Marjolein; LIBBRECHT, Nele; ANSEEL, Frederik; and LIEVENS, Filip. A closer look at the relationship between justice perceptions and feedback reactions: The role of the quality of the relationship with the supervisor. (2008). Psychologica Belgica. 48, (2-3), 127-156. Research Collection Lee Kong Chian School Of Business. Available at: https://ink.library.smu.edu.sg/lkcsb_research/5727

Two field studies were undertaken to investigate the nature of the relationship between justice perceptions and feedback reactions. Previous work suggests that the relationship between procedural justice and feedback reactions is mediated by the quality of the relationship with the supervisor. However, there are also good theoretical reasons to hypothesise that the relationship between justice perceptions and feedback reactions is moderated by relationship quality. Across two field studies, we found support for both mediated and moderated relationships. Results of the moderator analyses showed that the positive relationship between justice perceptions and feedback reactions was more pronounced for subordinates in a low-quality relationship with their supervisor. The present results provide useful suggestions for enhancing feedback reactions in organisations.
Providing feedback to employees (i.e., giving people information about the outcomes of their achievements with the purpose of stimulating development and learning) is believed to be essential for maintaining and increasing employee motivation and satisfaction (Jawahar, 2006). Although meta-analytic results have shown that feedback interventions do not always increase performance, they suggest that these interventions improve performance on average (Kluger & DeNisi, 1996). In organisations, the assumption also prevails that giving employees feedback is essential to improve individual and organisational performance. Traditionally, organisations rely on performance appraisal and performance reviews as vehicles for giving employees feedback, thereby improving their performance (Gregory & Levy, 2008).
Although almost every large organisation uses some type of performance appraisal method, there seems to be quite some dissatisfaction about performance appraisal. Lawler (1994, p. 106) aptly summarised this discontent-ment: "The problem -and it is well documented -is that most performance appraisal systems do not motivate individuals nor guide their development effectively". In recent years, there have been calls to conduct more research about the efforts organisations can undertake to turn performance appraisal into a more effective feedback intervention (e.g., Levy & Williams, 2004). In this regard, it seems especially important to improve our understanding of why and when employees are inclined to accept and use feedback given to them. Only when employees are prepared to use and accept the feedback they receive during performance appraisal, it can be expected that performance appraisal leads to employee development (Anseel, Lievens, & Levy, 2007).
The present paper aims to gain a better insight into the factors that can enhance feedback reactions in performance appraisal. Recent research suggests that two principles are of key importance. On the one hand it seems important for employees to establish a good relationship with their supervisor (the provider of feedback). On the other hand it is crucial for employees to have the feeling they are treated in a fair manner during the performance appraisal (Elicker, Levy, & Hall, 2006;Leung, Su, & Morris, 2001). In this study, we will explore both a mediated and moderated model explaining the relationship between these two principles (relationship quality and procedural justice perceptions) and feedback reactions. While previous research suggested that relationship quality and justice perceptions have (indirect) main effects on feedback reactions supporting a mediated model, we believe that theoretical work would also be supportive of other interrelationships. More specifically, we hypothesise that maybe the influence on feedback reactions exerted by one factor depends on the influence of the other factor. In other words, we will not only test mediated main effects, but we will also examine the interaction effect between relationship quality and procedural justice on feedback reactions. In order to guarantee the robustness and generalizability of our findings, we examine the hypotheses in two different field studies with different types of performance appraisal and different operationalisations of the variables studied. A better understanding of the role of the two assumed principles (relationship quality and procedural justice) in determining feedback reactions following performance appraisal may enable practitioners to develop strategies for improving performance appraisal in organisations.

Importance of feedback reactions for development
The way employees react to their supervisor's feedback has been shown to be a key determinant of future employee motivation and development (Cawley, Keeping, & Levy, 1998). Keeping and Levy (2000) concluded that the reactions of feedback receivers are probably the best criterion to evaluate performance appraisal systems. Hence, it is not surprising that feedback reactions have already been examined in numerous organisational contexts (e.g., selection, 360°-feedback, assessment centres, etc.) (for an overview, see Anseel & Lievens, 2006). Theoretical models concerning the feedback process (e.g., Ilgen, Fisher, & Taylor, 1979) suggest that two types of feedback reactions are important in determining employee development after feedback, namely feedback acceptance and perceived feedback utility. Feedback will result in development and improved performance only if employees are willing to accept and use feedback for further development (Ilgen et al., 1979;Kinicki, Prussia, Wu, & McKee-Ryan, 2004).
Although all these factors have been found to influence feedback reactions, probably the most important factor in feedback interventions is the sign of the feedback message (Anseel & Lievens, 2006). Feedback sign (favourable or unfavourable) is important because it has a tremendous influence on how employees respond to performance appraisals (Landy & Farr, 1980). This finding is in line with the assumptions of "self-enhancement" theory, which asserts that individuals react more favourably to positive appraisals than they do to negative appraisals (Schrauger, 1975). People are motivated to elevate the positivity of their self-conceptions and will do anything to protect their self-concepts from negative information. People are concerned with increasing the positivity of the self as a means for achieving a high level of self-esteem (Sedikides & Strube, 1997). In organisational research, this assumption has also been supported: positive feedback leads to more favourable employee feedback reactions, whereas negative appraisals cause dissatisfaction (Anseel & Lievens, 2006;Bannister, 1986;Brett & Atwater, 2001;Facteau, Facteau, Schoel, Russel, & Poteet, 1998;Halperin, Snyder, Shenkel, & Houston, 1976;Illies, De Pater, & Judge, 2006;Stone & Stone, 1985;Tonidandel, Quiñones, & Adams, 2002). Although feedback sign is an important predictor of employee reactions towards feedback, it does not offer many developmental strategies for practice. Thus, besides feedback sign, it is important to look for situational factors that can be controlled by the organisation. Given the great importance of feedback sign as a predictor of feedback reactions, this factor will be included as a control variable in all analyses.

Procedural justice perceptions
The fairness of performance appraisals has been identified as an important criterion in judging their effectiveness and utility for organisations (Erdogan, 2002). Research investigating the effects of justice in organisations is typically grouped under the name organisational justice theory (Greenberg, 1987;1990). Colquitt, Conlon, Wesson, Porter, and Ng (2001) investigated the dimensionality of organisational justice and found evidence for four distinct types of justice. In the organisational justice literature, initially a distinction was made between distributive justice and procedural justice (Adams, 1965;Deutsch, 1975;Homans, 1961;Leventhal, 1976). Later on researchers introduced two other factors of organisational justice, namely interpersonal and informational justice, that are both grouped under the name 'interactional justice', defined as the interpersonal treatment people receive as procedures are enacted (Bies & Moag, 1986). Distributive justice deals with the fairness of the distribution of tangible outcomes. Conversely, procedural justice focuses on the fairness of the procedures to achieve those outcomes (Greenberg, 1987(Greenberg, , 1990; see also Bies & Moag, 1986, for other justice principles). In other words, procedural justice refers to the fairness perception of the means by which outcomes are allocated, but not necessarily to the outcomes themselves (Cropanzano, Bowen, & Gilliland, 2007). An important requirement for feedback to be accepted is that the procedures used during performance appraisal are perceived to be fair and just (McDowall & Fletcher, 2004). If an employee is treated fairly by a supervisor or an organisation, he or she is more likely to perceive the feedback to be accurate (Leung et al., 2001;Reis, 2002). Various studies have confirmed that procedural justice is important in the context of performance appraisals. In the 1970s, researchers found that many employees perceived their organisation's performance appraisal to be unfair (Levine, 1975). Furthermore, employees perceived the appraisal system to be fairer when they got the opportunity to express their feelings ('voice') (Landy, Barnes, & Murphy, 1978;Landy, Barnes-Farrell, & Cleveland, 1980). Recent studies (Jawahar, 2007;Kavanagh, Benson, & Brown, 2007;Roberson & Stewart, 2006) have provided further evidence that in a performance appraisal context there is a positive relationship between procedural justice and the motivation to improve performance following performance appraisal. From a practical perspective, we expect procedural justice to be the type of justice that is most controllable by the organisation. Organisations can easily control the procedures by which employees receive feedback, or standardise rules as to how performance appraisals should be conducted. Because of this and the aforementioned evidence concerning procedural justice in shaping reactions to feedback, in these studies we will focus solely on investigating the relationship between this type of justice, relationship quality and feedback reactions.

Relationship quality
Research suggests that, in addition to fairness, a good relationship between employees and the supervisor providing feedback is crucial for feedback acceptance. In this regard, leader-member exchange theory refers to the quality of the relationship between supervisor and subordinate (Graen & Scandura, 1987). This theory suggests that supervisors determine what role employees will fulfil in the organisation (Graen, 1976). These roles define the quality of the relation between supervisor and subordinate (Lind & Zmud, 1991;1995). According to the LMX-model (Dienesch & Liden, 1986;Liden, Sparrowe, & Wayne, 1997), employees who are trusted by the supervisor are allocated more important roles to fulfil than employees whom the supervisor has a less favourable relationship with. In one of the first studies examining this relationship, LMX was found to be an important predictor of employees' reactions to performance feedback. Employees who reported a personal and trusting relationship with their supervisors, reported more positive reactions to feedback, while an impersonal and less trusting relationship between supervisor and subordinate led more to negative reactions (Snyder, Williams, & Cashman, 1984). Kacmar, Zivnuska, Witt, and Gully (2003) found in a study of 188 private sector workers that employees in a high-quality LMX relationship received higher performance appraisals than employees in a low-quality LMX relationship. In a sample of managers, Russel and Goode (1988) reported that satisfaction with the supervisor was related to performance appraisal satisfaction. Giles and Mossholder (1990) also reported a high correlation (r = .61) between supervisory satisfaction and performance appraisal satisfaction (see also Jawahar, 2006).

Mediation hypothesis
Although there seems to be relative consensus that both relationship quality and justice perceptions play an important role in shaping feedback reactions after performance appraisal, less is known about the specific interplay of these two factors in determining feedback reactions (e.g., van Knippenberg, De Cremer, & van Knippenberg, 2007). One group of studies suggests that high procedural justice is an antecedent of high relationship quality. Leung and colleagues (2001), for example, examined whether high interpersonal justice, which is often considered to be an aspect of procedural justice (Bies & Moag, 1986;Tyler & Bies, 1990), has an impact on an employee's attitude towards his or her supervisor. In two studies, fair feedback led to a more favourable attitude towards the supervisor, and both these variables were related to feedback acceptance. These findings provide support for the assumption that procedural justice can improve employees' relationship with the supervisor.
Another stream of studies suggests that high-quality relationship may be associated with honest and just behaviour by the supervisor. Elicker and colleagues (2006) found evidence for a relationship in this direction. They developed a theoretical model of justice perceptions during the feedback process and found that the relationship between relationship quality and feedback reactions was mediated by the perception of voice in the appraisal process and the perception of distributive, interactional and procedural justice. Thus, favourable feedback reactions following performance appraisal in a high-quality relationship could be ascribed to how employees were treated during the performance appraisal and the subsequent justice perceptions. Because these findings were based on cross-sectional data, we should, of course, be careful in drawing any conclusions about causal relationships (see Elicker et al., 2006).
Finally, Sparr and Sonnentag (2008) found that LMX was a mediator in the relationship between justice perceptions and employee well-being following the feedback process. They found that procedural justice (besides distributive, interpersonal and informational justice) led to improved LMX, which, in turn, increased subordinate well-being. As in the study by Elicker et al. (2006), true causality between fairness and LMX could not be concluded. Drawing on these findings, and as can be seen in Figure 1, we will explore whether the relationship between procedural justice perceptions and feedback reactions is mediated by relationship quality. The following hypothesis is formulated: Hypothesis 1a: The positive relationship between procedural justice perceptions and feedback acceptance and perceived utility will be mediated by relationship quality.

Moderation hypothesis
As discussed above, although studies have shown that relationship quality and perceived justice are important elements in feedback reactions following performance appraisal, few studies have examined how these factors are interrelated and exert an influence on feedback reactions. In previous studies examining the link between relationship quality and procedural justice, the correlation between both variables ranged from .38 to .50 (Elicker et al., 2006;Masterson, Lewis, Goldman, & Taylor, 2000;Sparr & Sonnentag, 2008). This correlation suggests that, although these variables are highly related, the effects of relationship quality and procedural justice on feedback reactions are by no means identical, and the interaction effect between both these variables may also be important. So, from an empirical point of view, apart from a mediated path through relationship quality, we believe that the effect of procedural justice on feedback reactions might also be moderated by the level of relationship quality.
This moderated relationship is not only possible from an empirical point of view, but theoretical work also seems supportive of such a relationship. Relational theories of procedural fairness for example predict that fair procedures signal to the employees that they are respected and accepted by group members (De Cremer & Tyler, 2005;Tyler & Blader, 2000. In this manner, procedural fairness is interpreted as a social influence process and there exists considerable evidence that people are more influenced by people that are alike or that are valued by them than by other people (Goethals & Nelson, 1973). This leads us to expect that procedural fairness will have more pronounced effects with low levels than with high levels of relationship quality. If employees do not have an a priori good relationship with their supervisor, the signalling function of fair procedures may become more important. Based on these insights from organisational justice theory we expect that a subordinate who has an unfavourable relationship with the superior (low relationship quality), but who feels that the performance appraisal was correctly conducted (high procedural justice), will be satisfied with the feedback and be more willing to accept and use it. From a practical perspective, high procedural justice seems to protect the subordinate from unfair negative feedback ratings from a malicious rater or from any unwanted 'political' rating behaviour that would arise during the performance appraisal process. Conversely, we expect that, in case everything did not go according to the rules during performance appraisal (low procedural justice), this can be compensated by a good supervisory relationship (high relationship quality). When procedural justice is low, the amount of relationship quality the employee has towards the supervisor can provide the employee with the certainty that the feedback received will not be disproportionately unfair. Having a good relationship with one's supervisor may appear to be a good protection to 'political games' in performance appraisal. In short, we expect feedback acceptance to be especially low when perceived justice is low and when relationship quality is low. When perceived justice is high or relationship quality is high, we expect more favourable feedback reactions. Figure 2 depicts this second hypothesis: Hypothesis 1b: The positive relationship between performance appraisal justice perceptions and feedback acceptance and perceived utility will be moderated by relationship quality. This positive relationship will be more pronounced for subordinates in a low-quality relationship with their supervisor than for subordinates in a high-quality relationship.

Study 1
The first study was conducted in a local division of a multinational global technology company. More specifically, we measured employee perceptions of feedback after they went through a performance appraisal discussion. In this first study, we focused on feedback acceptance as the dependent variable.

Participants and procedure
In the company we conducted this study, all employees have an annual performance appraisal review in which they receive feedback by their direct supervisor on their achievements of the last year. We informed 565 employees about the study via e-mail. The next week a cover letter discussing the study and containing an internet link to the actual questionnaire was e-mailed to the employees. Two weeks later a reminder was e-mailed. Study participation was voluntary. Two hundred and thirty-five employees completed the questionnaire, yielding a response rate of 41.60%. Due to a technical problem with the electronic data collection we had usable data only for 219 participants.
Most of the participants were men (82.6%) and the participants' ages ranged from 18 to 65 years (M = 30). Participants had an average tenure of 10 years in the company (SD = 8) and an average experience of 6 years in their current position (SD = 6). Most of the participants (62%) had their last performance appraisal review 8 months prior to the study and 8% had their last performance appraisal review 20 months prior to the study. The performance appraisal review for the other 30% of the respondents took place between 21 and 25 months prior to the study. Control variables Several studies found that the longer people work for an organisation, the less open they are towards receiving feedback. In other words, these studies found that organisational tenure is negatively related to feedback seeking behaviour (Ashford, 1986;Ashford & Cummings, 1985;VandeWalle, Challagalla, Ganesan, & Brown, 2000). Given these findings, years of tenure in the company, years of experience in the current position, and the moment of the last performance appraisal review were included as control variables in our analyses. These variables were assessed with single item measures that asked participants how many years of tenure they had in the organisation, how many years of experience they had in their current position, and when they had been given their last performance appraisal review. Gender and age were also included as control variables. In addition, we controlled for feedback sign in the first step of the regression as previous research indicated that negative feedback engenders unfavourable feedback reactions (Anseel & Lievens, 2006). We used three items to measure positive feedback and three items to measure negative feedback (Steelman, Levy, & Snell, 2004).
Respondents were asked to response to these items using a 7-point Likerttype scale ranging from 1 (strongly disagree) to 7 (strongly agree). Sample items are "When I do a good job at work, my supervisor praises my performance" for positive feedback and "In the performance session my supervisor tells me when my work performance does not meet organisational standards" for negative feedback. Internal consistency was .83 for positive feedback and .82 for negative feedback.

Procedural justice
A four-item procedural justice scale developed by Keeping, Makiney, Levy, Moon, and Gillette (1999; see also Keeping & Levy, 2000) was used. This procedural justice scale is specific to the performance appraisal context. Responses were made on a 7-point Likert-type scale, ranging from 1 (strongly disagree) to 7 (strongly agree). A sample item from this scale is "The procedures used to evaluate my performance were fair". The internal consistency of this scale was .96.

Leader-member exchange (LMX)
To measure the quality of exchange between supervisors and subordinates, we used the seven-item Leader-Member Exchange (LMX7) scale (Scandura & Graen, 1984). The LMX7 scale focuses on the nature of the general working relationship between an employee and his/her supervisor and is by far the most frequently used LMX measure (Gerstner & Day, 1997). In their meta-analysis, Gerstner and Day (1997) showed that the LMX7 measure has sound psychometric properties. The LMX measurement consisted of seven questions with 5-point Likert-type scales, with 1 indicating a bad relationship with the supervisor and 5 indicating a good relationship with the supervisor. A sample item is "How would you characterise your working relationship with your supervisor?". The internal consistency estimate of this scale (.91) was similar to the one of previous studies (i.e., Elicker et al., 2006;Scandura & Schriesheim, 1994).

Feedback acceptance
Acceptance of feedback refers to employees' belief that the feedback given during the performance appraisal review is an accurate portrayal of his or her performance (Ilgen et al., 1979;Kinicki et al., 2004). We used six items of Stone, Gueutal, and McIntosh's (1984) measure of feedback accuracy (Elicker et al., 2006;Keeping & Levy, 2000), which are typical items for measuring the extent to which employees perceived the evaluation as accurate. Employees indicated their responses on a 7-point scale, ranging from 1 (strongly disagree) to 7 (strongly agree). A sample item from this scale is "The feedback was an accurate evaluation of my performance". The internal consistency of this scale in the current sample was .88.

Results
Descriptive statistics, correlations, and internal consistency reliabilities for all study variables are presented in Table 1. We first conducted confirmatory factor analyses to examine the distinctiveness of our constructs (e.g., LMX, procedural justice and feedback acceptance). In a first model all three observed variables (i.e., LMX, procedural justice and feedback acceptance) were posited to load on a single latent factor. Conceptually, this model does not distinguish between the three observed variables. The second model hypothesised two distinct yet intercorrelated latent factors, wherein LMX and procedural justice were hypothesised to load on the first latent factor, and feedback acceptance was hypothesised to load on the second factor. Conceptually, this model distinguished between the two independent variables and the dependent variable. The third model hypothesised three distinct yet intercorrelated latent factors, wherein LMX was hypothesised to load on the first latent factor, procedural justice was hypothesised to load on the second latent factor, and feedback acceptance was hypothesised to load on the third latent factor. Conceptually, this model distinguishes between the three observed variables and considers them as measures for different constructs. The results of this analysis can be found in Table 2. The one-and two-factor  models showed no outstanding fit to the data. Conceptually however, this could be expected as procedural justice and LMX are measuring different constructs and were hypothesised to load on one factor. The three-factor model, however, fitted the data significantly better than the one-factor model in both studies, so we can conclude that the three constructs measured were empirically distinct from each other.
To test for the mediating effect of LMX (Hypothesis 1a), we used Baron and Kenny's (1986) multi-step regression procedure. We first controlled for the demographic and control variables and for feedback sign. Then, a 3-step analysis was conducted by (a) regressing the mediator (LMX) on the independent variable (procedural justice), (b) regressing the dependent variable (feedback acceptance) on the independent variable (procedural justice), and (c) regressing the dependent variable (feedback acceptance) on both the independent (procedural justice) and mediator (LMX) variables. According to Baron and Kenny (1986), in order for complete mediation to occur: (a) the independent variable must affect the mediator in the first equation; (b) the independent variable must be shown to affect the dependent variable in the second equation; (c) the mediator must affect the dependent variable in the third equation; and (d) the independent variable must no longer be significant in the third equation. As can be seen in Table 3, the independent variable (procedural justice) significantly predicted the mediator (LMX) (β = .23, p < .001). Procedural justice also affected feedback acceptance (β = .45, p < .001), as did LMX when controlling for procedural justice (β = .22, p < .01). Furthermore, as can be seen in Table 3, the significant positive effect of the independent variable (procedural justice) did not disappear when the mediator (LMX) was taken into account (β = .40, p < .001). Thus, the independent variable (procedural justice) remains significant when including LMX. To further test the mediation effect, a direct test of the full mediational path (Procedural justice perceptions -> LMX -> Feedback reactions) was conducted using a Sobel test (Sobel, 1982). Results of the Sobel test showed that the indirect path from procedural justice to feedback acceptance (z = 2.52, p < .05) was significantly different from zero, which is indicative of mediation (Baron & Kenny, 1986). Hence, Hypothesis 1a was partially supported. The impact of procedural justice on feedback acceptance was partially mediated by LMX.
Next, we conducted a hierarchical multiple regression analysis to see whether the interaction term was significant as proposed by Hypothesis 1b. We again controlled for the demographic and control variables and for feedback sign in the first step. In the second step, the main effects of the two centred independent variables (i.e., LMX and procedural justice) were entered in the equation. Finally, in the third step the interactive term computed using the centred variables of LMX and procedural justice was entered. As shown in Table 3 Mediation analyses of procedural justice, LMX, and feedback acceptance in Study 1 (N=219)  Table 4 Summary of hierarchical regression analysis of LMX and procedural justice on feedback acceptance in Study 1 (N=219) Step 1  Table 4, feedback sign explained a significant and substantial amount of variance of feedback acceptance (∆R 2 = .43, F(8, 203) = 18.74, p < .001). LMX and procedural justice perception explained a significant additional variance above these control variables (∆R 2 = .16, F(10, 201) = 28.44, p < .001). As hypothesised, the interaction between LMX and procedural justice perception was also significant (β = -.15, p < .05) and explained 1% of the variance in feedback acceptance above the previous predictors (∆R 2 = .01, F(11, 200) = 26.87, p < .05). Although 1% additional variance explained is rather modest, some authors (e.g., Aguinis, 1995in Haworth & Levy, 2001McClelland & Judd, 1993) noted that an interaction that accounts for as much as 2% of the variance is impressive for interactions in field studies, and that such trends, especially at an exploratory stage, should not be ignored. Based on this information, we believe it is warranted to conclude that even one additional percent of variance is notable.
Hypothesis 1b predicted that the positive relationship between justice perceptions in performance appraisal and feedback acceptance is moderated by LMX and that this positive relationship would be more pronounced for employees in a low-quality LMX relationship. To determine if the pattern of the interaction was consistent with our hypothesis, we plotted the interaction in Figure 3 where low LMX was presented as the mean of LMX -1SD, and high LMX was presented as the mean of LMX + 1SD (see O'Connor, 1998). In addition, simple slopes analyses were performed through special macros developed by O'Connor (1998). The standardised regression coefficients of the simple slopes were β = .52 (p < .001) for low LMX and β = .38 (p < .001) for high LMX: they are thus both positive and significantly different from 0. As can be seen from Figure 3, the low-LMX slope is, as hypothesised, a little bit steeper than the high-LMX slope, though it is clear that this difference is quite small. As predicted by Hypothesis 1b, Figure 3 reveals that the relationship between procedural justice and acceptance of feedback was slightly more pronounced for individuals with low LMX. In conclusion, the results from this first study thus show support for a (partially) mediated as well as a moderated model as both Hypothesis 1a and 1b were supported.

Figure 3
Interaction of LMX and procedural justice on feedback acceptance (Study 1)

Study 2
To examine the generalizability and robustness of the results obtained in Study 1, we tested our hypotheses in a different context. In Study 2, we examined reactions on received feedback from performance monitoring in a call centre. In performance monitoring, supervisors monitor how their employees perform by observing, examining, and/or registering their work behaviours, with or without technological assistance (Brewer & Ridgway, 1998;Stanton, 2000). In this context, calls were monitored by supervisors according to fixed procedures and employees received feedback afterwards. Electronic performance monitoring is making strong inroads in practice. For instance, more than 65% of companies surveyed by the American Management Association used employee monitoring or surveillance (Orthmann, 1998), and over 75% of large American companies electronically monitor their employees (Alder, 2001). As shown by Brewer and Ridgway (1998), monitoring seems to play an important role in the development and maintenance of effective work performance. For performance monitoring to be effective, it is crucial that employees are satisfied with the performance appraisal review system and perceive it as fair (Stanton, 2000).
Research revealed that employees express fear towards performance monitoring if it is unclear how the data will be used (Stanton & Julian, 2002). These results point to the importance of feedback in a performance monitoring context. In addition, performance monitoring is one of the forms of performance appraisal that leads to the most stress among employees (Hedge & Borman, 1995). Therefore, perceived feedback utility by employees is crucial in such a context. Thus, conducting this second study allowed us to test our hypotheses with perceived feedback utility as the dependent variable.

Participants and procedure
The study was conducted in a Belgian market research company. The company has its own call centre where employees call consumers at random to inquire information about certain products or services. During the data collection period, 135 employees worked for the call centre of which 30 came to the centre on a daily basis. The questionnaire was administered with a web-based (intranet) survey. Questionnaires were completed by 90 employees. Due to a technical problem with the intranet that was solved within one day, 7 questionnaires could not be used for further analyses (response rate = 61.48%).
Most of the respondents were female (54.2%) and the respondents age varied from 18 to 57 years (M = 23, SD = 6). Experience is expressed here as the number of shifts the employees had worked in the call centre. In this call centre, employees can work in a morning or evening shift. Employees' experience varied from 4 to 792 shifts, with an average of 157 (SD = 195).

Control variables
As in Study 1, we controlled for experience (expressed in number of shifts) and feedback sign (positive/negative) of the last performance appraisal review, in addition to demographic variables (gender and age).
Performance appraisals were gathered on review cards. These cards consist of 28 items which are scored by the rater with -1 (bad), 0 (average) or +1 (good). The individual item scores were added up and a total evaluation score was formed, ranging between -28 and +28. The feedback was mainly negative when the total score was negative and mainly positive when the total score was positive. Sample items used in the performance appraisal are "marking answers", "improvisation ability", and "pronunciation".

Procedural justice
According to Williams and Levy (1998), the perceived justice of the performance appraisal is determined by the insight one has in the performance appraisal system. The authors report a correlation of .54 between system knowledge and perceived justice. This led them to the conclusion that system knowledge is an important antecedent for procedural justice. In accordance with Williams and Levy (1992), we also use the Perceived System Knowledge (PSK) or the knowledge of the performance appraisal system as an indicator of perceived procedural justice. We used seven items of the scale developed by Williams and Levy (1992) that were specific to the current context. Respondents rated their agreement with each statement using a 7-point Likert-type scale, ranging from 1 (strongly disagree) to 7 (strongly agree). A sample item from this scale is "I understand how the performance appraisal system works". The internal consistency of this scale was .71. However, a confirmatory factor analysis found a better fit of the data when this variable was measured using six instead of seven items. Therefore, we omitted one item from this scale and conducted our analysis with the six remaining items. The internal consistency of the six-item scale was .75.

Supervisory trust
We used four items (Korsgaard & Roberson, 1995) to measure the level of trust respondents have in their supervisor. This measure allowed us to assess the quality of exchange between supervisors and subordinates. Employees responded to the items using a 7-point Likert-type scale ranging from 1 (strongly disagree) to 7 (strongly agree). The internal consistency of this scale was .84. A sample item is "I trust my manager".

Feedback utility
Perceived utility was measured with four items developed by Greller (1978). Each item was rated on a four-point scale: (1) I do not feel this way at all, not at all, (2) I feel somewhat like this, a little, (3) I feel generally like this, pretty much and (4) I feel exactly this way, completely. The internal consistency of this scale was .91. A sample item is "The appraisal helped me learn how I can do my job better". Table 5 presents the descriptive statistics, correlations, and internal consistencies of the Study 2 variables. In this study we conducted the same confirmatory factor analyses on the constructs measured as we did in Study 1.

Results
Here as well, a 3-factor model fitted the data significantly better than a 1-factor model: the assumed independence of the constructs measured was thus supported in this study as well. Results of these analyses can be found in Table 2. As was already mentioned, these analyses showed that a better fit was obtained when 'procedural justice' was measured using six instead of seven items. Thus, one item was omitted from the procedural justice scale.
To test Hypothesis 1a we conducted the same mediation analysis as in Study 1. In the first step, we controlled for the demographic variables and feedback sign (see Study 1). As can be seen from Table 6, justice was significantly related to supervisory trust (β = .38, p < .01) and to feedback utility (β = .31, p < .01). Supervisory trust significantly predicted feedback utility, controlling for justice (β = .32, p < .01). The addition of the mediator reduced the size of the direct effect of justice on feedback utility (β = .19, p > .05) and reduced the effect to non-significance, suggesting full mediation. We again conducted a Sobel test of the mediational path (Justice -> Supervisory trust -> Feedback utility). Results showed that the indirect path from justice to feedback utility (z = 2.22, p < .05) was significantly different from zero. These results imply that the independent variable (justice) affects the dependent variable (feedback utility) indirectly, through the mediating variable (relationship quality). Thus, Hypothesis 1a was supported.
Next, we conducted a hierarchical multiple regression analysis to examine Hypothesis 1b. As in Study 1, we controlled for the demographic variables and feedback sign in the first step. As can be seen in Table 7, the interaction term reached significance (b = -.21, p < .05) and the model with the interaction variable explained significantly more variance than the model with only the main effects of procedural justice and trust (∆R 2 = .05, F(7, 75) = 4.28, p < .05). This means that the level of trust in the supervisor moderated the relationship between procedural justice and feedback utility.
To determine whether this interaction was consistent with our hypothesis, as in Study 1 we used the O'Connor (1998) method. Here as well, the standardised regression coefficients of the simple slopes were calculated using the special macros developed by O'Connor (1998): only the one for low supervisory trust appeared to be significantly different from 0, namely β = .46 (p < .01). The high-trust coefficient did not reach significance: β = -.03 (p > .05). As can be seen from Figure 4, the low-trust slope is indeed steeper than the slope for high supervisory trust as the high-trust slope is not sig nificantly different from 0. Figure 4 thus reveals that there is a positive Table 6 Mediation analyses of procedural justice, supervisory trust, and feedback utility in Study 2 (N=83) Note. IV = Procedural justice; Mediator = supervisory trust. All paths are standardised betas. N = 83. Age, gender, experience and feedback sign were controlled for in all analyses. *p < .05, **p < .01, ***p < .001 Table 7 Summary of hierarchical regression analysis of supervisory trust and procedural justice on feedback utility in Study 2 (N=83) Feedback utility Step 1  relationship between feedback utility and justice if there is a low level of trust in the supervisor. When the employees have little trust in their supervisor, the perceived justice of the performance appraisal system has to be high to consider the feedback as useful. Thus, feedback is considered as useful if one of both predictors is high. Hence, Hypothesis 1b is supported in Study 2. In sum, results of Study 2 were also in support for both a mediated and a moderated model.

Discussion
The present study's aim was to examine organisational factors that may enhance feedback reactions in performance appraisal because feedback reactions are an important condition for employee development. On the basis of recent feedback literature, we expected two variables to be of main importance: on the one hand, the quality of the relationship with the supervisor, and on the other hand the perception of procedural justice. We explored two alternative models (a mediation and a moderation model) that may explain the interplay between procedural justice, relationship quality, and feedback reactions.
Across the two field studies, we found evidence for a (partially) mediated relationship between procedural justice and feedback reactions through relationship quality as hypothesised in previous models. Furthermore, the results of the two studies indicated that a moderated model (i.e., an interactive effect of procedural justice and relationship quality) explained a significant amount of variance in the dependent variables. Thus, the interplay between the variables under study suggests that, in line with our theoretical underpinnings, a moderated effect might also be a viable mechanism relating procedural justice to feedback reactions. Interpreting this moderated model shows that in order for feedback to be considered as useful and acceptable, it is necessary to have a perception of high procedural justice, especially when the quality of the relationship with the supervisor is low. That is, high justice can compensate for a low level of relationship quality: when the quality of the relationship with the supervisor is low, feedback will be more considered as useful when there is high procedural justice. It seems that when employees have a lower level of relationship quality, they are more influenced by perceptions of high procedural justice than when relationship quality is high. In this sense it seems that a high level of procedural justice becomes more important when there is a low level of relationship quality, supporting our hypothesis. The fact that these results were found in two different studies examining two different types of performance appraisal systems and using different operationalisations of the variables, adds to the robustness and generalizability of our results.
Given that both models were to a large extent supported in both studies, this leaves us in a difficult position to draw conclusions. Both models might be viable explanations for the interrelationships under study as our current results do not allow one to conclude that one model fits the data better than the other. Therefore, we believe the value of this study lies in its demonstration of the need for more additional empirical and theoretical work refining the effects of relationship quality and justice in performance appraisal. Our findings imply that it might be useful to go beyond the assumed mediated relationships between both variables and feedback reactions, and that their effects may be to some extent interdependent. However, our results clearly await further replication. Future research should therefore scrutinise possible interactive effects. Our results may also be a basis for refining current theoretical models. In the models of Elicker et al. (2006) and Roberson and Stewart (2006), for example, justice is depicted as a mediator of the relationship between relationship quality and feedback reactions, as well as of the relationship between feedback accuracy and motivation to improve performance. Based on our findings, it seems possible to refine the model of Elicker et al. (2006) by also considering relationship quality as a moderator that alters the relationship between justice and feedback reactions. In the model of Roberson and Stewart (2006), it might be worthwhile to add relationship quality as a moderator in the postulated relationship between justice and the motivation to improve performance. Thus, our findings can help other researchers to explore interaction effects between relationship quality and justice and to pay more attention to moderated, next to mediated, relationships when developing new models of feedback reactions.
Our results may also have important implications for organisational practice. The two variables that influence feedback acceptance, namely relationship quality and procedural justice, are controllable by organisations. Thus, organisations can plan interventions to improve relationship quality as well as (perceptions of) justice. Supervisors, for example, can be trained in building a better relationship with their employees, and companies can stimulate activities that increase mutual trust (e.g., social activities). Giving employees the opportunity to express their feelings and giving them voice may help in creating a procedural justice climate. Furthermore, supervisors can be trained in the correct use of the procedures and criteria relevant for feedback giving (e.g., consistently applying transparent appraisals).
Of course, given the limitations of our research design, we have to be careful when drawing conclusions. A first limitation is that both studies were conducted using a cross-sectional design. Therefore, it is impossible to draw causal connections between the different variables. As suggested by Elicker et al. (2006), longitudinal and experimental studies are necessary to extend the current knowledge regarding procedural justice, relationship quality, and feedback reactions. A second drawback is our reliance on self-reported measures. Although subjective perceptions and feelings are important, our results need to be confirmed by using objective measures of feedback utility and accuracy, relationship quality, and justice. In addition, we did not use an actual measurement of the subsequent employee development. Clearly, an examination of the degree to which employees take the feedback they receive into account and actually participate in development activities is an important issue for future research. Furthermore, we used a global measure of pro-cedural justice. Future research might investigate the relationship between the separate components of procedural justice and feedback reactions. Finally, we considered only relationship quality as a leadership characteristic. It would be interesting to investigate whether other leadership characteristics are also related to feedback reactions. One such characteristic for example could be charismatic leadership. One feature of a charismatic leader is that he or she communicates high performance expectations to employees, and expresses the confidence that these employees can attain those expectations (House, 1977;Shamir, House, & Arthur, 1993). Following this we expect that employees who work for a charismatic leader feel valued and respected, and will therefore be inclined to accept the feedback they receive from their leader. As the leader emphasises his/her expectations, employees will be confident that the feedback provided will be instrumental in reaching those expectations. For this reason, we expect that there might be a strong relationship between charismatic leadership and feedback reactions and that this variable may also compensate for low procedural justice.
In conclusion, in two studies we found that the effect of justice on feedback reactions in performance appraisal might not only be (partially) mediated by relationship quality, but that a moderated relationship is a viable explanation as well. Results from the moderated relationship, show that for feedback reactions to be favourable it is important to have a high procedural justice especially when relationship quality is low. On the one hand these findings show that further refinement of the current theoretical framework is necessary. On the other hand they offer organisations useful strategies for improving performance appraisal in practice.