In 1995, in response to direction from the Commander of the Army, a research team conducted a series of focus groups with some 900 service personnel to evaluate "morale in the Army." In their 1995 final report entitled "Modernization of the quality of life in the Army--Final report," the team offered a total of 110 recommendations (later refined into 39 critical content areas) considered significant for the improvement of quality of life (QOL) in the Army.

Between March and July 1996, Commander LFC issued a series of five action directives in which he stated his intent to change what he could with a view to improving QOL in the Army. At the same time, he directed that subordinate commanders take what actions they could.

In 1997, the Personnel Research Team (PRT) was commissioned to assist in the development of indices by which the effectiveness of improvements could be measured. In response to this request, Eyres (1997) recommended that PRT's "Conditions of Service Questionnaire" (COSQ), now known as the Quality of Life (QOL) Survey, become the primary tool with which to gauge the success of measures implemented. A number of additional measures were also recommended for evaluation. These included:

a. the thirteen "flag-items" from the Human Dimensions of Combat Readiness Index-Experimental (HDCRI-X) questionnaire (Reeves and Hanson, 1989). These items were selected from among the questions of the HDCRI-X as those which best reflect the following aspects of combat readiness from a human perspective:

(1) Personal morale (2 questions);

(2) Cohesion (2 questions);

(3) Leadership (3 questions);

(4) Ideology (2 questions): the extent to which respondents tend towards an occupational (low ideology) or a vocational (high ideology) perception of the military profession;

(5) Professional morale (2 questions); and

(6) Small group morale (2 questions);

b. a series of items developed by PRT (PRT items) intended to address respondents' perceptions of leadership within their unit. Separate questions were asked with respect to the following aspects of leadership attributes and behaviours of unit Master Corporals, Senior NCOs, Junior Officers, and Senior Officers:

(1) their leadership skills;

(2) their technical skills;

(3) their management skills;

(4) their ability to help subordinates resolve problems;

(5) their interest in helping resolve subordinates' problems;

(6) their day to day visibility in the unit;

(7) their accessibility; and

(8) their differential treatment by the military justice system; as well as

c. a single question to assess respondents' perceptions of the speed with which their unit handles grievances and complaints.

The intent of including the PRT items was to determine the degree to which they overlapped the information gained from the HDCRI-X. A low degree of overlap would indicate that they contributed information that the HDCRI-X did not, in which case the items should be retained. A high degree of overlap, on the other hand, would suggest that the items were redundant and could be eliminated from the questionnaire.

The flag-items and the PRT items were integrated into a fifty-item multiple choice questionnaire entitled "The Canadian Army Leadership Questionnaire" which is shown at Annex A. Respondents were asked to indicate their opinions on each question using five-point Likert-type scales (1 = "Strongly disagree", 2= "Slightly disagree", 3 = "Neither agree nor disagree", 4 = "Slightly agree", and 5 = "Strongly agree"). The Canadian Army Leadership Questionnaire and the COSQ were administered to a cross-section of 1810 LFC personnel during the 1997 administration of PRT's "Personnel Survey."

This paper presents the results of initial investigations of the use of these measures to assess subordinates' perceptions of leadership in the Canadian Army and some of the implications for the evaluation of operational effectiveness.

Among the symptoms of loss of confidence in leadership cited in the final report on modernization of QOL were "lack of leadership experience and training on the part of leaders" and perceptions that "leaders have lost touch with the soldiers." The authors cited their perception of:

These observations suggested as a working hypothesis that evaluations of leadership at various rank-levels differ as a function of the perceivers' own rank. It was predicted that subordinates’ evaluations of their superiors' leadership-related behaviours would be significantly lower than those reported by their superiors at all rank-levels. Specifically:

A random sampling technique was used to select a group of respondents that was representative of the demographic characteristics of the Army (by gender, rank, and first official language). Selected respondents were subsequently mailed a copy of the Personnel Survey along with a pre-addressed return mailing envelope. Respondents, who were advised that their participation was voluntary and anonymous, were asked to complete the questionnaire, seal it in the pre-addressed envelope, and place it in their unit's out-going mail.

Of the 1810 questionnaires mailed out, 913 were returned to PRT for analysis, a return rate of 50.4%; this is considered a good return-rate for a mail-out questionnaire. Biographic items within the Personnel Survey were used to determine the demographic characteristics of the respondent sample. These are presented in Table 1 below. In one respect, the sample is not considered representative of the Army at large: women were deliberately over-sampled to ensure that meaningful comparisons could be made between genders for the purposes of other questionnaires from the Personnel Survey.

General. For ease of comprehension and analysis, Respondent Rank was collapsed into four categories (Junior NCMs: Private to Master Corporal inclusive; Senior NCMs: Sergeant to Chief Warrant Officer inclusive; Junior officers: Officer-Cadet to Captain inclusive; and Senior officers: Major to General inclusive).

Statistical adjustment of the sample. Because women were over-sampled to ensure a sufficient number completed the personnel survey, it was necessary to adjust the sample statistically to make it representative of the demographic characteristics of the Army. This was done by using accepted re-weighting techniques available within SPSS.

The incremental contribution of the PRT items. Using the CANCORR macro instruction from SPSS, a cannonical correlation analysis was performed to calculate the information overlap or "redundancy " (Cooley & Lohnes, 1971) of the five scales emerging from the principal component analysis over the information already available from the six scales of the HDCRI-X. The analysis extracted three significant cannonical variates, the redundancy measures for which were summed to yield a measure of overall redundancy (Cooley & Lohnes, 1971, p. 172). Values of 20.1% for redundancy of the PRT items given the HDCRI-X scales, and 22.5% for redundancy of the HDCRI-X scales given the PRT items were obtained.

Factor structure of the PRT items. A principal component analysis was performed on the 37 PRT items using SPSS 6.1.3. VARIMAX rotation yielded the following five interpretable factors accounting for 59.6% of the variance:

a. Senior officer leadership attributes and behaviours;

b. Junior officer leadership attributes and behaviours;

c. Senior NCO leadership attributes and behaviour;

d. Master-Corporal leadership attributes and behaviours; and

e. Perceptions of disparate treatment by the military justice system of the various rank-levels. Because the focus of the present paper is on evaluations of leadership, the issue of the military justice system will not be discussed further.

All relevant leadership attributes and behaviours were observed to load on their respective rank-related leadership factor.

It had been intended to use the factor-structure to select those items with the strongest factor loadings for inclusion in future evaluations of QOL; a fairly demanding criterion of .7 was established as a cut-off which yielded the following pattern of leadership-related items:

Because of the commonalities in the patterns of various items loading, however, it was decided to create a scale score at each of the four rank-levels (MCpls, Snr NCOs, Jr officers, and Snr officers) for leadership attributes and behaviours that consisted of respondents' evaluations of: (a) leadership skills; (b) management skills; (c) technical skills; and (d) problem-solving skills.

Predictably, the reliabilities for the five constructed scales was quite high. Cronbach's a ranged from .86 (Jr officer leadership composite score) to .88 (Snr NCO leadership composite score). These are considered to reflect good scale reliabilities.

Table 2 presents the mean scores and standard deviations for Jr NCM, Snr NCO, Jr officer, and Snr officer respondents’ leadership composite scores for each of the four rank-levels under consideration. A graphic representation is also given in Figure 1.

Planned comparisons. While the distribution is properly multivariate, it was not considered appropriate to use a multivariate analysis of variance (MANOVA) due to considerations of heterogeneity of variance. For this reason, non-parametric testing (Mann-Whitney U) was used throughout. In an attempt to control Type-1 error, family-wise error-rate was set at .05. The outcomes of these comparisons are given below:

Overall, the three working hypotheses were confirmed. Whether through disinterest, complacency, uncritical evaluation, or even unconsciously through defence-mechanisms, superiors at all levels gave higher leadership ratings than did subordinates. This is not surprising: inferences about leadership frequently arise from observation of outcomes, particularly task-accomplishment that occurs as a result of the exercise of command or legitimate authority. It may also occur, however, not because of, but rather in spite of the exercise of influence.

Figure 1.  Mean Response to the four Composite Leadership Scales for each of the four Respondent Rank-levels

Dissatisfaction with leadership in the Canadian Army is not a new phenomenon. Nearly twenty years ago, Cotton (1979) cited the high degree of cynicism regarding the sincerity of officer claims that they will support their men; for, more often than not, the officer is moved somewhere else when individual soldiers require counsel or administrative support. (Soldiers) tend to define leaders as taskers, who see their men simply as means to the completion of short run tasks. (p. 40)

Research in a number of other military communities across the last half-century has illustrated similar patterns: nominal leaders and the nominally led differ in their opinions of whether leadership is effective or even happening at all. Some examples are illustrative.

In their post-Second World War study, "The American Soldier," Stouffer et al (1949) reported a consistent pro-organizational bias in US Army officers' assessments of subordinates' opinion. They reported a tendency for the officers to believe that their subordinates' opinions on given issues were more positive than anonymous questionnaires showed them to be. Thirty-five years later, citing

the 1984 results of an annual survey in the US Army, Gabriel (1985) reported similar findings:

He concluded that "...the army seems to admit that its studies of morale and leadership have reached the same conclusion for a decade: soldiers do not hold their leaders, officers or NCOs, in particularly high regard" (p. 22)

Korpi (1965) observed similar results in the Swedish Army. In a study of Swedish conscripts and their leadership cadre, he observed that leaders at all levels from platoon sergeants to company commanders consistently tended to over-estimate the favourableness of subordinates' responses to 21 morale-related questions. Not only was there an average absolute error of 22-25% per question, the degree of positive bias was generally observed to increase with rank/position. Significantly, having asked leaders to rate their confidence that they had accurately assessed opinion in their unit, he found that the more confidence they expressed in their accuracy, the less accurate their perceptions of morale actually were.

This convergence of findings across a number of military forces and a number of cultures highlights a significant shortfall in Canadian leadership doctrine, at least, which defines leadership as the process whereby subordinates are influenced to accomplish the mission in the manner desired by the leader. Significantly, the doctrine does not specify the nature of the influence process; in this regard, it fails to distinguish between leadership and command, and between willing following, indifference, and grudging compliance. Overall, the present study suggests that, like Gabriel’s (1984) soldiers, Canadian NCMs, notably junior NCMs, do not hold their officers in particularly high esteem. In general, NCMs appear to be at best indifferent towards the leadership demonstrated by Junior officers and appear only slightly more positive about Senior officer leadership. All ranks, on the other hand, appear to be satisfied with leadership shown by MCpls and Snr NCOs.

The present study indicates that individual leadership attributes and behaviours are not perceived in the same fashion when exercised at different rank-levels. Not surprisingly, the small degree of overlap between the HDCRI-X items and the four PRT composite leadership scales suggests that the latter measure aspects of leadership not addressed by the grosser leadership measure within the HDCRI-X.

These findings suggest the utility of subordinate evaluations of leadership within the Army as an aide to assessing operational readiness. Recently, the Canadian Army has taken measures to address this particular issue in the form of the "Unit Climate Profile" (Wild, 1988; Farley, 1997), a 62–item questionnaire intended to permit Commanders at all levels to assess soldiers’ perceptions of the following aspects of morale and leadership within their unit:

a. Personal morale;

b. Small group morale;

c. Professional morale;

d. Overall leadership;

e. Section-Commander leadership;

f. Platoon Warrant Officer leadership;

g. Platoon Commander leadership; and

h. Company Commander leadership

The UCP, does not, however, address leadership at the levels addressed within the present study (MCpl, Snr NCO, Jr officer, and Snr officer). The data presented here suggest that additional information is obtainable through those measures.

Derived from the HDCRI-X, the UCP was devised with the intent of providing Commanders at all levels with an objective assessment of the human aspect of operational readiness within their unit. Intended for anonymous group administration, it is administered by a military psychologist who introduces the questionnaire and its intent, and who deals with any questions. The typical respondent takes approximately 30-40 minutes to complete the questionnaire.

To date, the UCP has been administered to three rotations of the Canadian Contingent in Bosnia, and two contingents in Haiti. During the early administrations, data were entered manually, and analyzed by means of SPSS. A recent innovation has been the use of scannable electronic forms that reduces the delay between administration and feed-back to Commanders from several weeks to two to three days including report-writing.

Feedback debriefing takes the form of a graphical representation of the climate profile that illustrates the unit’s mean-score on each of the eight climate measures and a written report. Normally, separate profiles are prepared for company-sized sub-groups. Commanding officers are debriefed on the climate profiles of each of their sub-units, while sub-unit commanders receive feed-back only on their own sub-unit. To date, the nature of the debriefings on the results have been at the discretion of the superior commander: some have indicated their preference to be debriefed on "climate" within their contingent while others have deliberately chosen to have the outcomes remain the subject of confidential discussions between the military psychologist and individual Commanding Officers.

On operations, soldiers and commanders alike have given the UCP mixed reviews. Most soldiers greet it enthusiastically as an opportunity to provide their own perspective on leadership and morale within the unit—they ask questions before the session begins and afterwards are frequently interested in amplifying their answers or providing suggestions for future administrations; several suggestions indicating the failure to cover items of particular concern to a given respondent, for example, resulted in the inclusion of the open-ended response section. The more cynical, whether about questionnaires or about leadership, see the UCP as another meaningless exercise that ultimately will have no impact on their lives. This is probably more reflective of leadership practices within the unit than it is of the utility of the UCP itself.

Commanders, too, are of mixed opinion. Some are curious, even enthusiastic about the opportunity to get a more objective picture of what soldiers are thinking and what is happening within their unit. Others, sceptical and unconvinced, and (in the light of the present data) possibly over-confident, see the UCP as an usurping their authority and a needless impediment to operations. Yet others are concerned that the introduction of any subordinate evaluation of leadership will generate Commanders who will cater to their subordinates" rather than attend to their duty to lead.

But if some commanders are cynical about the potential of the UCP, the Commander of the Army is not. He has recently directed the annual administration of the UCP across the Army with the intent of "taking the pulse" of the army as a whole. His intent is to receive only the summary data of the Army as a whole without singling out units or formations.

Summary. The present study suggests that regardless of the reason, leaders in the Canadian Army may not be well-placed to assess the effects on subordinates of their leadership initiatives and those of their subordinate leaders. At all levels, they significantly over-estimate the success and effectiveness of those initiatives and as a result are likely unable to distinguish between effective leadership and command, between following and compliance, with attendant consequences on their units’ operational effectiveness .

One tool is now in place within the Canadian Army to demonstrate the effects of leadership on subordinates in the form of the Unit Climate Profile. The present study suggests, however, that more precision may be obtained by addressing leadership at the MCpl, Snr NCO, Jr officer, and Snr officer levels in addition to those already addressed within the UCP.

The benefits of initiatives to measure leadership are clear. Better leadership translates into increased operational effectiveness. And the present data indicate that subordinates are best placed to evaluate the leadership they experience—because in the end—leadership is not happening unless subordinates believe it is!

References

Annex A

The Canadian Army Leadership Questionnaire