History and Purpose of the ADF Activities Survey

The Chief of the Defence Force commissioned Infuse Pty Ltd to develop the ADF Activities Survey (Activities Survey) in 1991. The objective was to accurately and objectively measure working hours and a number of other factors affecting work within the ADF. The first application of the Activities Survey was carried out from 26 February 1992 to 26 March 1993. Approximately 60 personnel were surveyed each day for this period. The initial report was published in June 1993. From April 1994 the Activities Survey has continued in a reduced form with approximately 30 personnel surveyed each day in what is known as "the Trickle Study".

The objectives of the Activities Survey include providing:

• quantitative data to support future decisions about defence structures, pay, working conditions and other planning issues;
• general management information about work;
• identification of activities and workplaces which may require further study; and
• benchmark measures for work in the Australian Defence Force.

Design and Implementation of the Survey

The Activities Survey was designed to meet the standards for evidence before the Defence Force Remuneration Tribunal. During design and testing the instrument was subjected to intense scrutiny by the Australian Bureau of Statistics (ABS) and the Department of Industrial Relations (DIR) as well as groups within Defence. The result is a robust instrument which can be consistently applied across the whole of the Australian Defence Force.

This industrial relations motivation for the design of the Activities Survey is the basis for the focus on measuring factors that relate (or can relate) to pay and allowances.

The key difference between the Activities Survey and previous studies is that it measures activity rather than asking respondents to judge whether or not what they are doing at a given time constitutes work.

The Survey Instrument

The survey instrument is made up of two scripts, one for the prebrief and one for the debrief, and two forms, the Respondent’s Record and the Record of Interview (ROI).

The two scripts ensure that all respondents are briefed and debriefed consistently. The Respondent’s Record is carried and filled in on the survey day and the Record of Interview is completed by a Unit Survey Consultant (USC) during a debriefing session.

The Record of Interview is a single, folded A3 sheet. On the front of the form Global Qualifiers for the days surrounding the survey day and some broad details about the day itself are recorded. The inside of the form is a matrix where 25 primary activities are recorded against each half hour, with provision for a major and a minor activity within a half hour. Each half hour is further described by up to 62 contextual qualifiers.

Initially there were objections to the layout of the form, particularly from the ABS who advocated the use of a booklet style, like that used for the National Census, on the grounds that is easier to use. In testing it was shown that the booklet style ROI took considerably longer to fill in than the matrix style.

Outline of the Survey Process

Every four months new data from the service personnel systems is loaded to the Activities Survey database and a random sample of ADF members selected. Each person in the sample is randomly assigned a survey date in the next four-month survey period. The program that assigns survey dates distributes them evenly over the survey period. A unique case number is allocated to each combination of an ADF member and a survey date. This case number is used whenever possible in correspondence to help preserve the anonymity of the respondent.

The survey of an individual is conducted in three parts:

• the pre-briefing session when the USC explains what is required from the respondent and hands over the paper work. The pre-briefing occurs as close as possible to the survey date.
• the day of the survey when the respondent uses the Respondent's Record to record their activities each half-hour for the 24 hour period starting at 0600hrs on the survey day.
• the de-briefing session when the USC fills in the Record of Interview (ROI) form using the information on the Respondent's Record supplemented by the questions in the de-briefing script.. The de-briefing session occurs as soon as possible after the survey day.

The USC takes a copy of the completed ROI and sends the original to their PUSC. The PUSC checks the form and if there are no problems or errors, forwards them to the Survey Centre. Any problems or errors found at this stage are resolved by the PUSC and the USC. The copies are stored until advice is received from the Survey Centre that they may be destroyed.

When ROI forms are received by the Survey Centre, they are checked for completeness and accuracy. Correction Signals are sent to the USC responsible for the case to provide any missing information, correct errors and clarify ambiguities in liaison with the survey respondent. Once the forms have been double checked, the information is entered by the Data Entry Operator. The forms are filed and stored until 12 months after the Descriptive Statistics Report (Report 1) for the survey year has been produced. They are then destroyed as classified waste.

Administrative Structure

Activities Survey administration and data entry is done by the ADF Activities Survey Centre (the Survey Centre) in the Directorate of Strategic Personnel Planning and Research (DSPPR) in the Headquarters Defence Personnel Executive (HDPE). Technical support for the survey administration and data analysis is done by the civilian company Infuse Pty Ltd.

There is a network of ADF personnel in units across the country who assist with the Activities Survey:

• Unit Survey Consultants (USCs): the point of contact for survey respondents. They conduct the pre-briefing and de-briefing interviews and complete the Record of Interview and Job Description forms.
• Principal USCs (PUSCs): the principal point of contact in units for the Survey Centre. Each unit has a PUSC who is responsible for all USCs in their unit, or if there are several units in one location, responsible for the USC in that location. PUSCs check survey forms for completeness and accuracy before forwarding them to the Survey Centre, and follow up cases for which an ROI has not be returned.

Much of the survey administration is automated. Extensive use is made of the Defence signals system for communicating with USCs. Signals about cases to be surveyed, movement of cases between units and non-return of survey forms are all automatically generated from the administrative database for the Activities Survey.

Analysis of Data

Return rates

Return rates for the Activities Survey have been consistently high since it started. Some of the reasons for this include the use of the signals system to keep track of people and follow up non-return of survey forms; the interview-based format for collecting information; and the use of the formal chain of command for enforcing compliance. An important part of the success of the survey is the support of the USC network provided by the Survey Centre.

From the start of the Trickle Study in 1994, a "Missed Case" return has been filed for cases which cannot be surveyed on the day. The "Missed Case" return contains information about the status of the survey day: leave type, leave accrual, required to work and shift work.

Table 1Return Rates by Survey Year

Two standard reports are produced for each year of the Trickle Study, one covering Descriptive Statistics for the survey year and one analysing changes between survey years. The annual Descriptive Statistics report contains several different types of analysis:

• analyses of working hours, patterns and activities for the ADF as a whole and each of the three services. Demographic information is combined with the survey data to allow analyses by service and rank. The results are expressed as means, standard deviations and frequency distributions.
• analysis of annual working patterns. This is done by constructing a random set of complete year samples. The complete set of available data is used to construct 1000 samples of 364 survey days from each week of the survey for each Service. Each sample is stratified by day of week so that it contains 52 cases for each day of the week. Aggregate statistics are calculated from each sample and combined to form a set of annual aggregate statistics.

In addition to the regular reporting, supplementary reports are produced to answer a range of research questions. A full list of reports is included at the end of this paper.

Discussion

From 1992 until recently the primary users of the Activities Survey were the Financial Conditions of Service branch and the RAN. Both used the data to inform decisions about working hours, working patterns and remuneration.

The measurements of working hours, patterns and activities obtained using the Activities Survey have a high face validity—that is they are a good match for individuals’ experience. On occasions when the face validity appeared to be poor, further investigation revealed that the results from the Activities Survey were correct. An example of this arose in one of the early reports where the number of field days that would attract Field Allowance in the Activities Survey data only accounted for about 70% of the amount of Field Allowance in pay statistics. Further investigation showed that the rules for Field Allowance were being incorrectly applied.

The success of the Activities Survey is grounded in the thoroughness of the work done to design and test the instrument. Key concepts underlying the final design can be summarised as follows:

• The primary activity descriptors overlap. This means that every activity in a day can be described by at least one of them. Since the survey started in 1992, no respondent has been unable to choose a primary activity to describe what they were doing.
• The form can equally well be describe the day of a clerk and that of someone from the SAS.
• In testing it was shown to be extremely difficult, if not impossible for the USC and the respondent to work out how many hours of work were being recorded. This is achieved by the separation of questions about what activities were undertaken from questions about whether what was done was Part of Job, Job Related or Private.
• The nature of the variables allows for multiple definitions of work.
• A consequence of the requirement for consistency is the level to which everything is defined in the de-briefing script. Something as apparently simple as the question "Were you cold?" is accompanied by this strict definition.

• The matrix on the inside of the form allows for on-the-spot clarification and validation. For example: "What were you doing in the water that was noisy during time periods 5-10?"
• The Global Qualifiers place the survey day in context. This enables the data to be analysed from several different view points.

One of the more interesting applications of the survey instrument was during the deployment of Australian troops as part of the United Nations Transitional Authority in Cambodia. The deployment lasted from September to November 1992 and every person deployed was randomly allocated a survey date in this period. The data gathered allowed quantitative comparison of working hours, activities patterns and conditions for a deployed force with those of the Defence Force as a whole. The analysis shows that deployed personnel worked long hours with little respite in the form of weekend or other leave. They experienced severe working conditions arising from the nature of the deployment, the environment and their working activities. A lower proportion of Officer Rank personnel were stationed in Cambodia than is typical of the Army as a whole. Officer Rank working hours were significantly higher than those of Other Ranks, which contrasted with the normal situation where working hours for the two groups were similar.

Earlier this year, a study of the changes in Army working hours from 1992 to 1997 was carried out for the Chief of Army. The study broke the Army into four groups: Combat, Support, Training and Headquarters. Days worked per annum and daily working hours were analysed by rank band for each group, the Army as a whole and Royal Australian Navy (RAN) personnel posted to ships. The study found that, in the Army:

• except for the Combat group, the average number of days worked per annum is decreasing; and
• except for the Combat and Training groups daily working hours were decreasing.

With the data now available, a much broader range of questions might now be addressed. The issues that could be covered include:

• the relationship between extreme working hours and conditions on attrition and retention;
• the work contexts in which major Occupational Health and Safety hazards occur and the level of use of protective equipment in these contexts;
• the differences in working hours, activities and patterns between men and women in similar jobs;
• the amount of Physical Training and Sport being reported as "Part of Job" and whether this is enough to maintain required fitness levels;
• the differences in sleep patterns between RAN personnel on older classes of vessel and those on newer classes of vessel;
• the change in work patterns when a new commander is posted to a large formation; and
• the amount of time spent on ceremonial activity in different types of units and how this has changed with time.

The above list is a only a selection of the possible issues for investigation. As the ADF moves towards smaller numbers and the consequent requirement for more and more sophisticated workforce monitoring and management, the importance of the research capability provided by the Activities survey will become greater and greater.

Gilks, D., Buckley, R., ADF Activities Survey Standard Report 1, 1993: 1992-3, Descriptive Statistics, June 1993.

Gilks, D., Buckley, R., ADF Activities Survey Standard Report 1a, 1993: 1992-3, Appendices 1-10, 14-19, June 1993.

Gilks, D., Buckley, R., ADF Activities Survey Supplementary Report 1, 1993: Analysis of Working Hours related to Service Allowance Compensation for Excessive Hours, September 1993.

Gilks, D., Buckley, R., ADF Activities Survey Supplementary Report 1, 1994: 1992-3, Annual Analysis of Working Patterns by Force Element, January 1994.

Warn, Major J., ADF Activities Survey Supplementary Report 2, 1994: 1992-3, The Environmental Conditions under with Australian Defence Force Personnel Work, January 1994.

Warn, Major J., ADF Activities Survey Supplementary Report 1, 1994: 1992-3, Hours of Work as Sea in the Navy, January 1994.

Warn, Major J., ADF Activities Survey Supplementary Report 1, 1994: 1992-3, Hours of Work for the Formations in Land Command, January 1994.

James R. Warn, David Gilks, and Robert E. Buckley, Measuring time use and activity patterns, a data matrix approach, 29th Annual Conference, Australian Psychological Society, September 1994.

Robert E. Buckley, David Gilks, and James R. Warn, An information system supporting the ADF activities and working hours survey 1992-1993, Proc. 6th Annual Canberra Conference, Australian Unix Users Group, February 1995.

Gilks, D., Buckley, R., ADF Activities Survey Standard Report 1, 1995: Trickle Study 1994-5, Descriptive Statistics, August 1995.

Gilks, D., Buckley, R., ADF Activities Survey Supplementary Report 1, 1995: 1992-3 and 1994-95, Analysis of Ship to Shore Work and Other Factors, December 1995.

Gilks, D., Buckley, R., ADF Activities Survey Standard Report 1, 1996: Trickle Study 1995-6, Descriptive Statistics, August 1996.

Gilks, D., Buckley, R., ADF Activities Survey Standard Report 2, 1996: Analysis of Change 1992-3, 1994-5 and 1995-6, December 1996.

Watson, T., ADF Activities Survey Supplementary Report 1, 1997: Air Surveillance Operator Musterings, Statistical Information, July 1997.

Watson, T., Gilks, D., Buckley, R., Rayner, C., ADF Activities Survey Supplementary Report 2, 1997: Working Hours and Conditions of Personnel Deployed to Cambodia as members of the United Nations Transitional Authority, September - November, 1992, September 1997.

Gaudry, T., Gilks, D., Buckley, R., Rayner, C. ADF Activities Survey Standard Report 1, 1997: Trickle Study 1996-7, Descriptive Statistics, January 1998.

Tomlinson, L., Gilks, D., Chapman, S., DPSR Research Note 2/98: A Comparative Study of Army Working Hours April 1992 – March 1997, March 1998.

Ward, D. DPSR Research Note 3/98: An Overview of Work Patterns Within 3 Brigade April 1997 – March 1997, March 1998

Gaudry, T. ADF Activities Survey Standard Report 2, 1997: Analysis of Change 1992-3, 1994-5, 1995-6 and 1996-7 April 1998.

Chapman, S., DPSR Research Report 1/98: Operation Damask Activities and Working Hours of RAN Personnel Deployed to the Persian Gulf Aboard HMAS Melbourne June–September 1996, May 1998.

Gaudry, T., Gilks, D., Buckley, R., Rayner, C. ADF Activities Survey Standard Report 1, 1998: Trickle Study 1997-8, Descriptive Statistics, September 1998.