Frank C. Gentner & Paul H. Cunningham

University of Dayton Research Institute - Human Factors Group (UDRI-HFG)
Dayton, OH 45469-0150, USA

In an effort to identify existing and potential Measures of Effectiveness (MOEs) for the US Air Force Medical Service (AFMS), US Armed Services medical-related universal and mission essential tasks were collected, analyzed, and organized. Although the US military Universal Joint Task List contains many medical-related tasks and metrics, the final draft of the Air Force-specific Task List (AFTL) contained only three major tasks with medical or health concerns specifically named. In comparison to other Services, the AF tasks are written at a more general level. Consequently, many "non-medical" tasks could apply to the AFMS. This paper highlights recent efforts to facilitate Major Air Command (MAJCOM) and unit personnel in constructing Mission Essential Task Lists (METLs). To this end, we developed an Access database that contains tasks and metrics matched from the AFTL and the Joint and Service task lists. Recommendations for using the database for METL development and for identifying metrics of mission performance are discussed. Finally, the paper describes a proposed follow-on study to collect the needed additional MOEs from total quality management metrics, civilian efforts, and from initial local efforts to develop well-formulated candidate METLs then to categorize them into easy-to-use tables organized by aspects of medical care.

The Human Systems Center’s Developmental Plans (HSC/XR), the Air Force Research Laboratory’s Human Effectiveness Directorate, Warfighter Training Research Division (AFRL/HEAA), and the Air Force Medical Support Agency (AFMSA) collaborated with UDRI-HFG to identify, analyze, and organize existing Measures of Effectiveness (MOEs) and Measures of Performance (MOPs) which might be suitable to evaluate Air Force Medical Service (AFMS) functions. In this study, the Joint Service medical-related mission tasks were matched with those of each individual military Service and also identified associated MOEs/MOPs. Using a Microsoft Access database of mission tasks and metrics AF tasks and metrics were compared with those of other Services. After a brief explanation of mission tasks and the DoD and AF approach to universal and tactical METLs, we will highlight the method and findings to date.

A mission task is a "discrete event or action, not specific to a single unit, weapon system, or individual, that enables a mission or function to be accomplished—by individuals or organizations" (CJCSM 3500.04A, 1998, p 1-5). Mission tasks must be discrete, or different and separate from other mission tasks. However, the relationships between a particular task and multiple other tasks may be complex (i.e., one task supports or enables other tasks). This concept is the basis for the development of tasks found in the CJCSM 3500.04A, Universal Joint Task List (Draft, 1998) and AFDD 1-1, Air Force Task List (1998). When tasks are used in association with specific missions of particular organizations, they must become more detailed in definition.

Conditions. Conditions are variables of the environment, or situation in which a unit, system, or individual is expected to operate, that affect performance. Conditions are classified by (1) physical environment (e.g., climate, outer space, or specific terrain), (2) military environment (e.g., threat, command relationships), and (3) civil environment (e.g., political, cultural, and economic factors).

Standards or MOEs. MOEs provide a measure of how well an organization or force must perform a task under a specific set of conditions for a specific mission. The measures express the minimum acceptable proficiency required for a particular task. These MOEs are called standards. These standards, when linked to conditions, provide a basis for planning, conducting, and evaluating a mission or training event. During the collaborative METL development process, an organization selects measures from the list, modifies them as needed, or can create different measures as needed.

The AFTL is structured on the institutionally accepted Air Force core competencies (AFDD 1-1, 1998 p. 19):

A supporting task is a "specific activity that contributes to the accomplishment of a MET. Supporting tasks are accomplished at the same command level or by subordinate elements of a force." An enabling task is a "specific activity that makes it possible to accomplish a MET" (AFDD 1-1, 1998, p.2). Tasks must be judiciously selected because if all enabling tasks were listed as METs in each organization, the list would become excessively long, unmanageable, and thereby of little or no utility to the organization. AF Medical Service tasks are primarily enabling tasks for the rest of the Air Force. In other words, without these tasks the force might not be able to continue to perform, and without the assurance of prompt and effective treatment when needed, morale and retention might reduce mission task performance in all areas.

Two primary types of mission task lists exist: the (1) universal and the (2) tactical or METLs. The Chairman of the Joint Chief of Staff has a universal task list entitled the Universal Joint Task List (UJTL), and each Service has both a universal and a METL.

A METL is a complete list of tasks considered essential to the accomplishment of assigned or anticipated missions for a particular organization. A METL includes associated conditions and standards and may identify command-linked and supporting tasks. A METL differs from a universal task list in that it is specific to a particular organization. METLs are to be developed by MAJCOMs, Numbered Air Forces, and wings.

The Joint and Service task lists (both UTLs and METLs) may be used to support education and training, test and evaluation, operational planning, and readiness assessment.

At the MAJCOM level, performance measures assess how well the command is accomplishing METLs in support of Air Force goals. Subordinate organizations (Numbered Air Forces, wings) develop their own strategic plans, which support the MAJCOM- and Air Force-level strategic plan by identifying their own measurable tasks on which to focus attention (see Figure 1). Those goals will also focus attention on quality issues related to the organization’s METL. The mission performance process, explained in Volume 2 of the Air Force Strategic Plan, includes three components, each essential to operationalizing quality: Strategic Plans, Task Lists, and Measures. At the MAJCOM and unit level, progress in moving toward the AF goals will ultimately be assessed by how well each unit accomplishes its mission essential tasks. AF Operations (AF/XO, 1998) message 171257Z Aug 98, MAJCOM METL Development, requires MAJCOMs and units down through the wing level to develop their own unique METL by 15 Jan 99, and these METLs and metrics are to be furnished to the MAJCOM Inspector General (IG) for inclusion in their evaluation process.

In the process of determining whether medical-related Air Force Mission Essential Task Lists (METLs) existed in the field, we polled all MAJCOM and selected base-level AFTL points of contact to determine whether they had developed MOE/MOPs that might be useful to other units. For most units, July 1998 was too early in the METL development process to find METLs, since the AFTL directive AFDD 1-1 had not yet been signed by the Air Force Chief of Staff. Air Education and Training Command (AETC) had developed a sample set of METLs and metrics for AETC units (AETC, 1998), but these samples were not inclusive of all AFMS missions. We located only a few units that had begun work, and some were using the approach of amplifying the AF/SG goals as their METL outline. On 17 Aug 98, AF/XO mandated that MAJCOMs and organization down through wing level develop METLs by 15 Jan 99.

Air Force medical-related tasks can be divided into those specifically mentioned with medical, rescue, or health functions and those that must be performed by many functional communities, including the Air Force Medical Service.

To identify the similarities and differences between medical tasks, medical-related tasks from the Joint and each Service into an Access database were oriented so that comparisons could be made. Table II presents a sample of this comparison. Note that the Joint, Army, and Navy universal task lists are almost identical in the medical area, and that they contain many specific medical items. However, the Air Force task list is much more generic, with medical tasks subsumed under higher level tasks. Thus the AFTL will require the MAJCOMs and lower organizations to develop their own medical Mission Essential Tasks (METs) and metrics.

The broad nature of the Air Force Task List (AFTL) does not provide a structure or robust niche for developing Air Force medical-related METLs. Therefore, some early medical METL developers are using the Air Force Surgeon General’s (AF/SG) goals as the structure to organize their unit METLs. The match of tasks and metrics developed by other Services that is presented by the Access database should be useful to METL developers. In addition, other sources, such as civilian medical evaluation criteria, might be useful in developing AF METLs at MAJCOM level and below. When AF METL work does begin in earnest, because of the broad nature of the AFTL, it appears that the AFMS and MAJCOM/SGs should suggest a structure to assist medical personnel with their task of developing local METLs, as the other Services have done for their subordinate units. The AF/SG goals include:

Provide Support to Employed Forces and Returning Casualties

Build a Managed Care System that Integrates Quality, Cost and Access

Be the Leader of Comprehensive and Integrated Programs of Disease Prevention, Health Promotion and Fitness

Promote a Safe and Healthful Environment

Provide a Responsive and Sensitive Health Care Atmosphere

These goals provide a good starting place for developing AF medical METLs.

As part of this study, we also conducted an abbreviated literature search for medical MOEs/MOPs developed in the civilian community that might be useful to the Air Force Medical Service. Three organizations performing health care surveys and storing healthcare information were identified as possessing medical performance data. These organizations included the Joint Commission on Accreditation of Hospitals (JCAHO), the Health Plan Employer Data and Information Set (HEDIS), and the Foundation for Accountability (FACCT). The criteria developed by these organizations can be used to help identify more detailed AF METL metrics.

The fact that most AFMS units had not begun METL-development and since MAJCOMs and organizations down through wing level must develop METLs by 15 Jan 1999, there the possibility that many of these organizations will need assistance in developing substantive medical-related METLs and metrics. Further research in this area could assist them with developing descriptive tasks and meaningful metrics. Many of the products developed could assist AF METL developers with deriving descriptive METLs and metrics, especially if refined by additional work.

AFTL and AF METL development should be viewed as an iterative process in which each time through, the METL becomes more comprehensive and useful. During this iterative process, it is important to see the work of other units in a consolidated way so that their work can be used as "memory joggers" to the process. AFMS units should have the benefit of a consolidated list of good ideas to choose from.

We suggest that experts in this area be tasked and funded to construct a more comprehensive collection of medical mission tasks and metrics than was possible with the limited time and funding available for the pilot study. The proposed study could collect additional MOEs and MOPs from Total Quality Management metrics, civilian efforts, and from initial MAJCOM and local efforts to develop METLs, then flesh-out and refine these metrics into categories of medical care. These metrics could be organized into a taxonomy, catalogued for easy access and sharing. This resulting listing of metrics could be used as a "shopping list" for unit-level METL developers. It is anticipated that such a study could assist AFMS personnel with developing improved METLs and metrics by reducing the effort required from each field unit.

AF/XO (1998). All MAJCOM Message 171257Z Aug 98, MAJCOM METL Development. Washington, D.C.: AF Operations.

AFDD 1-1, Air Force Doctrine Document: Air Force Task List. (1998). Washington, D.C.: Department of the Air Force. ). [On-line]. Available: http://www.usafdoctrine.maxwell.af.mil/doctrine/info/1-1.htm

Air Education and Training Command (AETC) (1988). Task lists & mission essential task lists including AETC task list and AETC mission essential task list (METL) (Draft 04 June 1998). [Online] available: http://www.do.aetc.af.mil/DO_WEB/DIVISNS/aos/metl/metl.htm#draft

Air Force Surgeon General (AF/SG) (1998). AF/SG Goals. [On-line] Available: http://usafsg2.satx.disa.mil/~ccx/graphics/2sggoals.gif

Ballard, J.R. & Sifers, S.C. (1995) JMETL: The Key to Joint Proficiency. Joint Forces Quarterly [on-line] http://www.dtic.mil/doctrine/jel/jfg_pubs/2209.pdf.

Best, P.S., Gentner, F.C., Cunningham, P.H., Schopper, A.W., & Morris, J.D. (1997). Evaluating Warfighter (Aircrew and Maintainers) Performance in Aeronautical Systems Using Measures of Effectiveness, (CSERIAC RA-97-005). Wright- Patterson AFB, OH: Crew System Ergonomics Information Analysis Center (CSERIAC) – University of Dayton Research Institute (UDRI).

CJCS Manual 3500.04A, Universal Joint Task List (DRAFT). (1998). Washington, D.C.: Department of Defense, Chairman of Joint Chiefs of Staff (CJCS).

CJCS Manual 3500.04A, Universal Joint Task List. (1996). Washington, D.C.: Department of Defense, Chairman of Joint Chiefs of Staff (CJCS).

DA Pamphlet 11-XX, Army Universal Task List (DRAFT). (1998). Washington, D.C.: Department of the Army.

Gentner, F.C., Cunningham, P.H., Cummings, M.A., & Tailor, C. (1998). Comparison of Measures of Effectiveness for Military Medical Services (Interim Report) (CSERIAC TR-98-004). Wright-Patterson AFB, OH: Crew System Ergonomics Information Analysis Center (CSERIAC) – University of Dayton Research Institute (UDRI). [On-line] Available: http://usafsg2.satx.disa.mil/~ccx/library/medmoe.pdf

OPNAV Instruction 3500.38, Universal Naval Task List. (1996). Washington, D.C.: Department of the Navy.