AB-23 - Paper
Air Force Occupational Analysis and Promotion Test Development
for the 21st Century
George Kailiwai III, Lt Col, USAF
Air Force Occupational Measurement Squadron
Randolph AFB TX
Summary
The Air Force Occupational Measurement Squadron (AFOMS) envisions "highly trained, professional and fairly promoted Air Force people who perform the right tasks in the right jobs." To achieve this vision, the AFOMS is automating the Air Force Occupational Analysis Program (AFOAP) and the Air Force enlisted promotion test development process. The AFOMS recently modeled its decisions, actions, and activities through an Integrated Definition Function Modeling Method (IDEF0). Developed from a well-established graphical language called the Structured Analysis and Design Technique (SADT), IDEF0 identified the AFOMS activities, processes, functions, products, and linkages through schematic constructs and conceptual schemata. While IDEF0 effectively described what AFOMS does, the AFOMS is currently using the Data Modeling Method (i.e., IDEF1X) to construct a logical relational database design. By using these two methods, the AFOMS plans to develop a "corporate database" which will fully support its Department of Defense (DoD) Automated Information Systems (AISs) with standardized data and make all AFOMS data accessible to squadron members through an intranet. When connected to the internet, the corporate database will enable the AFOMS to administer and collect Job Inventory (JI) surveys by way of the worldwide web, and also make AFOMS products available to all Air Force and DoD members. The impediments to these automation initiatives are primarily scarce resources as the AFOMS balances its near-term mission accomplishment requirements with long-term benefits. The AFOMS hopes to bring Air Force occupational analysis and promotion test development into the 21st century by overcoming these obstacles.
Introduction
The mission of the Air Force Occupational Measurement Squadron (AFOMS) is to "improve Air Force capabilities through occupational analyses, promotion tests and professional development study guides" (Kailiwai, 1997). To accomplish this global mission, about 160 psychologists, education and training specialists, information managers, computer scientists, and personnelists work together to perform four functions. These functions include analyzing every enlisted Air Force specialty or occupation, identifying the most critical tasks within that specialty, annually revising about 350 enlisted Specialty Knowledge Tests, Promotion Fitness Examinations and USAF Supervisory Examinations, and writing and publishing the enlisted promotion fitness and supervisory examination study guides. Squadron members perform their jobs with the squadrons vision or desired end-state in mind. That vision is "highly trained, professional and fairly promoted Air Force people who perform the right tasks in the right jobs" (Kailiwai, 1997). Although the efforts of the squadron primarily impact over 200,000 enlisted personnel, its mission is expanding to include an increasing number of officer and civilian career fields, and specialties in the US Marine Corps, Navy, Army and Coast Guard which are similar to those in the US Air Force.
The AFOMS is a one-of-a-kind squadron. Although the AFOMS is one of three squadrons in the Headquarters Air Education and Training Command Directorate of Operations (HQ AETC/DO), the squadrons mission transcends across training and readiness boundaries and covers a variety of personnel and enlisted professional development issues. Because of this global mission, the AFOMS is also the single focal point for supporting the Air Staff, other major commands, and the Air Force Personnel Center (AFPC) on occupational and task analyses, the Weighted Airman Promotion System (WAPS), and developing and publishing Air Force Pamphlet 36-2241 (i.e., the Promotion Fitness Examination and USAF Supervisory Examination Study Guides) (AFMS 16008X, 1994). The purpose of this paper is to describe how the AFOMS is bringing the Air Force Occupational Analysis Program (AFOAP) and the Air Force enlisted promotion test development process into the 21st century through automation.
Air Force Occupational Measurement Squadron Information Architecture (AFOMSIA)
The AFOMSIA is in a state of transition. Less than two years ago, the AFOMS migrated from a mainframe environment to a personal computer windows based environment. Today, the AFOMS has one network with over 150 network users connected to a Windows NT server.
The AFOMSIA currently has six internally developed automated information systems (AISs) and one contractor maintained system. These AISs include Task Inventory Analysis Requirements Application (TIARA); Airman Promotion Test Statistical Analysis Section (PROMO); Test Control Officer Network Administration System (TCNET); Mail List (ML); Automated Training Requirements Analysis System (ATRA); and Occupational Data Instrument Processing System (ODIPS). The one contractor maintained system is the Comprehensive Occupational Data Analysis Program (CODAP) suite of software.
The AFOMS uses TIARA to create Job Inventories (JIs) (i.e., the surveys used to collect task data from subject-matter experts). The JI lists typical tasks performed by all members in an AF specialty. PROMO provides historical and statistical data and status of all Air Force WAPS tests. This AIS also produces the promotion test score keys the Air Force Personnel Center (AFPC) uses to select enlisted personnel for promotion to the next grade. PROMO is the foundation for the Air Forces enlisted promotion system. TCNET is the AIS that manages and maintains the USAF Test Control Officer (TCO) network directory and Air Force Personnel Test (AFPT) worldwide testing system. Through TCNET, the AFOMS is able to communicate directly with every TCO in the Air Force to alert the TCO of any changes in test administration. ML randomly selects respondents who will receive JIs. ML generates statistical reports of populations within the USAF, Air National Guard (ANG), and AF Reserve Command (AFRC). ATRA serves as the collection and manipulation tool for task analysis (i.e., the compiliation of what personnel do when they perform tasks in terms of knowledge, skills, tools, references, equipment, and steps required to perform the task). ODIPS produces camera ready copies of occupational survey instruments and formats data from an optical scanner into a format usable by CODAP. Finally, CODAP is a computer-based system specifically designed to input, quantify, organize, summarize, and report data collected with job-task inventories. CODAP contains a set of analysis tools which enable AFOMS occupational analysts to analyze occupational data (e.g., data acquired through JIs.)
Although the current AFOMSIA works well, Department of Defense (DoD) regulatory guidance directs that data should be accessible and standardized. Specifically, DoDD 8320.1, DoD Data Administration, states "5.1 Implement data administration aggressively in ways that provide clear, concise, consistent, unambiguous, and easily accessible data DoD-wide, and that minimize the cost and time required to transform, translate, or research differently described, but otherwise identical data." The DoDD also states, "5.2. Standardize and register data elements to meet the requirements for data sharing and interoperability among Information Systems (ISs) throughout the Department of Defense" (DoDD 8320.1, 1991). With this regulatory guidance in mind, the AFOMS devised its AFOMSIA vision: Corporate database linked to an AFOMS search engine-equipped intranet and world-wide web in hypertext markup language which provides up-to-date standardized data elements derived from integrated engineering methodologies.
This vision contains several characteristics. First, a corporate database will help eliminate redundancy and "stovepipe" systems. With the use of a corporate database, there will be only one point of entry to update and maintain data for all end-users. The added benefit of a corporate database is that the AFOMS will be able to keep up with a rapid rate of change. The AFOMS search engine-equipped intranet will provide the means to automate the AFOAP through a "front-end" into CODAP and an Automated Analyst Request System (AARS). The "back-end" of CODAP will also be written in hypertext markup language (HTML) to facilitate ease of use. The AFOMS intranet will be instrumental in developing the Automated Test Production System (ATPS) and Automated Test Development System (ATDS). Because of test security, ATPS and ATDS will be maintained on a stand-alone local area network (LAN) which will receive data from AFOAP. In addition to occupational analysis data, the intranet will contain two Air Force Promotion Test study guides compiled by the AFOMSAir Force Pamphlet (AFPAM) 36-2241 Volume I, Promotion Fitness Examination Study Guide, and Volume II, USAF Supervisory Examination Study Guide. These study guides will be linked to primary and secondary references. Finally, the intranet will contain biographical information on AFOMS members.
By connecting the AFOMS corporate database onto the WWW, the AFOMS will be able to share data with the entire DoD. Because of the sensitivity of enlisted promotion test data, these shared data will primarily be AFOAP information. Initially, the AFOMS envisions finished products, that is, Occupational Survey Reports (OSRs) and Task Analysis (TA) reports as being available on the WWW. If these products are available in HTML (i.e., an ASCII format), DoD personnel will be able to use comercially available web browsers and search engines to download complete reports or sections of a report. Eventually, the AFOMS hopes to administer JI surveys via the WWW. The timing of JI administration on the WWW will be a function of when Air Force members will have unlimited access to the internet.
The AFOMS has been working to standardize its data elements across the DoD. As stated in DoDD 8320.1, data standardization will promote metadata and data element sharing among AISs both internal and external to the AFOMS. Acton Burnell, a Virginia-based software contractor, has been assisting the AFOMS with the data standardization of TIARA, PROMO, TCNET, ML, ATRA, ODIPS, and CODAP.
Finally, through integrated engineering methodologies, the AFOMS plans to model its activities and data relationships. The AFOMS has completed an Integrated Function Modeling Method effort (IDEF0) in February 1998, and hopes to begin data modeling (i.e., IDEF1X) to model its data relationships. The results and plans for IDEF0 and IDEF1X follow.
Integrated Function Modeling Method (IDEF0)
The first step in automating the AFOAP and the Air Force enlisted promotion test development process was modeling both systems through Integrated Definition (IDEF) methods (KBSI, 1998a). IDEF is a structured approach to enterprise modeling and analysis (KBSI, 1998a). "Through modeling, an organization can understand the behavior of its processes: how they work, why they work, and the results they produce" (deWitte, 1998, p. 2).
IDEF0, or Function Modeling Method, is a method designed to model the decisions, actions, and activities of an organization or system (KBSI, 1998a). IDEF0 was derived from a well-established graphical language, the Structured Analysis and Design Technique (SADT). The United States Air Force commissioned the developers of SADT to develop a function modeling method for analyzing and communicating the functional perspective of a system (KBSI, 1998a, p. 1). Through IDEF0, diagrams based on simple box and arrow graphics describe the inputs, outputs, resources, and constraints that are a part of an activity (deWitte, 1998, p. 2).
With IDEF0, the emphasis is on describing what an organization or system does, as opposed to how it does things. The description of the activities of an organization or system can be easily refined into greater and greater hierarchical detail until the model is as descriptive as necessary for the decision-making at task (KBSI, 1998a). In short, IDEF0 captures the elements needed to execute a process, models relationships vice steps, identifies core processes, and identifies redundant, non-value-added processes (deWitte, 1998, p. 2).
The AFOMS modeled its organization through IDEF0 in February 1998 when modelers from the Headquarters Air Education and Training Command Directorate of Communications and Information (HQ AETC/SC) Computer Systems Squadron (CSS) interviewed functional experts from the AFOMS. Beginning with the AFOMS top-most activities, the modelers then logically "decomposed" lower level activities. Through these structured interviews, the modelers identified 9 "top level" activities: 1) Develop Occupational Survey; 2) Develop Occupational Survey Report (OSR); 3) Develop Officer OSRs and Special Projects; 4) Develop Task Analysis; 5) Process Data; 6) Print/Distribute Surveys; 7) Manage Weighted Airman Promotion System (WAPS) Test Development; 8) Produce Study Guides; and 9) Support AFOMS Activities. Grouping the first 6 top level activities into "Perform Occupational Analysis" resulted in the following IDEF0 diagram for the AFOMS (Figure 1):
Figure 1. IDEF(0) of AFOMS Top Level Activities
The CSS modelers defined the activities in the AFOMS through a total of 91 box and arrow diagrams. Two to three hierarchical levels generally provided the necessary level of detail, however, the modelers needed a total of four levels to describe 5 complex activities. While IDEF0 effectively described what AFOMS does, the next step is to use a method to design the AFOMS corporate relational database. IDEF1X, the Data Modeling Method enables the AFOMS to design a relational database with a syntax designed to support the semantic constructs necessary in developing a conceptual schema.
Data Modeling Method (IDEF1X)
IDEF1X is most useful for logical database design after the information requirements are known and the decision to implement a relational database has been made (KSBI, 1998b). It is also helpful for modeling logical data types through the use of a classification structure or generalization/specialization construct (KSBI, 1998b, p. 2). This classification structure consists of identifier-independent or identifier-dependent entities, connection relationships, categorization relationships, attributes (i.e., properties used to describe an entity), and keys (e.g., primary, alternate or foreign). A definite advantage in using IDEF1X lies in its roots. Due to the strict standardization of DoD projects, IDEF1X will probably escape having numerous variants evolve from it (KSBI, 1998b, p. 3).
The AFOMS is currently using the IDEF1X method to design its logical corporate database. Modelers from Acton Burnell and the CSS have begun the process of defining entities, connection relationships, attributes or properties used to describe an entity, and primary, alternate, and foreign keys. Modelers from Acton Burnell and the CSS plan on completing the IDEF1X of the AFOMS by 31 Dec 98. In 1999, AFOMS programmers plan on using the resultant diagrams as the blueprint for building the AFOMS corporate database.
Challenges
The biggest challenge facing the AFOMS is the scarcity of resources. Because AFOMS mission is an operational one, our primary emphasis lies in developing enlisted promotion tests, analyzing enlisted specialties, and writing study guides. With increasingly austere budgets, the money to pay for these initiatives have either come from "out of hide" (i.e., savings accrued throughout the fiscal year normally used to pay for mission costs), or from end-of-year "fallout" money. Such was the case for FY 98. The AFOMS devoted $39,000 of its own money to fund ATPS, and the HQ AETC/DO transferred about $75,000 to pay for two test development team rooms, an additional server, and the cabling for ATDS. These costs represent "seed" money to allow AFOMS programmers to develop the software for both ATPS and ATDS.
Finding the funds to automate the AFOAP remain elusive. The AFOMS has estimated that the cost to develop the front and back ends to CODAP is $216,900. The software and hardware costs to develop the AFOMS corporate database with the intranet and web-based administration of JIs is $171,800. Despite these initial costs, the AFOMS believes these near-term costs will more than justify the long-term investment. This long-term investment in automation will help to bring the AFOMS Air Force Occupational Analysis Program and the Air Force enlisted promotion test development process into the 21st century.
References
AFMS 16008X. (1997). Air Force Occupational Measurement Squadron. Randolph AFB: GPO.
deWitte, P. S. (1998). "IDEF enterprise engineering methodologies support simulation."
http://www.idef.com/articles/inside-the-process/Inside-the-process.htm
DoDD 8320.1 (1998). DoD Data Administration. Washington, DC: GPO
Kailiwai, G. (1997, December). "AFOMS overview briefing." Unpublished briefing.
Knowledge Based Systems, Inc. (KSBI). (1998a) "IDEF0 function modeling method."
http://www.idef.com/idef0.htm
Knowledge Based Systems, Inc. (KSBI). (1998b) "IDEF1X data model method."
http://www.idef.com/idef1x.htm