The U. S. Navy currently uses the Aviation Selection Test Battery (ASTB) to select pilots and naval flight officer candidates prior to their entry into basic flight training. The ASTB began operational use in 1992, and consists of 5 scored paper-and-pencil subtests that are converted to composite scores that are predictive of training performance and attrition. One of these subtests is the Biographical Inventory (BI). The BI is a series of questions on an individual’s background and life experiences, and it was originally validated to predict early flight training attritions. Factor analytical studies have shown the 71 item BI to measure traits such as Risk-Taking Behavior, Athletic Orientation, Military Interest, Engineering Background and Independence. Since 1992, an unscored bank of non-cognitive test items, called the Aviation Interest (AI) test, has been administered to all applicants to the Navy’s flight training program. The current study evaluated the factor structure of this unscored subtest with the current BI and the incremental validity the AI items might provide using a dichotomous pass/fail criterion. The factor analysis showed a large amount of overlap between the items in the AI and BI. However, at least one factor, labeled rebelliousness, was unique to the AI test. Using a horizontal weighting method for individual test item responses, a combined AI/BI test key is proposed to improve the current predictive validity of the ASTB (R = .096) and/or replace outdated BI items.

Attrition in the training of naval aviators is a continual concern for the U. S. Navy for both financial and operational reasons. The least expensive training pipeline is the helicopter syllabus, in which a late stage attrition of a student represents a cost of approximately $500,000.00. The more expensive training pipelines such as Jet/Strike pilot training cost well over 1 million dollars for each failure. In addition to the financial costs, failures during training ultimately impact the manpower levels in the fleet, which can negatively impact operational readiness. The reasons for attrition from flight training are varied. A certain percentage of prospective aviators are dropped from the training program for medical reasons. However, the majority of the attritions out of training are due to academic or flight performance failures or to individuals just choosing to leave the program, referred to as a Drop On Request (DOR). DORs include individuals who have chosen to leave the program due to motivational problems or an expressed lack of interest in the flight program. The overall attrition rate in U. S. Navy pilot training fluctuates on an annual basis, but for the last 20 years or so the rate has stayed between 15 - 25% of the entering student population. The majority of these failures occur during the first 100 hours of flight training.

One means to reduce attritions or maintain an acceptable level of training failures is via ab initio selection tests validated to predict performance and non-performance based failures. The Navy has used a paper-and-pencil test to predict performance during initial flight training since World War II. The first test battery developed, the Aviation Classification Test, included both cognitive/spatial abilities tests and a biographical inventory which focused on past life experiences and an individual’s personal interests. Subsequent versions of the test in 1952, 1971, and 1992 retained these measures as predictors of performance. The current version of the test, the Aviation Selection Test Battery (ASTB), includes an empirically keyed Biographical Inventory, as well as an experimental bank of biographical items.

The ASTB is a multiple-choice test that was developed and validated jointly by the Naval Aerospace and Operational Medical Institute and the Educational Testing Services to predict initial ground school and primary flight training performance in the Navy, Marine Corps and Coast Guard pilot and naval flight officer (NFO) curricula. Ground school includes coursework in basic engine properties, aerodynamics and navigation, as well as more advanced academic work on specific aircraft systems during the earliest phase of their flight training. For student naval aviators, primary training lasts approximately 4 months and includes the above mentioned systems coursework, flight simulator sessions and actual flight time in the T-34C Mentor. Subsequent to their completion of primary training, students are placed into one of 3 specific aircraft training curricula, the maritime, jet, or helicopter pipelines. The ASTB was not validated to predict performance or attrition of student naval aviators beyond this ‘pipelining’.

The current form of the ASTB test has two equivalent forms that take approximately two and one-half hours to administer, and includes six timed subtests. The subtests include the Math/Verbal Test (MVT), the Mechanical Comprehension Test (MCT), the Spatial Apperception Test (SAT), the Aviation and Nautical Information Test (ANI), the Biographical Inventory Test (BI), and the Aviation Interest Test (AI). The MVT contains thirty-seven questions and evaluates basic math skills and paragraph comprehension. The MCT evaluates the candidates’ knowledge of basic mechanical principles and physics. The SAT measures a candidate’s ability to match an "inside-out" or "cockpit view" to an "outside-in" or "wingman’s view". The ANI tests a candidate’s specific knowledge about aviation and nautically-related material. The BI is a 71 item standard biographical inventory in which candidates respond to questions regarding their academic background and performance, as well as their life experiences. Factor analytical studies have shown the 71 item BI to measure traits such as Risk-Taking Behavior, Athletic Orientation, Military Interest, Engineering Background and Independence. The AI has not been validated against flight performance but was included in the test battery to allow for the continued validation of the 49 alternate test items that are included in this section. The goal was to use these items in future versions of the ASTB.

For applicants in Naval and Marine Corps aviation programs, the subtest scores on the ASTB are combined into a total of 3 composite scores for the pilot program, the Academic Qualification Rating (AQR), the Pilot Flight Aptitude Rating (PFAR), and the Pilot Biographical Inventory (PBI). The AQR and PFAR are weighted composites of the MVT, MCT, SAT, and ANI. The AQR was validated to predict academic performance in aviation preflight indoctrination and ground school. The PFAR is validated against a criterion of primary pilot performance (i.e., flight and simulator performance). The PBI score was derived separately from the BI subtest. The PBI was validated to specifically predict non-medical student pilot attrition from ground school and primary training. The predictive validity of the ASTB subtests for flight performance and success in training range from .23 to .40 (Multiple R), depending on the subtest and specific criterion used for analysis (Frank & Baisden, 1992). The current study is an initial description and evaluation of the items in the AI subtest for possible inclusion in a revised version of the ASTB.

The sample used for the factor analysis in the current study included all individuals (applicants) who took the ASTB between November 1992 and January 1998. A subgroup from this sample (N= 2177) included all individuals with completion data (Pass or non-medical Failures) from ground school and primary flight training that had been initially screened for entry into flight training using the ASTB. This sample was used for the empirical keying and regression analysis of the AI test items.

The Statistical Package for the Social Sciences (SPSS for Windows 8.0) was the source of the statistical analysis used in the study. The item factor analysis was done using Principle Components Procedures with a Varimax rotation. A factor analysis of the the applicant’s BI and AI item responses was done to evaluate the underlying structure of these tests. Using the criterion data sample, Chi-square analysis was performed to determine which AI items were significantly related to the pass-fail criterion. These items were then empirically keyed using a horizontal weighting method that was identical to the method used for the original BI key (Morrison, 1991; Guion, 1965). The SPSS Logistic Regression analysis program was used to evaluate the incremental validity of the AI test items over the current BI test score.

A principle component analysis was performed to assess the item intercorrelations for the BI and AI tests. The principle component analysis and orthogonal rotation yielded a 11 -component solution which accounted for 28.1% of the total item variance. Labels for the principle components obtained were assigned based upon a review of the general content of the items that made up the component. The first component (Comm-Source) was defined by items reflecting an individual’s commissioning source and accounted for 7.9% of the variance. Commissioning sources include the Naval Academy, ROTC programs, or Aviation Officer Candidate School. These items all came from the BI. The second component (Military Orientation) accounted for 5.1% of the variance and reflected an individual’s military orientation and interest prior to joining the service. The third component (Engineering) accounted for 2.7% of the item variance and reflected an individual’s engineering background and classroom experience. The fourth component (Risk-Taking) accounted for 2.3% of the variance and reflects an individual’s participation in adventurous activities and cited reasons for wanting to enter naval aviation. This factor included a number of AI test items. The fifth component (Science, Non-Engineering) accounts for 2.1% of the variance and reflects an individual’s science coursework in high school and college. This component primarily includes AI test items. The sixth component (Athletics) included both AI and BI items reflecting a person’s athletic participation in high school and college. This component accounted for 2.0% of the variance. The other 5 factors, Age, High School Performance, Swimming Experience, Outdoor Sports, and Rebelliousness/Activism individually accounted for less that 2.0% of the variance. The Rebelliousness/Activism component was the only component to include exclusively AI test items. This component reflects an individual’s past participation in protests and their perception of themselves as being rebellious.

Using the training pass/fail criterion, the AI test items were evaluated for significance (p<.05) using a Chi-square test. For each significant item, the horizontal weighting scoring method was applied and a composite score (AIcomp) for these items was developed for every subject. Out of the total sample of 2177, there were a total of 189 non-medical attritions from training. Of the 49 total items in the AI, 38 were significantly related to the criterion and included in the AI key. Several of the items included in the key were related to the unique factor Rebelliousness/Activism in the AI portion of the test.

Stepwise logistic regression with the pass/fail criterion was done using the AIcomp and the PBI score. The regression model showed that the PBI, uncorrected for range restriction, was a significant predictor of the probability of attrition ( R = .096), however, the AIcomp score was also a good predictor of eventual success in the program (R = .260).

DISCUSSION

The results of this study demonstrate that the Aviation Interest subtest of the ASTB is measuring many of the same constructs as the current Biographical Inventory that is used to screen applicants to the U. S. Navy pilot flight training curricula. The overlap between the two tests was seen in the engineering orientation/experiences, academic performance, and athletics. The component Rebelliousness/Activism was unique to the AI subtest, and these items were also found to be predictive of success in flight training.

This study also demonstrated the potential for a significant increase in test validity using the AI subtest in conjunction with the current BI score for predicting success in training. The AI keying, although not cross-validated using the present sample, suggests that that a substantial increase in predictive power can be realized using these subtest items. Over the last few years, a number of items from the BI have been dropped due to their perceived linkage to socio-economic status (SES) in applicant populations, with a corresponding drop in the predictive validity of the subtest. The incremental validity demonstrated by the AI items may allow the substitution of these alternatives to maintain the overall validity of the test battery, while excluding test items which may be perceived as being culturally or SES sensitive.

Frank, L. & Baisden, A. (1994). The 1992 Navy and Marine Corps Aviator Selection test Battery Development. Presented at the 1994 Annual Meeting of the military Testing Association, Williamsburg, VA.

Guion, R. (1965). Personnel testing. New York: McGraw-Hill.

Morrison, T. (1991). Validation of Biographical Inventory: U. S. Navy/Marine Corps Aviation Selection Test Battery. Proceedings of the 27th Meeting of Department of Defense Human factors Engineering Technical Group, San Antonio, TX.