Imta 2011 technical aptitude pres ii
0 likes233 views
This document summarizes two analyses examining predictors of technical aptitude and technical training performance. Analysis I found that a measure of technical aptitude was related to post-training test scores, but individual technical knowledge subtests were not. Analysis II found that the IT Aptitude Battery provided incremental validity over existing selection composites in predicting technical aptitude, particularly technical knowledge aptitude. Overall, the analyses suggest technical aptitude may be a better predictor of technical training outcomes than measures of technical knowledge alone.
More Related Content
Viewers also liked (11)
Similar to Imta 2011 technical aptitude pres ii (20)
Recently uploaded (20)
Imta 2011 technical aptitude pres ii
- 1. Predicting Technical Aptitude: Relations between Predictor Variables, Technical Aptitude and Technical Training Performance (O.N.R. Contracts Nr. N00014-10-M-0087 & N00014-10-C-0505) Ryan Glaze & Martin J. Ippel CogniMetrics, Inc., San Antonio, TX 1 Paper presented at the 53rd Annual Conference of the International Military Testing Association, Bali (Indonesia). October 31 November 4, 2011
- 2. Preview Analysis I Overview of ASVAB and Technical Knowledge Tests Introduction to Technical Aptitude Proposed Model Methods, Results, and Discussion Analysis II Incremental Validity Analysis Predicting Technical Aptitude with ASVAB Selection Composites and the ITAB Methods, Results, and Discussion 2
- 3. ASVAB US Armed Forces must select, classify, and train personnel to work in highly technical work environment ASVAB and Technical Knowledge Subtests used for technical Navy Ratings General Science (GS) Mechanical Comprehension (MC) Auto Shop (AS) Electrical Information (EI) 3
- 4. Technical Knowledge Subtests Technical Knowledge Subtests have several limitations: Represent arbitrary and limited sample of domain Measures Technical Knowledge but not Technical Skills Provide modest predictive validity 4
- 5. Technical Aptitude Technical Aptitude (Ippel & Glaze, 2011) Technical Knowledge Aptitude (TKA) Aptitude to learn technical knowledge (concepts) Derived from performance on eight common knowledge tests Technical Skill Aptitude (TSA) Aptitude to learn technical skills Derived from performance on seven common skill tests 5
- 6. Technical Aptitude Technical Aptitude (TA) Represents a construct with a short logical distance to criterion performance in Apprentice Technical Training (ATT) performance Will be used to assess construct validity of TK subtests 6
- 10. 10 Technical Aptitude AFQT GS MC AS EI Post-Test
- 11. 11 Technical Aptitude AFQT GS MC AS EI Post-Test
- 12. 12 Technical Aptitude AFQT GS MC AS EI Post-Test
- 13. 13 Technical Aptitude AFQT GS MC AS EI Post-Test
- 14. 14 Technical Aptitude AFQT GS MC AS EI Post-Test
- 15. 15 Technical Aptitude AFQT GS MC AS EI Post-Test
- 16. Method 410 Navy Recruits participating in A.T.T. program ASVAB TK Subtests: GS, MC, AS, EI AFQT Considered a measure of crystallized intelligence A.T.T. post-training test scores Eight common knowledge tests Seven common skill tests Dichotomously scored (Pass/Fail with 70 point cut score) 16
- 17. Method Technical Knowledge Aptitude IRT-based ability estimate derived from common eight knowledge tests Technical Skill Aptitude IRT-based ability estimate derived from common seven skill tests 17
- 18. Results: First Knowledge Test 18 RMSEA = 0.000 PClose = 0.886 WRMR = 0.040 Technical Aptitude AFQT GS MC AS EI Post-Test
- 19. Technical Aptitude AFQT GS MC AS EI Post-Test Results: First Knowledge Test 19 .508* .316* .189* .226* .084 .352*
- 20. Results: First Knowledge Test 20 Technical Aptitude AFQT GS MC AS EI Post-Test .137.130 .175 .530*
- 21. Results: First Knowledge Test 21 Technical Aptitude AFQT GS MC AS EI Post-Test .217* .017 -.119 -.021
- 22. Results: All Knowledge Tests 22 MeanRMSEA = 0.000 MeanPClose = 0.868 MeanWRMR = 0.039 Technical Aptitude AFQT GS MC AS EI Post-Test
- 23. Results: All Knowledge Tests The paths between Technical Knowledge Aptitude, Technical Knowledge Subtests, AFQT were nearly identical for all knowledge tests Focus will be on: AFQT Post-test TA Post-test 23
- 24. Results: All Knowledge Tests 24 Technical Aptitude AFQT GS MC AS EI Post-Test .530* .084
- 25. Results: All Knowledge Tests AFQT was related to all Technical Knowledge Tests, but not post-training test scores Technical Knowledge Aptitude was related to post-training test scores, but not Technical Knowledge test scores Indirect effects of Technical Knowledge Aptitude on post-training test scores via TK subtests were not significant 25
- 26. Results: First Skill Test 26 RMSEA = 0.024 PClose = 0.477 WRMR = 0.187 Technical Aptitude AFQT GS MC AS EI Post-Test
- 27. Results: First Skill Test 27 Technical Aptitude AFQT GS MC AS EI Post-Test .066 .373*.536* .179* .267*
- 28. Results: First Skill Test 28 Technical Aptitude AFQT GS MC AS EI Post-Test .218* .174* .032 .064
- 29. Results: First Skill Test 29 Technical Aptitude AFQT GS MC AS EI Post-Test .076 .062 -.029 .062
- 30. Results: All Skill Tests 30 MeanRMSEA = 0.024 MeanPClose = 0.477 MeanWRMR = 0.187 Technical Aptitude AFQT GS MC AS EI Post-Test
- 31. Results: All Skill Tests The paths between Technical Skill Aptitude, Technical Knowledge Subtests, AFQT were nearly identical for all skill tests Focus will be on: AFQT Post-test TA Post-test 31
- 32. Results: All Skill Tests 32 Technical Aptitude AFQT GS MC AS EI Post-Test .218* .066
- 33. Results: All Skill Tests AFQT was related to all Technical Knowledge Tests, but not post-training test scores Technical Skill Aptitude was related to post- training test scores, but not Technical Knowledge test scores Indirect effects of Technical Knowledge Aptitude on post-training test scores via TK subtests were not significant 33
- 34. Results: Technical Aptitude Technical Knowledge Aptitude was only slightly related to Technical Skill Aptitude (r = .136) Technical Aptitude was related to post- training test scores, but not Technical Knowledge test 34
- 35. Analysis II Results of Analysis I suggest Technical Aptitude was related to post-training test scores, but Technical Knowledge was not Analysis II seeks to identify predictors of Technical Aptitude Current predictors of training performance consist of various ASVAB Selection Composites (ASC) I.T. Aptitude Battery (ITAB) was designed to measure technical aptitude 35
- 36. Method Two selection composites that the Navy currently uses to assign recruits to ratings ASC01 consists of WK, PC, AR, and MC ASC02 consists of MK, AR, GS, and EI ITAB consists of two fully interactive tests Hidden Target Test Battery Test 36
- 37. Results 37 Variable 1. 2. 3. 4. 1. ITAB 2. ASC01 .33 3. ASC02 .31 .78 4. TSA .17 .16 .34 5. TKA .30 .44 .69 0.14 All Correlations significant at p < .01.
- 38. Results 38 Predictors Multiple R Incremental Validity Selection Composite Test ASC ITAB R2 F Sig R2 %R2 F sig ASC01 TKA 0.383 0.174 0.22 57.66 p < .0001 0.027 13.99% 14.11 p < .001 ASC02 TKA 0.66 0.095 0.484 191.1 p < .0001 0.008 1.68% 6.504 p < .05 ASC01 TSA 0.117 0.132 0.041 8.703 p < .01 0.015 57.69% 6.574 p < .05 ASC02 TSA 0.318 0.071 0.12 27.81 p < .0001 0.005 4.35% 2.146 p > .05
- 39. Results Selection composites (ASC01 and ASC02) significantly predicted Technical Knowledge Aptitude and Technical Skill Aptitude Technical Knowledge Aptitude ITAB provided incremental validity over selection composites for Technical Knowledge Aptitude Technical Skill Aptitude ITAB provided incremental validity over ASC01, but not ASC02, for Technical Skill Aptitude 39
- 40. Thank You 40