Made with FlowPaper - Flipbook Maker
2024-2025 Exam Report Certification Exam for Athletic Trainers Prepared by: Ian Hembry, PhD Director of Psychometrics and Examinations Confidential: Not for Redistribution May 2025 BOC 2024-2025 Exam Year Report 1 Introduction The Board of Certification, Inc., (BOC) is a non-profit credentialing agency that provides certification for the athletic training profession. Although it had already operated for 20 years as a committee of the National Athletic Trainers Association, the BOC was incorporated in 1989 to govern the certification program and the standards for recertification. The entry-level certification exam is designed to establish a standard benchmark for the profession. The BOC serves the public interest by developing, administering and continually reviewing a certification process that reflects current standards of practice in athletic training. Standard Setting and Equating of Exam Forms The modified Angoff process was used to establish the performance standard for the forms based on PA8. In April 2023, a panel of nine certified ATs reviewed the scored items for forms PA801 and PA802. The cut score for Form PA802 was equated to the standard set on Form PA801. All new forms assembled under PA8, including PA802, will be equated using the Rasch model. The logit equivalent will establish for the new form in reference to the standard established for PA801 after the first exam window in which they appear. Score Reporting Because exam forms may have different difficulty, providing raw scores can be misleading. As a result, many programs, including the AT Exam, use scaled scores. Scaled scores are particularly useful because they provide the basis for meaningful long-term comparisons of results across different forms of an exam. Scaled scores are used because, over the life of every testing program, situations arise in which changes in the length of the exam occur, a decision is made to assess more or fewer areas, the number of items that are scored versus unscored (experimental) changes or forms of the exam of different difficulty are compared. To ensure the scores have comparable meaning as these changes occur, the equated scores developed for all candidates are converted via linear transformation to a scale of 200 to 800 with the passing standard reported as 500. The BOC provides scaled scores and pass/fail standing to candidates approximately two to four weeks after the close of an exam window. Candidates pass or fail based on their exam performance compared to the equated, criterion- referenced passing standard for the form taken. Analysis of the Certification Exam Candidate Performance Statistical reports are generated for each form and exam window. All exams administered during the 2024-2025 exam year have been classified as either first-time exams, retake exams or other exams: ▪ First-time exams – exams taken by candidates who never previously sat for any form of the AT Exam ▪ Retake exams – exams taken by candidates who previously sat one or more times for the AT Exam Candidates Excluded from This Report Beginning in 2018, candidates who were administered the AT Exam via paper and pencil (i.e., ADA candidates) have been included in the information provided in Table 1. They were omitted when the statistics were computed for other years reported in Table 1. Also, for the exam year 2024-2025, these candidates (n = 2 individuals and 3 exam administration events) are excluded from the analyses in the rest of the report. No candidates completed less than 25% of their exam. In this scenario, the candidates record would have been excluded from statistical reports because of the likelihood that their performance was affected by construct-irrelevant variance, such as being late to the site, limited English proficiency or other issues. This report includes 3,791 administrations of the AT Exam, a 2.2% decrease from the 2023-2024 exam year (3,876). The number of first-time exams decreased from 2,285 in 2023-2024 to 2,256 in 2023-2024, a 1.3% decrease. Retake exams decreased from 1,591 in 2023-2024 to 1,535 in 2024-2025, a 3.38% decrease. BOC 2024-2025 Exam Year Report 2 Pass Rates Table 1. Historical BOC Exam Counts and Pass Rates Year First-time Pass % Pass Retake Pass % Pass All Pass % Pass PA7 2017-2018 1,606 1,091 67.9% 1,172 334 28.5% 2,780 1,426 51.3% 2018-2019* 3,974 3,091 77.8% 1,622 645 39.8% 5,596 3,736 66.8% 2019-2020* 3,913 3,038 77.6% 1,626 636 39.1% 5,539 3,674 66.3% 2020-2021* 3,035 1,871 61.6% 1,916 576 30.0% 4,951 2,447 49.4% 2021-2022* 2,625 2,027 77.2% 1,495 611 40.9% 4,120 2,638 64.0% 2022-2023* 2,427 1,799 74.1% 1,317 477 36.2% 3,744 2,276 60.8% PA8 2023-2024* 2,285 1,569 68.7% 1,591 463 29.1% 3,876 2,032 52.4% 2024-2025 * 2,256 1,683 74.6% 1,535 449 29.3% 3,791 2,132 56.2% *Note. Total numbers include ADA paper and pencil administrations. Table 2. Historical BOC Exam Scaled Score Summary Statistics Cohort N Mean Median Std Dev Min Max All 2024-25 3,788 510.2 510 98.5 200 760 First-time 2,256 550.1 560 92.3 200 760 Retake 1,532 451.4 460 75.1 200 670 All 2023-24 3,866 502.0 500 100.4 200 800 First-time 2,282 537.1 540 98.9 200 800 Retake 1,584 451.4 450 78.7 200 750 All 2022-23 3,738 525.0 530 100.0 200 800 First-time 2,424 556.6 570 96.8 200 800 Retake 1,314 466.8 470 77.1 200 710 All 2021-22 4,118 532.6 540 107.7 200 800 First-time 2,624 568.4 580 102.9 200 800 Retake 1,494 469.8 480 84.8 200 710 All 2020-21 4,953 498.1 495 87.7 200 750 First-time 3,035 521.5 520 88.2 200 750 Retake 1,911 461.0 470 72.8 200 690 All 2019-20 5,528 540.2 532 66.1 358 735 First-time 3,912 558.8 558 65.2 382 735 Retake 1,616 495.2 489 42.3 358 683 All 2018-19 5,591 539.0 532 66.3 340 729 First-time 3,970 557.6 555 64.8 352 729 Retake 1,621 493.4 488 44.0 340 642 All 2017-18 5,369 536.9 538 71.3 275 743 First-time 4,012 557.8 560 65.1 304 743 Retake 1,357 475.0 478 49.9 275 665 BOC 2024-2025 Exam Year Report 3 Exam Form Reliabilities and Other Summary Data The performance of the forms of the AT Exam used during the year is consistent with reporting requirements for the National Commission for Certifying Agencies accreditation. Reliability is assessed using Cronbach’s alpha (Cronbach, 1951), the most widely used measure of overall exam form reliability; Livingston-Lewis (Livingston & Lewis, 1995), a measure used for estimating the decision consistency (i.e., the reliability of pass/fail decisions based on the exam); and the standard error of measurement (SEM) presented in raw score units, a measure of the precision of the exam form. The results indicate that each scored set meets general guidelines for reliability. In addition, the standard errors of measurement (SEM) are acceptably small, indicating that each scored set functions as an acceptably precise measurement instrument. Summary Statistics concerning the quality of the AT Exam as a measurement instrument indicate that the exam complies with psychometric requirements that pertain to certification and licensure exams. Reliability coefficients were consistent with prior exam years. Estimates of equivalence across forms for the various parts of the exam are acceptable. Likewise, candidate performance on all parts of the exam is consistent with the public protection mission of the BOC. References American Educational Research Association, American Psychological Association, National Council on Measurement in Education (2014). Standards for Educational and Psychological Testing. Washington, D.C.: AERA. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. Equal Employment Opportunity Commission (EEOC), U.S. Civil Service Commission, U.S. Department of Labor, and U.S. Department of Justice. (1978). Uniform Guidelines on Employee Selection Procedures. Federal Register, 43 (166), 38290–38315. Impara, J. C., & Plake, B. S. (1997). Standard setting: An alternative approach. Journal of Educational Measurement, 34, 353–366. Livingston, S. A., & Lewis, C. (1995). Estimating the consistency and accuracy of classifications based on scores. Journal of Educational Measurement, 32, 179-197. National Commission for Certifying Agencies (2016). Standards for the Accreditation of Certification Programs. Washington, DC: Institute for Credentialing Excellence. Kolen, M. J., & Brennan, R. L. (2004) Test Equating, Scaling and Linking: Methods and Practices Statistics for Social Science and Behavioral Sciences (2 ed.). Springer-Verlag New York Inc. Kuder, G. F., & Richardson, M. W. (1937). The theory of the estimation of test reliability. Psychometrika, 2, 151–160. Next >