Made with FlowPaper - Flipbook Maker
2025 Exam Report Orthopedic Specialty Exam May 2026 2 of 6 BOC 2025 Annual OSE Report Introduction The Board of Certification, Inc., (BOC) is a non-profit credentialing agency that provides certification for the athletic training profession. Incorporated in 1989, the BOC governs the certification program for Athletic Trainers (AT) by establishing both the standards for the practice of athletic training and the continuing education requirements. The BOC serves the public interest by developing, administering and continually reviewing a certification process that reflects current standards of practice in athletic training. In 2021, the BOC launched the Orthopedic Specialty Certification (orthopedic specialty) for ATs. The Board Certified Specialist in Orthopedics (BCS-O®) credential is available to ATs who have acquired specialized education and focused experience in orthopedics beyond the requirements for the ATC ® credential. It is the first and only board-certified specialty for ATs who specialize in orthopedics, and it is backed by the BOC’s rigorous process and standards. Standard Setting and Equating of Exam Forms The modified Angoff process was used to establish the performance standard for the Orthopedic Specialty Exam (OSE). In October 2021, a panel of eight certified ATs with a specialty practice in orthopedics participated in the study. All new forms will be equated to the passing standard established for Form PA101 using a method that is appropriate for small programs after the first exam window in which they are used. Score Reporting Raw scores for the OSE are reported as scaled scores. Scaled scores are particularly useful because they provide the basis for meaningful long-term comparisons of results across different forms of an exam. New forms are introduced to ensure that items are not overexposed for exam security purposes, to maintain relevance to the profession, and to keep up with any changes that may occur to the composition of the blueprint, exam length, or other assessment mechanism. These forms are built to the same content specifications as the existing blueprint, but they are not necessarily constructed at the same difficulty level. Equating is a statistical process that transforms new form raw scores onto the same scale as the original base form scores to adjust for minor differences in difficulty across forms. The equated scores are then converted via linear transformation to a scale of 1000 to 1450, with the passing standard reported as 1200. The BOC provides scaled scores (for fail applicants) and pass/fail standing to applicants approximately two to four weeks after the close of an exam window. Applicants pass or fail based on how their exam performance compares to the criterion-referenced passing standard. Because the content areas specified for the exam were validated as critical for specialty practice, the items and the forms are intended to assess essential knowledge and/or skills for newly certified orthopedic specialists. Overall scores from the exam can be used to make inferences about the relevant knowledge and skills that applicants have acquired. Scores on the OSE are not, however, intended as predictors of future success in the specialty. 3 of 6 BOC 2025 Annual OSE Report Analysis of the OSE Applicant Performance Applicants may qualify to take the OSE by completing an accredited residency in orthopedics or by satisfying alternative eligibility criteria. Applicants taking the OSE in the period addressed in this report were classified as either first-time exams or retake exams: First-time exams: Exams taken by applicants who never previously sat for any form of the OSE Retake exams: Exams taken by applicants who previously sat one or more times for OSE This report includes 24 exam events of the OSE across two administration windows, a slight increase from the 2024 testing year (22). Of the 24 exam events, two (8.3%) were graduates of an athletic training residency in orthopedics and 22 (91.7%) qualified by meeting other requirements. Additionally, 20 (83.3%) of the exam events involved applicants taking the exam for the first time, while 4 (16.7%) were retake attempts, each involving a unique individual (see Table 2). Pass Rates Table 1 provides annual pass rates for the OSE since 2021 by eligibility pathway; and Table 2 provides annual pass rate by Retake status. Table 1. Historical Counts and Pass Rates for the OSE Overall and by Eligibility Pathway Year Path 1 Residency Pass % Pass Path 2 Other Pass % Pass All Pass % Pass PA103 2021 5 5 100.00% 10 4 40.00% 15 9 60.00% 2022 2 1 50.00% 15 6 40.00% 17 7 41.17% 2023 5 3 60.00% 17 8 47.06% 22 11 50.00% 2024 3 3 100.00% 19 11 57.89% 22 14 63.64% 2025 2 1 50.00% 22 16 72.73% 24 17 70.83% Table 2. Historical Counts and Pass Rates for the OSE Overall and by Retake Status Year First-time Pass % Pass Retake Pass % Pass All Pass % Pass PA103 2021 15 9 60.00% --- --- --- 15 9 60.0% 2022 12 5 41.67% 5 2 40.00% 17 7 41.2% 2023 15 9 60.00% 7 2 28.57% 22 11 50.00% 2024 14 11 78.57% 8 3 37.50% 22 14 63.64% 2025 20 14 70.00% 4 3 75.00% 24 17 70.83% 4 of 6 BOC 2025 Annual OSE Report Table 3 presents pass/fail statistics for each form administered by modality during the 2025 exam year. The overall pass rate for 2025 was 70.83% Table 3. BOC OSE Exam Pass Rates by Form and Modality Form Frequency Percent Modality Pass Fail Total Pass Fail PA103 Onsite 9 5 14 64.29% 35.71% LRP 8 2 10 80.00% 20.00% Total 17 7 24 70.83% 29.17% All 17 7 24 70.83% 29.17% Distribution of Candidate Scores Table 4 presents scaled score summary statistics for the 2025 annual testing cohort, alongside the previous years of testing. Table 4. Historical OSE Scaled Score Summary Statistics Grouped by Retake Status Cohort N Mean Median Std Dev Min Max All 2025 24 1220.0 1215 57.8 1056 1326 First-time 20 1221.0 1221 63.0 1056 1326 Retake 4 1215.0 1212 19.9 1194 1242 All 2024 22 1225.1 1212 52.5 1158 1344 First-time 14 1242.9 1245 57.8 1158 1344 Retake 8 1194.0 1194 18.1 1170 1224 All 2023 22 1196.2 1203 42.9 1122 1272 First-time 15 1205.6 1218 43.7 1134 1272 Retake 7 1176.0 1182 35.7 1122 1212 All 2022 17 1196.1 1188 65.7 1110 1368 First-time 12 1201.5 1188 74.6 1110 1368 Retake 5 1183.2 1176 41.0 1134 1236 All 2021 15 1233.2 1200 97.3 1104 1410 First-time 15 1233.2 1200 97.3 1104 1410 Retake -- -- -- -- -- -- 5 of 6 BOC 2025 Annual OSE Report Exam Form Reliabilities and Other Summary Data The performance of the OSE form used during the year is consistent with reporting requirements for the National Commission for Certifying Agencies accreditation. Reliability is assessed using Cronbach’s alpha (Cronbach, 1951), the most widely used measure of overall exam form reliability; Livingston-Lewis (Livingston & Lewis, 1995), a measure used for estimating the decision consistency (i.e., the reliability of pass/fail decisions based on the test); and the standard error of measurement (SEM) presented in raw score units, a measure of the precision of the exam form. A new form was introduced in September 2024, and reliability and decision consistency estimates for the exam were in an acceptable range for 2025. Summary and Conclusions Statistics concerning the quality of the OSE as a measurement instrument indicate that the exam complies with psychometric requirements that pertain to certification and licensure exams. Collectively, reliability estimates were appropriate for the use of making pass/fail decisions for candidates. Moreover, items are consistently evaluated statistically, through independent SME review after each window, and aggregate review over previous administrations, to ensure the relevance and validity of all items. The BOC is committed to maintaining the integrity and fairness of its exam forms. 6 of 6 BOC 2025 Annual OSE Report References Brennan, R. L., & Kane, M. T. (1977). An index of dependability for mastery tests. Journal of Educational Measurement, 14, 277–289. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. Kuder, G. F., & Richardson, M. W. (1937). The theory of the estimation of test reliability. Psychometrika, 2, 151–160. Next >