Made with FlowPaper - Flipbook Maker
Exam Report for 2022-2023 Testing Year Board of Certification (BOC) Certification Exam for Athletic Trainers April 2023 Credentialing Examination Consulting, LLC 2 of 5 BOC 2022-2023 Testing Year Report Introduction The Board of Certification, Inc., (BOC) is a non-profit credentialing agency that provides certification for the athletic training profession. Although it had already operated for 20 years as a committee of the National Athletic Trainers Association, the BOC was incorporated in 1989 to govern the certification program and the standards for recertification. The entry-level certification exam is designed to establish a common benchmark for the profession. The BOC serves the public interest by developing, administering and continually reviewing a certification process that reflects current standards of practice in athletic training. Standard Setting and Equating of Exam Forms The modified Angoff process was used to establish the performance standard for the forms based on PA7. In April 2017, a panel of twelve certified ATs reviewed the scored items for forms PA701 and PA702. Some minor changes were made to the scored set of items on form PA702. When the forms were introduced in May 2017, the cut score for Form PA702 was equated to the standard set on Form PA701. All new forms assembled under PA7, including PA711 and PA712, have been equated using the Rasch model after the first testing window in which they are used to the passing standard established for PA701. Score Reporting Because exam forms are possibly of different difficulty, providing raw scores can be misleading. As a result, many programs, including the BOC exam, use scaled scores. Scaled scores are particularly useful because they provide the basis for meaningful long-term comparisons of results across different forms of an exam. Scaled scores are used because over the life of every testing program, situations arise in which changes in test length occur, a decision is made to assess more or fewer areas, the number of items that are scored versus unscored (experimental) changes or forms of the test of different difficulty are compared. To ensure the scores have comparable meaning as these changes occur, the equated scores developed for all candidates are converted via linear transformation to a scale of 200 to 800 with the passing standard reported as 500. The BOC provides scaled scores and pass/fail standing to candidates approximately two to four weeks after the close of a testing window. Candidates pass or fail based on how their test performance compares to the equated, criterion- referenced passing standard for the form taken. Analysis of the Certification Exam Candidate Performance Statistical reports are generated for each form and testing window. All BOC exams administered during the 2022- 2023 testing year have been classified as either first-time exams or retake exams: ▪ First-time exams – exams taken by candidates who never previously sat for any form of the BOC exam ▪ Retake exams – exams taken by candidates who previously sat one or more times for the BOC exam Candidates Excluded from this Report Beginning with 2018, candidates who were administered the BOC certification exam via paper and pencil (i.e., ADA candidates) have been included in the information provided in Table 1. They were not included when the statistics were computed for other years reported in Table 1. Also, for the testing year 2022-2023, these candidates (n = 5 individuals, but 7 testing events) are excluded from the analyses in the rest of the report. No candidates completed less than 25% of their exam, but if there had been they would have been excluded from statistical reports because of the likelihood that their performance was affected by construct irrelevant variance, such as being late to the site, limited English proficiency or other issues. 3 of 5 BOC 2022-2023 Testing Year Report There are 3,744 administrations of the BOC certification exam included in this report, which represents a 9.2% decrease from the 2021-2022 testing year (4,120). The number of first-time exams decreased from 2,654 in 2021- 2022 to 2,427 in 2022-2023, which represents an 8.6% decrease. Retake exams decreased from 1,466 in 2021- 2022 to 1,317 in 2022-2023, a 10.2% decrease. Pass Rates Table 1. Historical BOC Exam Counts and Pass Rates Year First-time Pass % Pass Retake Pass % Pass All Pass % Pass RD/PA6 2011-2012 3,222 2,653 82.3% 1,664 696 41.8% 4,886 3,269 66.9% 2012-2013 3,631 2,935 80.8% 1,319 507 38.4% 4,950 3,442 69.5% 2013-2014 3,679 3,048 82.8% 1,210 497 41.1% 4,889 3,545 72.5% 2014-2015 3,768 3,039 80.7% 1,283 466 36.3% 5,051 3,505 69.4% 2015-2016 4,059 3,357 82.7% 1,308 547 41.8% 5,367 3,904 72.7% 2016-2017 4,119 3,444 83.6% 1,204 465 38.6% 5,323 3,909 73.4% 2017-2018 2,406 2,149 89.3% 185 85 45.9% 2,591 2,234 86.2% PA7 2017-2018 1,606 1,091 67.9% 1,172 334 28.5% 2,780 1,426 51.3% 2018-2019* 3,974 3,091 77.8% 1,622 645 39.8% 5,596 3,736 66.8% 2019-2020* 3,913 3,038 77.6% 1,626 636 39.1% 5,539 3,674 66.3% 2020-2021* 3,035 1,871 61.6% 1,916 576 30.0% 4,951 2,447 49.4% 2021-2022* 2,625 2,027 77.2% 1,495 611 40.9% 4,120 2,638 64.0% 2022-2023* 2,427 1,799 74.1% 1,317 477 36.2% 3,744 2,276 60.8% *Note. Total numbers include ADA and paper and pencil administrations. 4 of 5 BOC 2022-2023 Testing Year Report Table 2. Historical BOC Exam Scaled Score Summary Statistics Cohort N Mean Median Std Dev Min Max All 2022-23 3738 525.0 530 100.0 200 800 First-time 2424 556.6 570 96.8 200 800 Retake 1314 466.8 470 77.1 200 710 All 2021-22 4,118 532.6 540 107.7 200 800 First-time 2,624 568.4 580 102.9 200 800 Retake 1,494 469.8 480 84.8 200 710 All 2020-21 4,953 498.1 495 87.7 200 750 First-time 3,035 521.5 520 88.2 200 750 Retake 1,911 461.0 470 72.8 200 690 All 2019-20 5,528 540.2 532 66.1 358 735 First-time 3,912 558.8 558 65.2 382 735 Retake 1,616 495.2 489 42.3 358 683 All 2018-19 5,591 539.0 532 66.3 340 729 First-time 3,970 557.6 555 64.8 352 729 Retake 1,621 493.4 488 44.0 340 642 All 2017-18 5,369 536.9 538 71.3 275 743 First-time 4,012 557.8 560 65.1 304 743 Retake 1,357 475.0 478 49.9 275 665 All 2016-17 5,323 551.3 554 72.2 265 751 First-time 4,119 569.6 577 67.2 315 751 Retake 1,204 488.8 489 50.7 265 678 All 2015-16 5,367 543.5 544 67.5 217 729 First-time 4,059 561.2 564 62.8 356 729 Retake 1,308 488.6 489 49.9 217 661 All 2014-15 5,051 543.3 542 71.4 259 733 First-time 3,768 563.0 567 67.1 272 733 Retake 1,283 485.5 484 48.7 259 650 All 2013-14 4,889 529.7 536 57.1 272 680 First-time 3,679 545.1 548 51.9 332 680 Retake 1,210 482.8 488 45.5 272 650 All 2012-13 4,950 524.0 524 54.0 302 674 First-time 3,631 539.0 548 51.0 302 674 Retake 1,319 484.0 488 41.0 314 632 All 2011-12 4,886 525.0 524 54.0 230 692 First-time 3,222 542.0 548 51.0 272 692 Retake 1,664 491.0 494 44.0 230 644 All 2010-11 5,711 490.0 494 67.0 200 672 First-time 2,963 508.0 517 71.0 200 672 Retake 2,748 470.0 476 56.0 220 624 5 of 5 BOC 2022-2023 Testing Year Report Exam Form Reliabilities and Other Summary Data The performance of the forms of the BOC certification exam used during the year are consistent with reporting requirements for the National Commission for Certifying Agencies accreditation. Reliability is assessed using Cronbach’s alpha (Cronbach, 1951), the most widely used measure of overall exam form reliability; Livingston- Lewis (Livingston & Lewis, 1995), a measure used for estimating the decision consistency (i.e., the reliability of pass/fail decisions based on the test); and the standard error of measurement (SEM) presented in raw score units, a measure of the precision of the exam form. Summary Statistics concerning the quality of the BOC certification exam as a measurement instrument indicate that the exam complies with psychometric requirements that pertain to certification and licensure tests. Estimates of equivalence across forms for the various parts of the exam are acceptable. Likewise, candidate performance on all parts of the exam is consistent with the public protection mission of the BOC. However, the reliability coefficients for the testing year were lower than in previous years. References American Educational Research Association, American Psychological Association, National Council on Measurement in Education (2014). Standards for Educational and Psychological Testing. Washington, D.C.: AERA. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. Equal Employment Opportunity Commission (EEOC), U.S. Civil Service Commission, U.S. Department of Labor, and U.S. Department of Justice. (1978). Uniform Guidelines on Employee Selection Procedures. Federal Register, 43 (166), 38290–38315. Impara, J. C., & Plake, B. S. (1997). Standard setting: An alternative approach. Journal of Educational Measurement, 34, 353–366. Livingston, S. A., & Lewis, C. (1995). Estimating the consistency and accuracy of classifications based on scores. Journal of Educational Measurement, 32, 179-197. National Commission for Certifying Agencies (2016). Standards for the Accreditation of Certification Programs. Washington, DC: Institute for Credentialing Excellence. Kolen, M. J., & Brennan, R. L. (2004) Test Equating, Scaling and Linking: Methods and Practices Statistics for Social Science and Behavioral Sciences (2 ed.). Springer-Verlag New York Inc. Kuder, G. F., & Richardson, M. W. (1937). The theory of the estimation of test reliability. Psychometrika, 2, 151–160. Next >