Made with FlowPaper - Flipbook Maker
Exam Report for 2021-2022 Testing Year Board of Certification (BOC) Certification Exam for Athletic Trainers April 2022 Introduction The Board of Certification, Inc., (BOC) is a non-profit credentialing agency that provides certification for the athletic training profession. Although it had already operated for twenty years, the BOC was incorporated in 1989 to govern the certification program, including the exam, for entry-level certification and recertification standards for Athletic Trainers. The entry-level certification program is designed to establish a common benchmark for entry into the athletic training profession. The BOC serves the public interest by developing, administering and continually reviewing a certification process that reflects current standards of practice in athletic training. Standard Setting and Equating of Exam Forms The modified Angoff process was used to establish the performance standard for the forms based on the PA7. In April 2017, a panel of 12 Certified Athletic Trainers reviewed the scored items for forms PA701 and PA702. Some minor changes were made to the scored set of items on form PA702. When the forms were introduced in May 2017, the cut score for Form PA702 was equated to the standard set on Form PA701. All new forms assembled under PA7, including PA709 and PA710, have been equated using item response theory (IRT) methods to the passing standard established for PA701 after the first testing window in which they are used. Score Reporting Because exam forms are possibly of different difficulty, providing raw scores can be misleading. As a result, many programs, including the BOC exam, use scaled scores. Scaled scores are particularly useful because they provide the basis for meaningful long-term comparisons of results across different forms of an exam. Scaled scores are used because over the life of every testing program, situations arise in which changes in test length occur, a decision is made to assess more or fewer areas, the numbers of items that are scored versus unscored (experimental) changes or forms of the test of different difficulty are compared. For scaled scores, the passing standard (number of items answered correctly) on the BOC exam form is always reported as the same scaled score. The equated scores are then converted via linear transformation to a scale of 200 to 800 with the passing standard reported as 500. The BOC provides scaled scores and pass/fail decisions to candidates approximately two to four weeks after close of a test window. Candidates pass or fail based on how their test performance compares to the equated, criterion-referenced passing standard for the form taken. Analysis of the Certification Exam Candidate Performance Statistical reports are generated for each form and testing window. All BOC exams administered during the 2021- 2022 testing year have been classified as either first-time exams or retake exams: ▪ First-time exams – exams taken by candidates who never previously sat for any form of the BOC exam ▪ Retake exams – exams taken by candidates who previously sat one or more times for the BOC exam Performance in 2021 rebounded when compared to the significant drop in 2020 that was likely due to the COVID- 19 pandemic. Pass rates for the first-time and retake groups were similar to previous years 2019 and 2018. Candidates Excluded from This Report Beginning with 2018, candidates who were administered the BOC Certification Exam via paper and pencil (i.e., ADA candidates) have been included in the information provided in Table 1. They were not included when the statistics were computed for other years reported in Table 1. These individuals (n = 2) were excluded from the analyses in the rest of the report. Also, candidates who completed less than 25% of their exam were excluded from statistical reports because of the likelihood that their performance was affected by construct irrelevant variance, such as being late to the site, limited English proficiency or other issues. There are 4,120 administrations of the BOC Certification Exam included in this report, which represents a 16.8% decrease from the 2020-2021 testing year (4,951). The number of first-time examinations decreased from 3,035 in 2020-2021 to 2,625 in 2021-2022, which represents a 13.5% decrease. Retake exams decreased from 1,917 in 2020-2021 to 1,495 in 2021-2022, a 22.0% decrease. Pass Rates Table 1. Historical BOC Exam Counts and Pass Rates Year First-time Pass % Pass Retake Pass % Pass All Pass % Pass RD/PA6 2011-2012 3,222 2,653 82.3% 1,664 696 41.8% 4,886 3,269 66.9% 2012-2013 3,631 2,935 80.8% 1,319 507 38.4% 4,950 3,442 69.5% 2013-2014 3,679 3,048 82.8% 1,210 497 41.1% 4,889 3,545 72.5% 2014-2015 3,768 3,039 80.7% 1,283 466 36.3% 5,051 3,505 69.4% 2015-2016 4,059 3,357 82.7% 1,308 547 41.8% 5,367 3,904 72.7% 2016-2017 4,119 3,444 83.6% 1,204 465 38.6% 5,323 3,909 73.4% 2017-2018 2,406 2,149 89.3% 185 85 45.9% 2,591 2,234 86.2% PA7 2017-2018 1,606 1,091 67.9% 1,172 334 28.5% 2,780 1,426 51.3% 2018-2019* 3,974 3,091 77.8% 1,622 645 39.8% 5,596 3,736 66.8% 2019-2020* 3,913 3,038 77.6% 1,626 636 39.1% 5,539 3,674 66.3% 2020-2021* 3,035 1,871 61.6% 1,916 576 30.0% 4,951 2,447 49.4% 2021-2022* 2,625 2,027 77.2% 1,495 611 40.9% 4,120 2,638 64.0% *Note. Total numbers include ADA and paper and pencil administration Table 2. Historical BOC Exam Scaled Score Summary Statistics Cohort N Mean Median Std Dev Min Max All 2021-22 4,118 532.6 540 107.7 200 800 First-time 2,624 568.4 580 102.9 200 800 Retake 1,494 469.8 480 84.8 200 710 All 2020-21 4,944 498.1 495 87.7 200 750 First-time 3,032 521.5 520 88.2 200 750 Retake 1,912 461.0 470 72.8 200 690 All 2019-20 5,528 540.2 532 66.1 358 735 First-time 3,912 558.8 558 65.2 382 735 Retake 1,616 495.2 489 42.3 358 683 All 2018-19 5,591 539.0 532 66.3 340 729 First-time 3,970 557.6 555 64.8 352 729 Retake 1,621 493.4 488 44.0 340 642 All 2017-18 5,369 536.9 538 71.3 275 743 First-time 4,012 557.8 560 65.1 304 743 Retake 1,357 475.0 478 49.9 275 665 All 2016-17 5,323 551.3 554 72.2 265 751 First-time 4,119 569.6 577 67.2 315 751 Retake 1,204 488.8 489 50.7 265 678 All 2015-16 5,367 543.5 544 67.5 217 729 First-time 4,059 561.2 564 62.8 356 729 Retake 1,308 488.6 489 49.9 217 661 All 2014-15 5,051 543.3 542 71.4 259 733 First-time 3,768 563.0 567 67.1 272 733 Retake 1,283 485.5 484 48.7 259 650 All 2013-14 4,889 529.7 536 57.1 272 680 First-time 3,679 545.1 548 51.9 332 680 Retake 1,210 482.8 488 45.5 272 650 All 2012-13 4,950 524.0 524 54.0 302 674 First-time 3,631 539.0 548 51.0 302 674 Retake 1,319 484.0 488 41.0 314 632 Exam Form Reliabilities and Other Summary Data The performance of the test forms used for the BOC Certification Exam and are consistent with reporting requirements for National Commission for Certifying Agencies accreditation. Reliability is assessed using Cronbach’s alpha (Cronbach, 1951), a measure typically used for estimating overall exam form reliability; Brennan-Kane (Brennan & Kane, 1977), a measure used for estimating the decision consistency (i.e., the reliability of pass/fail decisions based on the test); and the standard error of measurement (SEM) presented in raw score units, a measure of the precision of the exam form. Summary Statistics concerning the quality of the BOC Certification Exam as a measurement instrument indicate that the exam complies with psychometric requirements that pertain to certification and licensure tests. Notably, estimates of reliability and equivalence across forms for the various parts of the exam are acceptable. Likewise, candidate performance on all parts of the exam is consistent with the public protection mission of the BOC. References American Educational Research Association, American Psychological Association, National Council on Measurement in Education (2014). Standards for Educational and Psychological Testing. Washington, D.C.: AERA. Brennan, R. L., & Kane, M. T. (1977). An index of dependability for mastery tests. Journal of Educational Measurement, 14, 277–289. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. Equal Employment Opportunity Commission (EEOC), U.S. Civil Service Commission, U.S. Department of Labor, and U.S. Department of Justice. (1978). Uniform Guidelines on Employee Selection Procedures. Federal Register, 43 (166), 38290–38315. Impara, J. C., & Plake, B. S. (1997). Standard setting: An alternative approach. Journal of Educational Measurement, 34, 353–366. National Commission for Certifying Agencies (2016). Standards for the Accreditation of Certification Programs. Washington, DC: Institute for Credentialing Excellence. Kolen, M. J., & Brennan, R. L. (2004) Test Equating, Scaling and Linking: Methods and Practices Statistics for Social Science and Behavioral Sciences (2 ed.). Springer-Verlag New York Inc. Kuder, G. F., & Richardson, M. W. (1937). The theory of the estimation of test reliability. Psychometrika, 2, 151–160. Next >