As a certifying organization serving the radiology community since 1934, The American Board of Radiology (ABR) is not a stranger to inquiries and concerns regarding its policies and procedures from well-meaning members of the radiology community. In fact, we welcome this feedback, which most often comes to us directly through our own surveys, professional organizations, and individual radiologists, and we react accordingly. In this instance, we have been invited to respond to an opinion piece entitled “The Validity and Timing of the ABR Core Exam,” by Kerridge and Gunderman . We are pleased to be able to do so.
In their article, the authors state that we as radiologists should “subject the board exam to continuousscrutiny,” and in this case they have selected the Core Exam as their intended subject for analysis. Because the ABR serves the entire radiology community, we, in return, have a reciprocal responsibility to scrutinize this critique in order to extract any valid criticisms and suggestions, as well as to expose and correct interpretations of information unsupported by research or based only on unsubstantiated anecdotes or innuendo. In this case, we have had an opportunity to do both. Interestingly, counter to our expectation to learn more of an objective case for perceived weaknesses in the Core Exam, the article’s opening pronouncement that the Core Exam is “not working”—an oddly biased beginning for an open-minded assessment—certainly raises the stakes for a high level of proof for such a judgment. The pass/fail results for the Core Exam have been similar to previous ABR diagnostic radiology examinations, and our statistics indicate that it is a psychometrically sound test with high reliability and validity. So, how did the authors come to this conclusion?
In our reading of the article, it is apparent that this supposition by Kerridge and Gunderman was derived in large part through fundamental misunderstandings of the construct and purpose of the Core Exam, as well as by ascribing attributes to the examination that were never intended or, in some cases, unlikely to be achieved by any single examination. Not surprisingly, this disconnect led to the erroneous conclusion that the Core Exam is “not working” because it is not doing what it is actually not designed to do. The most constructive approach for us to use to respond to these allegations of deficiencies in the examination is to review for our readers the intended purpose of the Core Exam, as well as its successes in achieving this purpose; to illuminate what the Core Exam is not intended to do; and to acknowledge areas in which the process may be improved.
Purpose of the Core Exam
The Core Exam is the first step in the certification process for diagnostic radiology, designed to determine whether or not a candidate demonstrates acquisition of the core professional knowledge covering a broad spectrum of the specialty. Because possession of core knowledge serves as the underlying platform for further learning, the Core Exam is administered at precisely the time that many people believe a core curriculum should be mastered—after 36 months of learning and before residents enter the final phase of training in which they refine their clinical skills and pursue more specialized knowledge, should they so choose.
Psychometric Validity of the Core Exam
The authors begin their paper by revealing that they have structured their analysis of the Core Exam based on its achievement of six types of validity. These, we learn, are derived from a list of six types of validity as embedded in one section of ABR’s Quality and Safety Domain Specification and Resource Guide , which provides definitions of different types of validity that may appear as questions on the Core Exam. We mention this only to discount any implication that all of these six types of validity were ever stated by the ABR to be significant psychometric components of the Core Exam. In fact, we know of no examination in existence that accomplishes all of these in equal representation.
The analysis of validities as applied to the Core Exam that follows suffers from a basic lack of thorough understanding of what most core knowledge examinations are intended to accomplish. As correctly quoted by the authors from the ABR website, the ABR Core Exam is an initial qualifying examination that “tests knowledge and comprehension” important for diagnostic radiology. It answers the basic question of whether a candidate is able to demonstrate acquisition and understanding of a core body of information that is basic and relevant to our field. However, as the article progresses, much of the discussion concerning the ABR Core Exam’s validity recycles repeatedly to the Core Exam’s failure to measure or predict accurately a host of other factors unrelated to the purpose of the examination, sometimes distorting the definitions of validity in the process.
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
What the Core Exam Is Not Intended to Do
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Timing of the Exam
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Conclusion
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
Get Radiology Tree app to read full this article<
References
1. Kerridge W.D., Gunderman R.G.: The validity and timing of the ABR core exam. Acad Radiol 2016;
2. The American Board of Radiology : Quality and safety domain specification and resource guide.2016.The American Board of RadiologyTucsonpp. 34-35.
3. Gunderman R.B., Ballenger Z., Heitkamp D.E.: Avoiding testocracy. Radiology 2012; 256: pp. 332-335.
4. Gunderman R.B.: The perils of testing. Acad Radiol 2001; 8: pp. 1257-1259.