Next, we want to talk a little bit about some educational concepts, and we're going to begin our educational concepts with something called Bloom's Taxonomy. It's very famous among educators. So Bloom's Taxonomy is a hierarchy of learning, and when we write questions, we're really trying to strive to assess as high a level as possible on a test. Usually we're satisfied when a board-style multiple choice test question hits somewhere in the middle of this taxonomy. We don't just want to recall a fact. We want to apply that knowledge that we have to relevant clinical scenarios. So when you think about it, it really isn't about remembering little minutiae and factoids. So don't make your test questions about little minutiae and factoids, even if they're easy questions to write. We really want things to be clinically relevant. So as you decide what you're going to write about or what you've been assigned to write about and how to make that a board-style question, we want you to be able to apply that information into something clinically meaningful. Now there are three cornerstone concepts of test item writing. The first is fairness. That means that someone who knows the material should not get the question wrong just because of the way the item is constructed. It shouldn't be confusing. It shouldn't be designed to be tricky. It should just test what it is that they know and not be right or wrong because of the way someone reads the question. The next cornerstone concept is validity. Someone who does not know the material should not be able to answer the question correctly because of the way the item is constructed. This means that we should be avoiding testmanship-style questions where someone can get the right answer simply by being a good test taker and assessing the way the question is worded, even though they really have no knowledge about the topic. I can tell you, as I've been reviewing some of the boards that I have seen with their maintenance of certification tests, I really think that I could pass the maintenance of certification test for certain other boards even though I haven't trained a day in their specialty simply because I'm a good test taker and I can figure it out because of good testmanship. That is an issue of validity when you can pass in that way, and we want to avoid that in the way that we design our questions. The final cornerstone concept of test item writing is value, and that means that the test item should be test information that's worth testing. You know there are a couple of important points of why we do pick the kinds of test questions that we pick on the board exams. One is so that it discriminates. If every single person gets the question right, even though it's a question that is information that should be known, if everybody already knows it, there's no point in having it on the test because it doesn't discriminate anymore. So you're going to find questions that are harder when you're taking a board-style test because we're not testing the most basic knowledge on a board certification test because every single question that we offer on a test needs to discriminate in some way, otherwise it has no real value. But at the same time, there needs to be an inherent importance to the information. So just because a test question discriminates a testing population does not mean it's information worth testing. So it's not just about discrimination, it's also about the value of the information. So keep that in mind.

Bloom's Taxonomy

learning hierarchy

test questions

clinically relevant

cornerstone concepts