NCWM.net

First Draft - January 2007

This guide was prepared to assist those work groups preparing curriculum materials as they prepare test questions. These test questions will be used both as aids to training delivery and also as a measuring stick in any future certification effort. If the certification program is to have credibility, it is vital that the test questions adequately evaluate that the student has achieved the multiple milestones in each curriculum area.

As you write your questions, please remember that we have set the bar at a level of application, the third in Bloom’s taxonomy. Thus we expect that the trainee will KNOW certain things, UNDERSTAND other things, and be able to APPLY the remainder. We are not looking for higher learning levels in Bloom’s Taxonomy for basic inspectors and we will not be testing for analysis, integration, or evaluation.

Testing for Knowledge

A test question for knowledge is usually in the form of a true/false, multiple choice, or fill-in-the blank question. At this point, the Committee is suggesting that developers focus on multiple choice and fill-in-the blank questions, such as questions 1 and 2 below. With true/false the person has a 50-50 chance of guessing and getting the right answer. Please note that at this level the trainee need only demonstrate that he/she knows the information and not necessarily that he/she understands it or can apply it.

1.  Which statement best describes the legal standing of NIST Handbook 44? (Answer: B) 

1. Handbook 44 is a federal regulation published by the National Institute of Standards and Technology that preempts the states.
2. Handbook 44 is adopted either by act of the state legislature or through promulgation in  regulation by the state.
3. Handbook 44 is amended each year and all states agree to abide by the actions of the National Conference on Weights and Measures.
4. Handbook 44 is adopted as part of the administrative policy by order of the state director.

2.  A paragraph beginning with "S." in any of the NIST Handbook 44 Codes is a ____? (Answer: Specification)

Testing for Understanding

A test question for understanding is usually a multiple choice question, such as questions 3 and 4 below. Questions concerning understanding often ask the trainee to pick the best response in situations where more than one answer could be correct in some respect. For example, in Question 3, answer B could be a correct answer if the equipment was manufactured after the effective date. Answer C. is a better answer since it is more specific and also includes items brought into the state after the effective date. Please note for understanding the trainee needs to demonstrate that he/she knows and understands the information and not necessarily that he/she can apply it.

3.  A nonretroactive requirement is best described by which of the following statements? (Answer: C)

1. A nonretroactive requirement is enforceable on all equipment up to the terminal date.
2. A nonretroactive requirement is enforceable only on new equipment after the effective date.
3. A nonretroactive requirement is enforceable on equipment manufactured after the effective date or brought into the state after the effective date.
4. A nonretroactive requirement is enforceable on equipment with an NTEP Certificate granted after the effective date.

4.  Which of the following best describes the difference between "d" and "e" in the Scales Code? (Answer: D)

1. The value of “e” is always displayed while “d” may or may not be.
2. The value of “d” is always smaller than or equal to “e”.
3. The display of values for “d” must always be different in size or character from “e”.
4. When “d” does not equal “e” the tolerances are applied to the value of “e”.

Testing for Application

A test question for application should be either be a multiple choice question or a “Yes/No with reason” question, such as questions 5 and 6 below. Questions concerning application will usually require the trainee to perform multiple steeps to reach the correct answer. In the field, they will not be guided to the correct section of the handbook, but will have to find it based on their knowledge and experience. For example, the question may provide information about the situation and some test results. The trainee must then decide whether to apply maintenance or acceptance tolerances and then evaluate the test results against the appropriate tolerances for that test. In question 5 below, the person must see that the scale is subject to the non-retroactive requirement in Scales Code S.1.7.(b) and then correctly deduce that the only correct response is an overload error. The Yes/No with reason question (question 6) also requires several steps but goes further in that it also requires the trainee to state the nature of any violation and cite the section of the Handbook that is violated. This is critical as this reason and citation would have to be indicated on any official stop-use order issued for the violation. Please note that the trainee needs to demonstrate that he/she knows, understands, and can apply the requirements.

5.   You are inspecting a new price computing sale (30 x 0.01 lb) in a deli that was placed in service last week. It has an NTEP CC # 99-205. You place a 1 lb weight on the scale and press the tare key. You then place an additional 29.2 pounds of test weights on the scale. Which of the following is an acceptable indication for this test load? (Answer: A)

1. Overload error
2. 29.24 lb
3. 29.18 lb
4. 29.16 lb

6.  You are inspecting the scale at right and find that it has no zero tracking. With the scale at zero as indicated, you add 0.1 d (0.002 lb) to the platform and the scale indicates a stable 0.02 lb. Is this acceptable? Yes or No (No must include reason and citation) ____. (Answer: No –The digital zero indication must be maintained accurate within +/- ¼ d of true zero or the scale must have a center zero indicator. Scales Code S.1.1.1.)

Initially the Committee is looking to build a bank of test questions that evaluate if the trainee has reached the milestones in each curriculum segment and cover a range of difficulty. Any exam that is prepared will include a mix of questions at each appropriate level in Bloom’s Taxonomy from the curriculum, and varying levels of difficulty from easy to challenging. In that way, the test can be fair yet still differentiate those that really have mastered the discipline from those that haven’t.

After the questions are prepared and tested (testing method to be developed), the Committee would then split the questions into two groups. The first group, called “sample questions”, would be widely circulated for use in training programs. Instructors could use the sample questions in their training or as part of quizzes or final exams to measure effectiveness of the training. Most important, trainees would be exposed to the kinds of questions and the range of difficulty that would be included in a certification exam.

The second group of questions would be secured for use in a certification exam program. The Committee envisions charging some group to administer the certification exam and assist in the grading. That group would also create alternative exams or periodically change the questions so the exam is not the same for candidates that fail to pass the first time. Please look to set the bar so it is fair yet represents the high level of ability you want working for you.

A long journey begins with one step. We are counting on our curriculum development teams to start generating our bank of test questions (with an answer key) based on the milestones they choose in the curriculum segment(s) they are preparing. If we work together to create a good range of difficulty in those questions, we can be well on our way toward that certification program we are shooting for. There is plenty of room for creativity in this effort, including the use of graphics and photographs.

back to top