This validation process ensures our examinations meet the highest testing standards.
Job Analysis
To establish a valid certification exam, we perform a job analysis (JA) for the exam’s discipline. Subject Matter Expert (SME) workshops are held to develop national operator surveys focusing on the tasks and capabilities required for successful job performance. Operational experts in the field complete these surveys to ensure their daily operational duties and technologies are reflected in the next generation of standardized operator certification examinations. They also rate the job tasks on scales for frequency of performance and potential seriousness of inadequate performance. The data from the responses is analyzed to prepare the exam content outlines (Need-to-Know Criteria) reflecting the most widely performed and significant job tasks, build updated operator certification exams, and create additional resources that are used as the cornerstone for certification programs across North America and the globe. For exams in use currently (2023), more than 5,000 industry stakeholders provided input on the significance of approximately 730 job tasks.
Item Development
Training and guidelines provided by WPI assist SMEs in developing effective test items. Only items that meet WPI’s specifications and cover topics from the Need-to-Know Criteria are accepted. Each exam item written for one of WPI’s standardized certification exams is developed through a rigorous psychometric process, is referenced to a chapter and page of industry reference material, has a statistical backing, and a psychometric analysis to determine its appropriateness to appear on a certification exam designed to assess the competence of an operator. Test items are not designed to be subjective nor to have a ‘best’ answer. Due to these strict item writing parameters, all WPI standardized exams are legally defensible.
Item Validation
Each phase of exam development is conducted by diverse committees of highly qualified SME volunteers from across the United States and Canada under the guidance of WPI’s psychometric partner, PSI Services LLC. WPI recruits and retains item writers from diverse backgrounds and locales to create our exam content. All SME item writers are selected based on very discerning criteria. They must:
- Hold an active Class IV (or upper level) certification in the jurisdiction in which they are licensed and for the discipline for which they are writing.
- Be actively working in the water or wastewater industry.
- Have at least 10-15 years of operating experience (most have 25-30 years).
- Be approved by C2EP as a qualified SME meeting the criteria above.
A minimum of five SMEs must validate all new questions before inclusion in an exam. Each item’s technical accuracy and relevance are verified during validation. In addition, the reference from which the item originates is reviewed to ensure that all referred content is current and accurate. Experts also confirm that the correct answer is the best answer and that the distractors are incorrect yet plausible. Furthermore, the WPI staff will evaluate each question for style and readability.
Our detailed validation procedure is described in:
- The pamphlet Validating Your Certification Exam
