TAEUSworks™ uses a carefully selected panel of subject matter experts (SMEs). These SMEs are industry leaders with experience and excellent professional reputations from all areas of the Patent Evaluation domain. To become a Patent Evaluator an SME must be certified.
Certification Test Development
In order to access and certify prospective Patent Evaluators, TAEUSworks™ uses online training technology and a network of third party courseware developers. These developers design scalable instruction models, instruction syllabuses for classroom and on-line use, certification testing and support aligned to geographic regions where the TAEUSworks™ Patent Evaluator Certification is available.
The TAEUSworks™ Administrator follows established industry practice (American Psychological Association, APA Standards) to develop a valid computer-administered test that will accurately determine whether or not a candidate possesses the defined skills. Certification test development capability can be broken down into four main components: Test Design, Test Item Development, Psychometric Analysis, and Test Construction and Distribution. The test resulting in certification is valid if it measures what it’s supposed to measure. Validity is not a mathematical construct; it is the result of a structured, documented process.
The structured documented process that is used to develop a valid certification of Patent Evaluators is comprised of the following elements:
▶ Define the skill and identify the components which make up that skill.
▶ Translate the components of the skill into specific objectives which define not only specific content but also a level of mastery.
▶ Construct a blueprint that considers the relative importance of the objectives.
▶ Use the blueprint to develop test items to measure a candidate’s knowledge of each objective. An ideal set of test items contains three times more items than are anticipated to be on the test. This allows for two test forms using the best items.
▶ Perform a technical review of the test items to fine-tune the items, ensure accuracy, and eliminate potential problems with the items.
▶ Develop the profile of a minimally qualified candidate to be used later in determining a cut score.
▶ Run a beta-test of the items with a population of candidates. An ideal beta-test has all the candidates answering all the items.
▶ Perform psychometric analysis of the items to ensure that they are doing what they are supposed to be doing, that is, identifying who of the candidates is qualified and who is not.
▶ Put the best test items into two statistically and psychometrically equal test forms (Form A and Form B).
▶ Score each of the beta candidates on each form and perform cut score analysis, ultimately resulting in the selection of a cut-score.
▶ Distribute the test forms globally through established certification testing centers.
Psychometric analysis is the technique to collect data from the administration of the test items and compute statistical properties of the test items. It evaluates the suitability of the test items for the purposes of measurement. Deliverables from this component include item analysis, test analysis, standard setting analyses, dynamic test simulation and analysis for different groups and item parameter drift over time, and custom research.
The TAEUSworks™ administrator has existing relationships with qualified psychometricians with over 20 years of psychometric analysis. Their unique competence is analysis into existing and potential measurement algorithms, developing custom software related to certification testing, and validating and documenting algorithm accuracy. The psychometrician is involved at every phase of the process, but the actual work of constructing the blueprint, developing test items, technically reviewing them, developing the profile, and determining a cut score requires the participation of subject-matter experts. The psychometrician will edit the test items to make sure that language is not a stumbling block. Naturally, the psychometrician will do the psychometric analysis and provide the analysis necessary to determine the cut score.
Evaluation Validation Process
Occasionally there will be patents on which members of the initial panel disagree substantially on the analysis. This type of discrepancy can occur in any environment where there are multiple raters. When this event occurs, the panelists first discuss their ratings among themselves and see if they can come to an agreement. If they cannot, then other expert panelists are brought in until there is sufficient agreement and resolution.
The selection and certification of expert patent evaluators, facilitated by a well-documented patent evaluation process and a sound certification test will ensure that the evaluation of the patents in a PatentBook reflect the true value of each patent.