This episode of the It Depends Podcast explores performance-based testing in professional certification and licensure. Amanda and Tim discuss the benefits and challenges of performance tests compared to traditional knowledge-based exams, covering various assessment types, implementation considerations, costs, and future trends in performance-based assessment.

Key questions answered:

• What is performance-based testing?
• What are some advantages of performance tests
• What challenges come with performance-based tests?

Topic Keywords:

Psychometrics • Validity • Reliability • Item Response Theory • Job Task Analysis • Cut Scores • Performance-Based Assessment • Test Security • Equating • Bias and Fairness

Bonus Question: What future trends do you see in performance testing and how should organizations prepare to adapt to these changes?

Find the answer here: ⁠⁠https://www.testcommunity.network/podcast-it-depends⁠⁠

Connect with us:⁠⁠⁠

Amanda Dainis, PhD, MPA⁠⁠⁠ ⁠⁠⁠⁠https://www.linkedin.com/in/amanda-dainis-phd-mpa-012b94119/⁠⁠⁠⁠⁠⁠⁠

Tim Burnett⁠⁠⁠ ⁠⁠⁠⁠https://www.linkedin.com/in/tburnett/⁠⁠⁠⁠

Ask a question: ⁠⁠⁠https://www.testcommunity.network/podcast-it-depends

Top Episode Takeaways:

📈 Test maintenance involves regularly monitoring test and item performance statistics to ensure content remains current and secure, and should be conducted annually at minimum or as frequently as monthly for large programs.

🔍 The more frequently a test is administered, the more often maintenance should be performed, including reviewing key test-level statistics, item-level metrics, and qualitative candidate feedback to proactively identify issues.

🛠️ Typical maintenance steps include analyzing test passing rates, reliability, item difficulty, discrimination, and distractor performance, investigating qualitative trends, and engaging subject matter experts to replace problematic items following standard development procedures.

Bonus Question: Can you list 3 key flags or indicators that would alert you to a problem with the maintenance of a test?

Find the answer here: ⁠https://www.testcommunity.network/podcast-it-depends⁠

Connect with us:⁠⁠⁠

Amanda Dainis, PhD, MPA⁠⁠⁠ ⁠⁠⁠https://www.linkedin.com/in/amanda-dainis-phd-mpa-012b94119/⁠⁠⁠⁠⁠⁠

Tim Burnett⁠⁠⁠ ⁠⁠⁠https://www.linkedin.com/in/tburnett/⁠⁠⁠

Ask a question: ⁠⁠https://www.testcommunity.network/podcast-it-depends

Top Episode Takeaways:

📐 Test blueprints provide the overall design and structure for an assessment. They define what content areas the test will cover and what proportion of the test each content area should comprise. Having a blueprint helps ensure the test adequately measures the intended knowledge and skills in a balanced way that reflects their importance.

🎯 Blueprints should be created for most types of assessments, including certification/licensure exams, educational tests, and even workplace skills tests. The blueprint is developed after conducting a job task analysis, curriculum review, or other process to determine the key knowledge and skills to be assessed. Subject matter experts should be involved in validating the blueprint.

✅ A good test blueprint will cover all the major content areas without overemphasizing or underemphasizing any topics, allocate questions to each content area proportionately to their importance, be validated by representative subject matter experts to ensure it reflects real-world requirements, be transparently shared with stakeholders so they understand what the test measures, and be updated periodically as the field evolves to ensure the test remains relevant and valid.

Bonus Question: Is there a particular platform or technology that you like to use when creating test blubprints?

Find the answer here: https://www.testcommunity.network/podcast-it-depends

Connect with us:⁠⁠⁠

Amanda Dainis, PhD, MPA⁠⁠⁠ ⁠⁠https://www.linkedin.com/in/amanda-dainis-phd-mpa-012b94119/⁠⁠⁠⁠⁠

Tim Burnett⁠⁠⁠ ⁠⁠https://www.linkedin.com/in/tburnett/⁠⁠

Ask a question: ⁠https://www.testcommunity.network/podcast-it-depends