Distinguishing Low, Medium and High Stakes Assessments

March 22, 2009 at 1:26 pm 2 comments


Trying to classify an assessments into a Low, Medium and High stakes category has some pros and cons.  On the plus side we can quick “range “ the time to create, deliver and report on an assessment, and range the impact to the person taking the assessment, and it can help conversations about an assessment.  All conversations have to be considered in context.  If you were having the conversation in the context of Psychometrics you would probably be distinguishing between low stakes and high stakes exams; if you were having the conversation with instructors and training you might be distinguishing assessments prior to a course, during a learning experience, tests after and course evaluations.

image

This chart can help us think about Low, Medium and High Stakes assessments in both context but without specific measure. Essentially it aids the conversation by promoting some distinctions and a vocabulary rather than providing a measurable outcome. That is in itself amusing because the world of assessments is all about the theory and technique of educational and psychological measurement.

In higher stakes assessments we tend to talk about candidates, in medium and lower stakes assessments we talk about students, employees and/or learners and in low stakes assessments we might talk about respondents. And so in our vocabulary, in the context of low, medium and high stakes assessments, we’ll talk about Participants who are the people that answer the questions in our assessments; they participate.

There are six terms that can help us provide some distinctions:

  1. Consequences to the Participant

    If the consequences to the participant are low then that helps us classify into a Low Stakes assessment but if the consequences are great (affecting Lives, Limbs, and/or Livelihoods) then it would be a High Stakes assessment.

  2. Legal Liabilities

    When stakes are high, consequences are high, and so in come the lawyers. I don’t want to turn this into a political debate but laws are written to protect our rights and lawyers help us understand the laws to do the right thing. All stakeholders in the assessment process have rights and responsibilities. Often this debate is taken from the side of the Participant but other Stakeholders have rights too. Assessments must be fair and reliable and fit for purpose. And we can’t go around certifying people to fix gas leaks that aren’t qualified.

  3. Proctoring/Invigilation

    When the stakes are high people are more motivated to cheat which requires that the assessment process is invigilated to prevent this and to promote trust in the assessment process.  As I fly around the world I woudl liek to know that my pilot and the air traffic controllers didn’t cheat on their exams. With certain kinds of, low stakes, assessments invigilation would provide an unwanted and undesirable level of supervision. Here’s some examples of assessments that we should probably not proctor/invigilate:

    1. Novice student taking an assessment where the goal is to provoke intrigue before a learning experience.
    2. Course evaluation where the moderator might choose to influence the outcome.
  4. Validity and Reliability

    Ideally all assessments should be reliable and valid.  They should work consistently over time and they should align with the subject matter that you are assessing. However, if we applied the same standards to low, medium and high stakes assessments we might never justify the costs for say Formative assessments or Course Evaluations.  We must always work ethically but we don’t need a $25,000 study on the validity and reliability of the assessment. When conducting a High Stakes test or exam we need to be sure that the assessment aligns with the topics that it is assessing and it is fair to all participant.

  5. Planning

    If you are administering a High Stakes test or exam to 5,000 Participants you’ll use a different plan than for 10 people evaluating a course. Planning will include considering how you’ll develop your assessment, how you’ll have expert(s) review, how will you maintain confidentiality and security, how will you deal with delivering the assessment, how will you provide accommodations for those with special needs, how will you report on the results to the Participant and Stakeholders.

  6. Psychometrician Involvement

    A Psychometrician is a Psychometrics professional familiar with the theory and techniques to measure knowledge, skills, abilities, attitudes, and personality traits.  Psychometricians are often involved with the development of High Stakes assessments such as tests and exams, and analyze the results to ensure that the assessments are valid and performing consistently and reliably.

It is worth reminding you that all assessments can be very valuable, within their context, regardless of how you categorize them.  Just because quizzes, designed to strengthen memory recall, and course evaluations, designed to measure in order to improve the learner’s environment, are low stakes does not mean that they provide low value. But those types of distinctions will follow in another blog entry!

Advertisements

Entry filed under: Assessment, Learning, Professional Life, Talent Management. Tags: , , , , , , , , .

Tests That Work: Designing and Delivering Fair and Practical Measurement Tools in the Workplace Types Of Assessments (Formative, Diagnostic, Summative, and Surveys)

2 Comments Add your own

  • […] Security” when they had insufficient distinctions around low, medium and high stakes assessments (see previous post on Distinguishing Low Medium and High Stakes Assessments) and/or the different types of assessments (see previous post on Types of Assessments Formative […]

    Reply
  • 2. Item Analysis « Eric Shepherd’s Blog  |  April 7, 2009 at 2:01 am

    […] You always look for Outcome Discrimination and Outcome Correlation to be positive and typically you look for values above +0.3 but your rigor and concern over the statistical properties of a test item will depend upon whether it is being used for low, medium and/or high stakes assessments. […]

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


Add to Technorati Favorites

Recent Posts

Categories


%d bloggers like this: