Assessment Design
Bespoke Situational Judgment Tests
Situational Judgement Tests (SJTs) are increasingly popular tools for use in selection and development presenting candidates with realistic, job-related situations, either described in writing or more recently on-line and web-based. A well designed assessment can be aligned to specific role based competences and underpinned by an objective scoring model. With online SJTs, organisations are able to assess complex capabilities, such as decision making, across a large number of candidates at low delivery cost with consistency and traceability. SJTs have been found to be valid predictors of job performance.
Sample question from a customer services situational judgment test.
Design Approach
We will design a bespoke Situational Judgement Test modelled on the behaviours of your highest performers. Our design service follows these major stages.
Job situation analysis
Building on role and competence profiles and using work analysis techniques we define the key activities and competencies for the role. We then define and explore a range of job-relevant situations based on the real-life experiences of high performers and experts. We work in small focus groups and interviews, using techniques such as behavioural event interviewing and critical incident analysis. The relevance of the situations is cross-validated with a large independent sample to optimise experts' time.
Assessment item writing
Working from this source material our specialist designers write the test items, removing the burden from experts and avoiding the common pitfalls. The situations and options are generalised from the real-life sources and to match the level of experience of the target audience. For example if the assessment is to be used for external recruitment the situations we need to remove reliance on knowledge of internal procedures which the candidates cannot be expected to have. The items are anchored to the key behavioural competencies, providing further insight through the choices candidates make in the final test.
Establishing an objective scoring model
The design process includes selecting and applying an appropriate scoring model. Scoring models generally require the candidate to make multiple decisions about each item, such as identifying both the most effective and the least effective responses to a situation, or rating the effectiveness of each option and an algorithm to calculate an item score based on the responses provided.
Ensuring job relevance
Each item is reviewed for relevance and reliability, and with a diversity review board before being included in the pilot assessment. Reviewing the situations across a broad sample of current role holders ensures the situations are job-relevant and will be recognised across the broad range of contexts in which the role-holders work.
Validation
The assessment is validated with reference groups and the results are correlated with available performance data and other relevant measures.
Test delivery and scoring
The final materials include the administration guide, which details the administrative process and a guide for interpreting the results. Our SJTs are normally delivered on-line using Talent Spotter®, which automates the marking process. Final materials may include printed versions of the assessment and the associated materials required for manually administered tests.
