Welcome to the Script Concordance Test
of the MBBS Program, Faculty of Health Sciences
The University of Adelaide, 2009

FOR STUDENTS AND EXAMINERS: Select this link for a discussion of how to answer SCT questions

FOR AUTHORS: Select this link for templates for new questions 

  FOR STUDENTS: Select this link for a discussion of how SCT examinations are graded  

 Select this link to practise answering SCT questions 

                                Select this link for a discussion of the expert reference panel results to the above questions

Follow these links for examples and discussion of SCT questions that have NOT yet been subject to expert reference panel review: 

Obstetrics and Gynaecology                 Surgery            Medicine          Paediatrics         Psychiatry             General Practice

For more information about this research, please contact:

• Dr. Paul Duggan, Senior Lecturer, Women's Health Centre, Royal Adelaide Hospital

Overview

Introduction

Welcome to an overview of the Script Concordance Test!

This page contains some background information about script theory and the script concordance test (SCT). You will also find instructions about how to complete the test, along with a practice example to familiarize you with its format.  

Background

The primary goal of a medical student or resident is to become a competent medical practitioner. An important part of competency in medicine, in addition to the development of a rich fund of knowledge, is the ability to make clinical judgments in uncertain or ambiguous situations. Doctors often face ill-defined problems during their routine clinical activities, and are equipped with decision-making skills that allow them to solve these problems efficiently and appropriately.

The traditional forms of examination that you are familiar with, such as multiple choice questionnaires and short-answer questions, assess "factual" knowledge very well, but do not necessarily evaluate clinical judgment. The Script Concordance Test (SCT) is designed to assess clinical judgment skills.

What are scripts?

The SCT is based on script theory. "Scripts" are defined as networks of knowledge that are stored in an individual's knowledge base. They are not facts; they are the links between facts that we accumulate throughout our training. As we evolve from novices to experts in medicine, these links develop and become organized, permitting more experienced clinicians to retrieve and use stored information in a rapid and efficient manner - that is, to make quick and sound clinical judgments. Scripts begin to form during the very first clinical encounter, and become refined with increasing patient exposures.

Example of a script

Let me give you a simple example of a script. One fact that you might have learned about headaches is that some have a "throbbing" quality. Another fact you may have acquired is that some headaches are "unilateral." Doctors are likely to encounter many patients with "throbbing" headaches who also report that their headaches are "unilateral"; that is, "throbbing" and "unilateral" are often-linked features of a common type of headache. The more patients you see with this combination of features, the more the connection between them strengthens in your knowledge base.

A third fact about headaches is that some "respond to indomethacin." You may have noticed that some of your patients with throbbing headaches do, indeed, improve with indomethacin. But as you gain experience you might find that the association between "throbbing" and "response to indomethacin" weakens. If you have never encountered a patient with throbbing headache who responds to indomethacin, a link between these features may not exist at all in your knowledge base.

What I have just described - this intricate network of connections or links between facts - is what is referred to as a "script". The particular set of links-between-features that I have put forth in this example may resemble part of a "Migraine script" that you have developed in the course of your training.  

Test Instructions

The SCT builds on the script model to test clinical judgment skills in examinees. Here is how it works (please read the following carefully):

Structure of the SCT

Stem –a clinical problem or scenario

Hypothesis or plan of action

Additional information

Decision based on additional information

How to answer a SCT question

Although the SCT question is primarily asking you to evaluate the effect of new information on an hypothesis or plan of action the stem is necessary to provide a clinical context to the question.  This will constrain the thinking of candidates and examiners alike within realistic boundaries and help recall relevant scripts.

The simplest way of answering a SCT question is to extend in your mind the Likert response by adding the phrase “than it was before the new information became available” as shown below.

A)much less likely than it was before the new information became available 

B)a little less likely than it was before the new information became available 

C)neither more nor less likely than it was before the new information becameavailable 

D)a little more likely than it was before the new information became available 

E)much more likely than it was before the new information became available 

In this approach all you need to consider is the effect that the new information has on the likelihood of the hypothesis or appropriateness of the proposed investigation or action.  You do not need to consider anything else.

Strategic answering

It makes sense if you are not sure whether new information changes things a little or a lot to always go for the less extreme option. In summative assessments the SCT questions are chosen to have a spread of modal responses that covers the range of Likert responses, so that strategy is not likely to be successful.

Can I rely on the information in the scenario and the question?

SCTquestions use clinical encounters in which there is uncertainty.  The level of uncertainty in a question reflects uncertainty in the real world. Some of this uncertainty is due to incomplete understanding of conditions, some of it is due to systematic error (which is predictable) and some (far less) is due to random error (which is not predictable).  What I have called “systematic error” is an important consideration in medical practice.  This type of error can be found in the reliability of a history through to reliability of examination findings and the accuracy of investigations.  The level of uncertainty is contextual and is to be taken at face value. For example, a history from a demented person is going to be much less reliable than from a normal, healthy person. Abdominal examination findings in an adult weighing180kg will be far less reliable than in someone weighing 60kg.  Some investigations are very reliable (e.g. serum HCG) and others less so (e.g.pelvic ultrasound). These are all factors that the expert clinician will take in to account in formulating a diagnosis and management plan. We expect you also to factor systematic error in to your decision-making, but we don’t expect you to be as good at this as an expert.

Example of a SCT question

You are evaluating a 60 year old man with a left hemiparesis in the emergency room.

As you can see, each case is followed by a series of questions. The above example case has 2 questions; others may have 3 to 5.  Each question provides different information for you to consider.  For each question, you are to choose one of the response options -2, -1, 0, +1 or +2. 

It is important to understand that each question is independent of every other question even though they refer to the same Clinical Case.  This means, in the above example, when you are considering "ischemic stroke" the ONLY additional information to consider is the sudden onset of the symptoms - i.e. the patient has NOT had dental work as well as the sudden onset of symptoms.

For Question 1, you need to consider whether a cerebral abscess as a possible cause of presentation is more or less likely if the patient had dental work ten days ago.  There is no absolutely correct answer to this question.  However, if you were thinking that a cerebral abscess could be due to a vegetative embolism, for example from a faulty mitral valve that was seeded with oral bacteria at the time of the dental work, you would probably choose to answer "+1: a little more likely".  If you think that is not a likely or relevant cause of a cerebral abscess, you would probably choose "0: neither less nor more likely" and so on. 

It is also important to note that the table headings and response options differ in different clinical cases, so please read them carefully.

Scoring

In a multiple choice questionnaire, there is only one right answer (e.g. A,B,C,D, or E). The SCT is scored somewhat differently, taking into account the variability in the responses of experts to particular clinical situations.

Your responses to each item will be compared with those of an expert panel. Your score will reflect how closely your clinical decisions match, or concord, with those of experts in the discipline.

Suppose, for example, that there are 10 members on the expert panel. On a given question, 8 members out of 10 may judge one answer (eg. +1) to be the best, and 2 out of 10 may choose a different answer (eg. +2). In this case, you will receive the most credit for answering like the 8/10 experts, less credit for answering like the 2/10 experts, and no credit at all for answering in any other way (eg. -2, -1, or 0). You are not penalized for responses that are not selected by the panel members.

The idea is that both examinees and experts activate relevant scripts to make their judgments; the SCT measures the gap between your scripts and those of experts. The smaller the gap, the closer you are to thinking like an expert.

Adapted from Dr Stuart Lubarsky, The Script Concordance Test in Neurology, McGill University, Canada

Revised May 2009