What’s in a name (1): scoring CORE measures

We may have caused a bit of confusion by introducing the term “Clinical score”.  Perhaps it’s not on the scale of the Capulet/Montague name tragedy (Shakespeare, 1591-1995?) but it may be worth explaining the scoring here as I do see mistakes and do get asked about this.

History

We started out scoring using the mean of the items and recommending pro-rating if not more than 10% of items were missing, i.e. using the mean of the remaining items.  That meant you could get a pro-rated mean overall score for the CORE-OM if as many as three items were missing, for the “non-risk” score if two of the non-risk items were missing, for the function and problems scores if one of their items was missing, and you couldn’t pro-rate if any items were missing for the well-being or risk scores.  You could get overall scores for the CORE-SF/A, CORE-SF/B if one of their items was missing (but not for domain scores as any missing item there means more than 10% of the items are missing).  Similarly, you could use a pro-rated score for the GP-CORE, the LD-CORE, the YP-CORE and the CORE-10 if one item was missing but pro-rating the CORE-5 was clearly impossible. 

All those scores had to lie between 0 and 4 by definition but they could be awkward looking numbers like 0.84 and over the early years we got feedback that many clinicians and managers didn’t like these “less than one and fractional” scores. 

“Clinical Scores”

With mixed feelings in the team, the idea of “Clinical Scores” came in: the item mean as above, but multiplied by 10 to get a score that in clinical samples would pretty much always be a x.y sort of number with x >= 1 and scores ranging between 0 and 40. The same rules about pro-rating were retained.  This “x10 = Clinical Score” gives that rather easy scoring for a complete CORE-10 or complete YP-CORE that the “Clinical Score” is just the sum of the 10 items completed (but if one item is omitted you still have to find the mean of the nine completed items and multiply that by 10).   For a completed  CORE-5 the route to the “Clinical Score” is almost equally easy: the Clinical Score is twice (2x) the sum of the five items’ scores.

We sometimes see people reporting the sum of the items: please don’t do that, we’ve never recommended that anywhere.  We also see people not saying explicitly that they’re using the original “mean item score” or the “Clinical Score”, please do say which you used even if it seems very obvious.  Finally, we encourage people always to be explicit about having used pro-rating (if you have) and to be explicit about numbers of incomplete questionnaires and numbers of items missed. This all maximise comparability of reports.  Non-comparable scoring may not be as lethal as Mantua family feud was to Romeo, Juliet and Mercutio, but it’s definitely to be avoided!

Reference

Shakespeare, W. (1591-1595, exact date uncertain) “Romeo and Juliet” available in many versions as the peer-reviewed format hadn’t been invented: quarto 1, quarto 2, first folio and later versions.  However, the fatal name issue is consistent in all.

Putting a CORE measure into software

Since the first of January (2015) anyone can put any CORE measure into software under the terms of the Creative Commons Attribution-NoDerivatives 4.0 International licence.  Previously, we had restricted this permission just to people using the measure in software just for a research project and otherwise only to CORE Information Management Systems (CORE IMS) and people sublicensing through them.  See our joint statement for more information.

The Creative Commons licence puts two restrictions on anyone putting the measures in software: they must acknowledge the origins of the measure and they mustn’t change the measures.  This is in line with the situation for paper reproduction of the measures which has always been on these “copyleft” terms.  However, both the acknowledgement and the “no changes” were easy for reproduction on paper: just print out the PDF and the acknowledgement is there in the copyright statement at the foot of every page and any printing to any sensible printer from PDF guaranteed no meaningful changes.

The situation is not so clear when a measure is put into software and we’re gaining experience, and remembering some of the early learning we did with CIMS some years back.

Acknowledgement is fairly easy: if the line “Copyright to CST: http://staging.coresystemtrust.org.uk/copyright.pdf” that is in the PDF versions is there, and if ideally the link is clickable and opens to that URL, then you have done the minimum we ask by way of acknowledgement.

No changes is  a bit more complicated as obviously the formatting is pretty well bound to change depending on the device on which the text appears.  Where the intention is for end users (patients, clients, service users, research or survey participants) to use the measure in the software the following must be true:

  1. The introduction (“IMPORTANT – PLEASE READ THIS FIRST This form has 34 statements about …” should be unchanged in content but the line “Please use a dark pen (not pencil) and tick clearly within the boxes.” can be replaced with whatever is sensible to tell the user how to fill in the items.
  2. The time frame (“Over the last week”) must come before the items and at the head of each block of items.  If the items are presented oen at a time, this must be there with each item.  We recommend if the items are presented sequentially that the system offer an option to see the introductory instructions (#1 above) with every page.
  3. The response anchors (“Not at all” to “Most or all the time”) must be unchanged and against each response option.
  4. Clearly the “Please turn over” instruction from the CORE-OM can be dropped but something similar should be used to try to ensure that people do page through all the items.
  5. The system does not have to calculate the overall score and present it to the end user.  If it does, it must do so correctly, see #6.
  6. Scoring is simple and the “clinical scoring” of 10x the the mean item score is used for all measures.  Perhaps counterintuitively the “well being” domain score is scored in the same direction as the other domains, i.e. higher scores indicate lower well being.  Prorating can be applied for any score provided that fewer than 10% of the items on the score have been omitted.  Clearly for scores with fewer than ten items this means that no prorating of missing items is supported.  It is the responsibility of whoever is putting the measure into software to ensure that the right items are mapped to the right scores, that “positively cued” items (e.g. “I have been happy with the things I have done” have their scores included correctly (i.e. scoring 4,3,2,1,0 rather than 0,1,2,3,4 for the “problem cued” items) and that scoring is correct.
  7. We are happy to review any implementation with you if we have time for this. Currently, if this is not particularly urgent, we do not charge for this but we may have to change that if the work load becomes significant.
  8. We could provide sample data to be entered for all the measures that could be used to check scoring and can help with that but that is a significant piece of work for which we would have to charge and sample data are under development currently.  The responsibilty will still remain with whoever programs the scoring.

This is all work in development and we will turn this into an FAQ as experience develops but experience so far has been that complying with the licence conditions and checking that is not onerous for the programmers/designers nor for us to check.

Do get in touch with us if you would like us to work with you on this or if you are confident you have done it fine and just want us to know you are making the measures available with computer support.

Best wishes,

Chris Evans (for CST)