The next installment of the online twitter journal club for nephrology (#NephJC) is the recently reported paper in NEJM by Smith-Blindman et al that examines how to image patients in the ED with suspected kidney stones. This article is also being discussed by the urology online twitter journal club (#UroJC). Go to www.nephjc.com for a quick tutorial on how this works. The twitter chat will be held at 9PM EST on Oct 7th. However, feel free to chat about this paper at anytime using the #NephJC hashtag on twitter. Below is a summary of the study originally posted on NephJC.com by the NephJC co-founder and eAJKD contributor Dr. Swapnil Hiremath.
What is this paper about?
This was a multi-centre trial, carried out in 15 academic hospitals, scattered over continental US from East, West and the Middle, with UCSF as the co-ordinating centre. At UCSF, there is representation from Radiology, Epidemiology, Urology and Emergency medicine physicians, but at other centres it is mainly the ER docs who are site investigators. This makes sense because the participants were recruited from the emergency departments – so adult patients with abdominal pain presenting to the ER.
Given the pragmatic nature of comparative effectiveness research, these are appropriately broad, to mimic real life clinical decisions.
Age 18 to 76 years
Flank or Abdominal pain
ER physician decided imaging was necessary to rule out a primary diagnosis of kidney stones
ER physician considered that patient was at high risk of serious alternate diagnoses (such as acute cholecystitis, appendicitis, bowel disorders, aortic aneurysm)
Men > 129 kg and women > 113 kg
Single kidney, kidney transplant, dialysis patients
- Participants were then randomized to three options:
- Point-of-care ultrasounds performed by ER physicians (who, it should be noted, had undergone training per the ACEP)
- Ultrasound performed in the Radiology department
- CT scans
This was an open label study, so everyone knew which study was being done (and admittedly this is hard to bind!).
This is where the story starts becoming quite interesting. Diagnostic accuracy was……not the primary outcome – it is, in fact, listed last even amongst the secondary outcomes.
The primary outcomes were:
- high risk diagnoses with complications related to missed or delayed diagnosis (i.e. may or may not be related to kidney stones)
- cumulative radiation exposure (see this for a nice explanation of radiation measures)
- cost, which is still being analysed and not completely presented in this manuscript.
The clinical outcomes were assessed at 3, 7, 30, 90 and 180 days. The definitions and adjudication seems quite meticulous and appropriate.
Secondary outcomes included
- serious adverse events
- return ED visits
- hospitalization after discharge from ED
- self reported pain scores
- diagnostic accuracy
The assumption for sample size was to detect a difference between study groups of 5% for events with a prevalence of 10%, 0.34% for events with a prevalence of 0.5% and 0.14 SD for radiation exposure. [NB: is prevalence the right word to use, as the authors do or incidence?]
As figure 1 shows nicely, of the 3638 patients screened, a good proportion, 2776, were randomized, and with a small number (3-5%) lost to follow up, about 900 patients in each group are included in the analysis. As expected in a study of this size, table 1 shows balanced baseline features, with roughly half women, median age ~ 40 years, with a pain score of about 8 (scale from 1 to 11). The clinical examination findings also look nicely spread (table 2).
Primary outcome 1:
This refers to adverse events from delayed or missed diagnosis. Very few patients had these: 6, 3 and 2, respectively – unsurprisingly not significantly different, but raising the question of adequacy of sample size with such low event rates.
Primary outcome 2:
Arguably, radiation exposure stacks this against CT, which obviously lost out – but not knocked out, with an exposure of 10.1 and 9.3 mSv with the two ultrasound techniques versus 17.2 with CT (p <0.001). Now why would ultrasound patients have radiation exposure at all? The more astute readers should know what’s coming….
Primary Outcome 3:
Now, though this analysis is not complete, they do state the cost was different, and on average, $25 were spent more on the ultrasounds done in the radiology dept (group 2) compared to CT. The point of care ultrasounds (group 1) were also cheaper, but we don’t know by how much. Yet.
Table 3 shows all the outcomes, serious adverse events were many – but all spread such that there were no significant differences. Though not listed in the methods, length of stay now pops up here, and was significantly higher (median 7 hours) in group 2 (ultrasound in the radiology dept) compared to the other two groups.
Diagnostic accuracy needs some unpacking. Table 3 presents the proportion of confirmed diagnosis at 6 months compared to diagnosis at end of ED visit irrespective of all the actual imaging tests performed. To note, confirmation of diagnosis was rather stringent (surgical removal – or passage of actual stone). Roughly a third in each group had confirmed diagnosis – with a p of 0.39. So why the emphasis above? Well, the diagnosis at the end of the ED visit was not always based on the ultrasound alone in Groups 1 and 2 – 40.7% in group 1 and 27% in group 2 also underwent a CT (and 5% in the CT group had an ultrasound) – so this diagnostic accuracy is a ‘pragmatic’, intention to treat and not a ‘purist’ definition. If one restricts the diagnostic accuracy to the first test only – the results are dramatically different (ultrasound had significantly lower sensitivity and specificity that CT). This would also explain the smaller than expected difference in radiation exposure and cost.
This study is one of those funded by the AHRQ, from the ‘comparative effectiveness’ research (CER) funding, which has been promoted as part of Obamacare to answer really important clinical questions. The Institute of Medicine defines one of the main purposes of CER as ‘to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.’ AHRQ funds and executes many such studies (see some great examples here) and they usually do get eventually published somewhere, but, like the Canadian agency CADTH, we feel that the reviews and data it produces is seen by policy makers perhaps, but does not get enough attention amongst end users like us.
The authors conclusions are that the initial test for patients visiting the ED with suspected kidney stones should be an ultrasound. This does not mean they should not have a subsequent CT scan if required, as many patients actually did in the study.
The #UroJC team is discussing the article starting Sunday, and the #NephJC team will discuss it Tuesday Oct 7th at 9 pm EDT.