An article appears reporting the positive effects of a new agent in a trial of 50 patients with septic shock.
(c) What criteria will you use to assess the validity of this article to your ICU?
The criteria for assessment of such an article include:
• Is the trials design valid and powered to achieve a result? It seems doubtful in this case but a large effect in a specific group may be detected.
• Was the hypothesis based on valid evidence?
• Were all the entered patients accounted for?
• Were the groups equivalent after randomisation?
• Was there proper blinding of study personnel?
• Apart from the experimental intervention were the groups treated equivalently?
• Was the statistical analysis appropriate?
• How large was the treatment affect?
• Can the results be applied to my patients?
Though not owrd-for-word identical, this question closely resembles Question 8 from the second paper of 2012, as well as Question 8 from the first paper of 2004. It discusses the assessment of the validity of a randomised controlled trial, which is discussed in greater detail elsewhere.
The answer is reproduced below, to simplify revision and damage SEO:
Is the premise sound?
- Is the primary hypothesis biologically plausible?
- Is the research ethical?
- If the results are valid, are there any disastrous logistical ethical or financial emplications to a change in practice?
Is the methodology of high quality?
- Were the inclusion/exclusion criteria appropriate?
- Was the assignment of patients to treatments randomised? If yes, then was it truly random?
- Were the study groups homogenous?
- Were the groups treated equally?
- Are there any missing patients? Is every enrolled patient accounted for?
- Was follow-up complete? Is the drop-out rate explained? Do we know what happened to the dropouts?
Is the reporting of an appropriate quality?
- Methods describtion should be complete: the trial should be reproduceable
- Do the results have confidence intervals?
- Results should present relative and absolute effect sizes
- Is a CONSORT-style flow diagram of patient selection available?
- Discussion should contain limitation, bias and imprecision
- Funding sources and the full trial protocol should be disclosed
Are the results of the study valid?
- Was there blinding? Was blinding even possible? Was it double-blind? If not, at least were the data interpreters and statisticians blinded?
- Was there allocation concealment?
- Was there intention-to-treat analysis?
- If there were sub-groups, were they identified a priori?
What were the results?
- How large was the treatment effect?
- How precisely was the effect estimated? (i.e. what was the 95% confidence interval)
Is this study helpful for me?
- Is this applicable to my patient? i.e. would my patient have been enrolled in this study?
- Does the population studied correspond with the population to which my patient belongs?
- Were all the clinically meaningful outcomes considered?
- Does the benefit outweigh the cost and risk?