How to evaluate drug information (Proceedings)

One can usually find many sources of information about drugs: FDA website, drug company websites and technical reports, VIN, journals, trade magazines, and so on. The important skill required of veterinarians is to assess that information to determine its usefulness in your daily practice. Below are some principles of evaluating drug information, with the goal of improving treatment and the practice of medicine.

Principles of Evidence-Based Medicine

Evidence-based medicine is the approach to making medical and therapeutic decisions that is "the integration of best research evidence with clinical expertise and owner/manager values." This approach differs from the other prominent methods of decision-making, the expert-based or apprentice-based approach, which predicates clinical decisions on what has been done before and on what the "experts" say, whether the experts are the clinicians who taught you in veterinary school or the authors of the review articles you read or the presenters at continuing education seminars. This approach to making clinical decisions emphasizes just-in-time learning, rather than just-in-case learning, during which you search for assistance with decision-making on the basis of the cases you are presented with.

Evidence-based medicine suggests the explicit use of the following steps in decision-making about drugs:

      1. Ask a clinically relevant and answerable question about the patient or patients. One commonly used approach is PICO, which refers to asking a question with the elements of patient, intervention, comparison, and outcome. An example of a PICO question encountered in bovine practice might be:

      2. What antimicrobial regimen (I) is most likely (C) to result in minimal weight loss (O) in stocker calves (P) on grass in Kentucky?

      3. Locate the best evidence to answer the question.

      4. Hierarchy of evidence levels outlined below

      5. Critically appraise the evidence for validity, impact and applicability.

      6. SIntegrate the appraisal with clinical expertise and the patient's and client's unique biology, values, and circumstances.

The critical appraisal step above (Step 3) often includes the assignment of a "level of evidence" to the evidence located in Step 2. There are as many schemes for assigning levels of evidence as there are advocates of evidence-based medicine, but one scheme designed to address the types of data for veterinary species was originated by the veterinary expert panels at the U.S. Pharmacopeia, when they were developing drug information monographs. Their scheme includes assigning a level of evidence (1-6) and a quality grade (A-E). The level of evidence refers to the type of data, as follows: \

      1. Species-specific evidence from at least one large randomized and controlled trial (RCT) or multiple small RCTs

      2. Species-specific evidence from a small RCT, disease models, large case studies, pharmacokinetic studies using surrogate endpoints, or evidence from well-designed trials in a different species that is considered appropriate for comparison

      3. Dramatic results from either well-designed, species-specific trials without controls or small case studies

      4. Pharmacokinetic studies without surrogate endpoints

      5. In vitro studies

      6. Opinions of respected authorities on the basis of clinical experience or reports of expert committees

The quality grades include: A (Good evidence to support a recommendation for use), B (Moderate evidence to support a recommendation for use), C (Insufficient evidence to support a recommendation for use), D (Moderate evidence to support a recommendation against use), and E (Good evidence to support a recommendation against use).

These levels and grades are assigned to each piece of evidence, e.g., [B-3]. The purpose of assigning a level of evidence to a particular set of data is to weigh evidence with a higher level more heavily. In addition, consideration of levels of evidence encourages us to search for the best evidence, rather than stopping at the first evidence that we find.

Sources of Drug Information (Step 2)

Major sources of drug information in veterinary medicine include textbooks (including periodicals like Veterinary Clinics of North America), professional magazines, peer-reviewed scientific journals, Freedom of Information Summaries from animal health companies published by the FDA, technical reports published by animal health companies, approved drug labels and inserts, and presentations at continuing education seminars.

Once Step 1 and Step 2 have been completed, Step 3 or the appraisal process can be performed on the evidence available. Below are suggestions for evaluating published papers, drug labels, and technical reports.

Evaluating Published Papers (Step 3)

Reading papers for content is an important first step in reviewing evidence. The next step should be to read the paper with a reflective and critical eye. Some questions to ask are outlined below. (Elements of these questions are partially adapted from "How to Read a Scientific Research Paper--a four-step guide for students," by Ann McNeal, School of Natural Science, Hampshire College, Amherst MA, as well as from the REFLECT Statement for Reporting Guidelines for Randomized Controlled Trials for Livestock and Food Safety, http://www.reflect-statement.org/home.html.)

Reflection and Criticism

What was the rationale for the study? Does the background for the study fit with your understanding of the field?

Was randomization performed and how?

Were the measurements appropriate for the questions the researcher was approaching? [Often, researchers use surrogate measures because they cannot measure something directly.]

Were the measures in this research clearly related to the variables in which the researchers (or you) were interested?

Do the animals in the study fairly represent the populations under study?

What is the one major finding?

Did you see patterns or trends in the data that the author did not mention?

Were there problems that were not addressed?

Do you agree with the conclusions drawn from the data?

Are these conclusions over-generalized or appropriately careful?

What other factors could have influenced, or accounted for, the results?

What further experiments would you think of, to continue the research or to answer remaining questions?

Evaluating Drug Labels (Step 3)

Similar to evaluating published papers, critically evaluating drug labels can provide useful information for making therapeutic decisions. The steps are similar, but the parts of a label are different than the parts of a published paper.

Reflection and Criticism

Is there a human health risk associated with this drug?

What species is this drug labeled for? And for what purposes?

Is this drug for a food animal? What is the withdrawal time?

Does the pharmacokinetic data fit with what you know about other similar drugs? Does it make sense in regards to the indication for its use?

Do the effectiveness studies reported relate to your animal population? Do they demonstrate adequate efficacy for the purpose for which you are selecting the drug?

What reactions have been reported?

Is there adequate information on this label for you to prudently use this drug? What other information would you like about this drug?

Evaluating Technical Reports (Step 3)

The initial steps in evaluating technical reports should include the same steps as outlined above for evaluating published papers Some questions to ask include:

Reflection and Criticism

Does this material cover a subject which interests me and is clinically important in my practice?

Has this material been published in independent peer reviewed journals?

Does the material include high-level evidence such as systematic reviews, meta-analyses, or double-blind randomised controlled trials against the drug's closest competitor given at optimal dosage?

Have the trials or reviews addressed a clearly focused, important and answerable clinical question which reflects a problem of relevance to patients? Do they provide evidence on safety, tolerability, efficacy and price?

Has each trial or meta-analysis defined the condition to be treated, the patients to be included, the interventions to be compared and the outcomes to be examined?

If a surrogate outcome measure has been used, what is the evidence that it is reliable, reproducible, sensitive, specific, a true predictor of disease, and rapidly reflects the response to therapy?

Are the results expressed in terms of numbers needed to treat, and are they clinically as well as statistically significant?

What biases do you expect there to be in this report?

How does this report differ from a hypothetical peer-reviewed journal article reporting on the same results?

Conclusion

Evaluating drug information from any source using the above principles of evidence-based medicine and methods of critiquing drug sources should result in better use of available information. It will also hopefully result in better treatment decisions and better medicine.

References

Cockcroft, P and Holmes, M, Handbook of Evidence-Based Veterinary Medicine, Blackwell Publishing, 2003.