A report appearing in the December 2017 issue of Value in Health, (a journal of the International Society for Pharmacoeconomics and Outcomes Research) concluded that assessments of high-risk medical devices in the European Union were of low quality – based on evidence from studies that were methodologically inadequate.

The authors of the report reviewed 1,376 health technology assessment reports (“HTAs”) of high-risk medical devices (“MDs”) that were filed between 2010 and 2015. Of the 93 that were eligible for analysis, the researchers rated each of these studies on the following criteria:

Evidence base

Was the evidence in the HTA based on submissions by the manufacturer, data identified by an independent source, or both?

Type of evidence

Was the evidence in the HTA based on “direct” (e.g. head-to-head trials) or “indirect” evidence?
Direct evidence from well-conducted randomized controlled trials were seen as providing the most valid estimates regarding the effectiveness of competing health care interventions.

Level of evidence
Further considerations of scientific evidence

What were the total number of studies considered in each HTA report?

The report ultimately found that while almost all assessments considered “based their evaluation on direct evidence from independent systematic literature searches, good-quality data were scarce.” The report further stated that “our findings illustrate that even if systematic reviews and RCTs [randomized controlled trials] were available for assessment, most of these studies showed an unclear or high RoB [risk of bias] according to the specific tools used in their reports.”

The report ultimately concluded that:

In the EU countries, MDs are essentially regulated in the same way they have been since the 1990s. This means that high-risk MDs can enter the market and be used in humans without the requirement of evidence from robust clinical studies. As a consequence, scientific evidence prior to market approval of high-risk MDs is often based only on evidence from studies that were methodologically inadequate.

In summary, the report proposed that “additional changes are necessary, specifically with regard to the marketing authorization process of MDs, with stricter quality requirements based on methodologically robust trials, possibly in combination with other evidence sources.”

Written by: Angela Dremann