Safety and effectiveness of implants for cerebral aneurysms substantiated by very poor evidence

In recent years, many new devices have come on the market to treat an aneurysm in the brain: a bulge of a blood vessel. Neurosurgeon Victor Volovici of Erasmus MC noticed that many of these devices are labeled “safe and effective,” while solid evidence for these claims is lacking.

No control group

With an international team, Volovici searched through 1,356 scientific studies that reported in JAMA Network Open on the safety and effectiveness of new treatment methods for brain aneurysm in more than 400,000 patients. Volovici was shocked by the results. ‘The quality of the studies is poor. In 80 percent of the studies, no control group is used to provide a benchmark for the results of the new implant. The claim ‘safe and effective’ is therefore improperly substantiated from a scientific standpoint. There is often no difference in complications or even mortality between studies that conclude a device is safe and those that say it is not. Researchers seem to be cutting corners when it comes to good science.’

Neurosurgeon Victor Volovici. Photographer: Esther Morren.

The problems in the evidence base occur mainly with the newer devices for endovascular treatment. These devices came on the market as a successor to two treatments for which the evidence is solid: brain surgery during which the skull is opened, called clipping. Or filling the aneurysm with platinum spirals via a blood vessel in the groin, called coiling. The goal of both treatments is to seal off the aneurysm.

Evidence is lacking

In recent years, the industry has developed new devices for treatment, such as so-called WEB devices and flow-diverting stents. ‘These implants are presented in practice as innovative treatments, which are even better than coiling and clipping. But evidence for this that stems from high-quality studies with a control group is lacking,’ Volovici said. The same, he says, applies to new surgical techniques. ‘I am at least as critical of those as I am of the endovascular studies.’

‘There is not enough evidence that the new implants are better than coiling or clipping, but there is also no evidence that they are worse.’

Volovici emphasizes that doctors should carefully make the choice of treatment together with other specialists during a multidisciplinary consultation. Then this choice should be discussed with the patient or the patient’s family. ‘If surgery and endovascular treatment are both possible, several factors play a role in making the choice. We consider, for example, the patient’s age and preference and the shape and location of the aneurysm.’

Patients treated with a new device in the Netherlands need not worry, says Volovici. ‘All patients are monitored regularly, and we have an extensive system of quality control in Europe before a new medical device enters the market. There is not enough evidence that the new implants are better than coiling or clipping, but there is also no evidence that they are worse.’

Tighten policies

Volovici is critical towards authors, peer reviewers and editors of scientific journals who call a new treatment safe and effective when no robust study has been done. ‘Conducting and publishing studies of questionable quality must stop. Doctors, patient associations and the industry should say: from now on, we’ll only conduct studies with enough patients from multiple hospitals and with a control group representing the current standard of care.’

Volovici expects regulatory agencies, such as the FDA in the United States, to tighten medical device approval policies. He is also hopeful that the Dutch nationwide research consortium CONTRAST-2, in which nine centers in the Netherlands are investigating the safety and effectiveness of various stroke treatment methods, will change clinical practice for the better.