10 February 2021
Before we can use a medicine to treat Covid-19, a well-designed clinical trial must show that it works. Illustration: Lisa Nelson
Size does count, and the Covid-19 pandemic has confirmed it. Thanks to the networks of scientists across the world who are running big clinical trials, we can better decide which medicines should be used.
Two examples of these networks are RECOVERY and SOLIDARITY. They carry out randomised, double-blind, placebo-controlled trials in many cities and countries that tell us how effective various medicines are for treating and potentially preventing Covid-19.
They have provided proof of medicines that work (corticosteroids) and equally importantly, those that don’t (such as hydroxychloroquine, azithromycin, lopinavir/ritonavir).
In this article, we explain why we should control the hysteria around potential ‘wonder drugs’ such as ivermectin to benefit people with Covid-19 and potentially save them from unnecessary harm.
The recognition that many medicines through the last few decades do not work, or even do harm, due to reliance on doctor enthusiasm and poor intuition, biases by researchers and patients based on their personal faith in certain drugs, and huge financial rewards by manufacturers and innovators that allow for data manipulation, has led to the modern if intimidating-sounding ‘double blinded, prospective, placebo-controlled randomised controlled trial (RCT)’.
RCTs are the gold-standard by which we judge the benefits of a medicine. A perfect RCT takes two or more groups of people, and randomly assigns them to receive either the medicine under investigation or a placebo that looks identical but does not contain the active drug. Whether the participant receives the medicine or placebo remains unknown to the researchers and the participants until the end of the trial (‘double-blind’). This stops personal beliefs about what works and what doesn’t, influencing the results, or, for instance, a doctor giving a medicine they believe works to a favoured patient, biasing the study.
Before the trial starts, the investigators agree on primary and secondary endpoints. These are events that determine the success or failure of the medicine. This stops ‘fishing expeditions’, where researchers trawl the data looking for patterns that are often caused by chance.
Examples of primary endpoints may include progression to severe illness, or death. Examples of secondary endpoints might include time to resolution of symptoms, length of hospital stay, length of intensive care stay, or drop in the amount of virus in a swab.
In such trials, size matters. The number of participants needed to show a statistically significant difference in the primary outcome is calculated before the trial starts. If the final numbers recruited are too small, then the study is said to be ‘under-powered’, and the study will be unable to tell us whether or not the result occurred simply by chance. The smaller the study, the less the chance of the evidence being credible.
Similarly, if the groups of participants are not well matched through randomization, for age, sex, pre-existing conditions, smoking, obesity etc., then any effect may not be due to the medicine under investigation, but rather, due to one or more of these differences.
For example, if more deaths are in the placebo group but there are twice the number of poorly controlled diabetics in that group, then the greater death rate may be due to the diabetes, rather than due to the beneficial role of the medicine in causing a reduction in deaths in the other group. The imbalanced nature of the two groups has created ‘bias’. Randomisation decreases this chance.
When we analyse studies, we study the level of bias there is. Bias looks for anything other than the medicine to affect the endpoint. Studies with high levels of bias undermine whether what we are seeing is actually due to the medicine. Scientists, clinicians and public health specialists will rarely accept the results, especially as all medicines have the ability to cause harm, and that should be avoided at all costs. Sadly, just because the trial is labelled as an RCT, doesn’t mean that it is necessarily good.
Any study of a new medicine that doesn’t randomise the groups is particularly problematic as the risk of bias is extremely high. These studies are generally called ‘observational’ studies. Let’s look at an example: Say I wanted to test if medicine A is effective in treating Covid-19. Instead of performing a double-blind placebo-controlled RCT, I decide to treat 100 Covid-19 patients in my hospital with medicine A plus the standard of care and give a different group of 200 patients, just the standard of care. There is no randomization, and death in the hospital is my primary endpoint.
Let’s say a total of 25% of the patients treated with medicine A die, whereas 95% of patients who don’t receive medicine A die. Sounds like medicine A is the bees’ knees!
Problem is, despite the seemingly magnificent results, the study may be very biased. Say, I as the doctor decided that because patients who were likely to benefit from medicine A would be those that were younger and with few pre-existing conditions. I decide to give medicine A to these patients. The result would be that patients not given medicine A, the control group, would be older and have more risk factors for dying in the first place. The result is a foregone conclusion – we cannot tell whether medicine A is a wonder drug or the participants who were in the standard of care group died because they had a greater chance of dying in the first place. This bias limits our ability to judge a medicine’s benefit and is one of the reasons why RCTs rather than observational studies are prized more.
In a rapidly developing pandemic with a new pathogen, treatment options are usually extremely limited. As new medicines take time to develop, the first months are usually characterised by looking to medicines that are already known to treat other infections or diseases, that have been shown to have activity in the laboratory against the new pathogen; in this case, the SARS-CoV-2 coronavirus causing Covid-19.
The antimalarial hydroxychloroquine, the antibiotic azithromycin, and the antiretroviral lopinavir/ritonavir are all examples of such medicines that showed some activity against SARS-CoV-2 in the laboratory. All produced hopeful signs of benefit in some patients and poorly conducted observational trials. This was probably aided by something called ‘’publication’’ bias – researchers often only publish their results when something works, and journals prefer publishing them. ‘Negative’ results are often seen as boring.
But all were found to be of no benefit when subjected to properly conducted double-blind placebo controlled RCTs undertaken in the UK (RECOVERY trial) or in multiple countries including South Africa (SOLIDARITY trial).
Moreover, the fact that hydroxychloroquine and azithromycin both can interfere with the electrical conduction pathway of the heart and lead to rhythm disturbances, caused considerable concern. This highlights another important factor that must always be considered when deciding to give a person any medicine; that all medicines may have significant adverse effects, whether it’s a simple pain killer or the most toxic form of chemotherapy.
Ivermectin now joins the lengthening list of medicines touted as ‘wonder drugs’ for the treatment or prevention of Covid-19 before good evidence is available. The debate whether it should be provided to prevent or treat Covid-19 has led to much fervour and passion.
Ivermectin is generally a veterinary antimicrobial that is used in humans to treat specific worm infections – onchocerciasis (river blindness), strongyloidiasis and filariases – as well as scabies. Most indications require a once-off dose of ivermectin. Much of what is being proposed for ivermectin, for both prevention and treatment, is much longer courses.
Although ivermectin is generally well tolerated at these doses, 1-10% of people will develop either cardiovascular, central nervous system, gastrointestinal, blood or liver adverse effects. In post-marketing surveillance reports of adverse events in people taking ivermectin since the drug was registered and licensed, the following is reported in less than 1% of people taking it:
Abdominal distention, abdominal pain, abnormal gait, abnormal sensation in eyes, anaemia, anorexia, anterior uveitis, ataxia, back pain, brain disease (rare; associated with loiasis), chest discomfort, chorioretinitis, coma, confusion, conjunctival haemorrhage (associated with onchocerciasis), conjunctivitis, constipation, drowsiness, dyspnoea, exacerbation of asthma, eye redness, eyelid oedema, fatigue, faecal incontinence, headache, hepatitis, hypotension, increased serum bilirubin, keratitis, lethargy, leukopenia, mental status changes, myalgia, neck pain, posterior uveitis, seizure, skin rash, Stevens-Johnson syndrome, stupor, temporary vision loss, toxic epidermal necrolysis, tremor, urinary incontinence, urticaria, vertigo, vomiting, weakness.
A not dissimilar list is likely to be true for most medicines, but again, the wide variety and potential severity of these adverse events are here to illustrate that any use of a medicine must weigh up the potential benefits and the harms. The national regulator, the South African Health Products Regulatory Agency (SAHPRA), has the responsibility of ensuring that medicines are only consumed by humans that show statistically proven benefit, and where the balance of benefit to risk of adverse effects does not swing towards harm. Similarly, the South African Medical Association, other national and regional professional bodies, and each and every healthcare professional that prescribes a medicine, has an equal duty of care.
The evidence for ivermectin’s benefit in Covid-19 falls short of that required to make a proper assessment. Like hydroxychloroquine, azithromycin, and lopinavir/ritonavir, it has some activity against the virus in the laboratory. It can prevent the action of certain proteins used by the virus to suppress our immune response against it. It might act to reduce binding of the virus to our cell receptors, and it has been shown to inhibit the replication of the virus when it is grown in the lab.
But it has been estimated that for these laboratory effects to be seen in humans, we would need a dose of ivermectin of up to 100x the maximum dose that is usually used to treat worm infections or scabies.
The National Essential Medicines List Therapeutic Guidelines Sub-Committee on Covid-19 Management, which reports to the National Department of Health, recently updated its reviews on ivermectin both for prevention and treatment of Covid-19. These reviews require painstaking analysis of all clinical studies including those published in ‘pre-print’, that is, before having been peer reviewed. This is important, as peer review often finds problems with studies that can affect the interpretation of results.
For prevention against developing Covid-19, the committee found that only two RCTs out of 171 publications could provide any analysable information on this question. Both were from Egypt; one was published but not peer-reviewed, and the other was an ongoing study that had not even been published yet.
This is extremely unusual practice in the medical research world. The trials had significant methodological problems that raised great concern about their reliability. These problems variously included a small size, the lack of crucial information about the trial randomisation process, missing information about time intervals, and even questionable definitions of the SARS-CoV-2 infection itself.
Sadly, the level of evidence for use of ivermectin as a treatment for Covid-19 is equally poor. The vast majority of studies that were included in the Committee’s final analysis had not undergone peer-review and contained very small numbers of participants. The studies were very difficult to judge together or compare as the dosing strategies and primary outcomes were different.
Many of the trials did not investigate ivermectin against a placebo, but rather tested whether a combination of ivermectin and another medicine which could have an effect on the virus. This makes it almost impossible to tell whether any effect that is seen is due to ivermectin, the companion medicine, or a combination of the two.
Many of the studies did not adequately report on adverse events. Lastly, there was evidence of possible publication bias, in so much as some studies were only added to trial registries after their completion, again a practice that is of concern in the modern research era. This implies that these trials might only have been advertised (posted) because they reported positive results.
Where does this leave us with regard to ivermectin? Bottom line – without credible evidence of benefit for the use of ivermectin as a prevention or treatment option for Covid-19, ivermectin rollout is not yet ready for primetime.
People argue that it “does no harm”, so why not just give it?
Well, the truth is that we just don’t know the harm that ivermectin could do in Covid-19 because: a) it’s not adequately tested by rigorously performed RCTs that give good data on adverse event rates, and b) because some of the studies have used doses/durations that exceed the normal dose/duration used to treat the conditions that we know there is benefit for, such as scabies and strongyloidiasis.
These dose differences might cause harm. A drug that is fairly safe when used in medically stable and otherwise healthy patients can end up having serious side-effects when used in critically ill patients, or in combination with other medications.
This has been precisely the experience so far in this pandemic with drugs such as hydroxychloroquine and azithromycin. One recent review concluded that when the two were given together to Covid-19 patients, as many South African doctors did without evidence in the early days of the pandemic, the mortality rate was almost doubled.
As clinicians, we fully appreciate the urgency of getting drugs that work against Covid-19 to patients, and we hope ivermectin is shown to work in proper studies. But we think that many of our colleagues are misguided, with the best of intentions, and don’t appreciate the potential danger to patients of using untested doses in the face of under-documented safety. We should wait for the results of larger, better conducted studies of ivermectin. Only once the evidence is in, and the question of effectiveness versus harm have been answered, should a decision be made by the regulator and healthcare professionals to prescribe ivermectin.