Dr Karolis Ažukaitis, Vice-Dean for Research and Innovation of VU Faculty of Medicine, a paediatric nephrologist. Photo: VU MF
What causes large amounts of junk science? Is it true that the pressure on scientists to be productive is driving the increase in reduced-value science? Why do children’s blood vessels lose elasticity, and what are the difficulties in correctly measuring vascular stiffness in children? We interviewed Dr Karolis Ažukaitis, Vice-Dean for Research and Innovation of VU Faculty of Medicine, a paediatric nephrologist, to find out more about this.
Having been part of many different research groups, Dr Ažukaitis has learnt about the principles of scientific methodology and has seen how many mistakes can occur in the research process that then lead to unreliable results. He began to understand the problem of junk science when he was in Australia at the University of Sydney, where the Cochrane Kidney and Transplant Group reviews research, tries to synthesise the results, and thereby, assesses the quality of the research.
Is medical research always meaningful?
Dr Ažukaitis pointed out at the beginning of our conversation that medical science is primarily intended for society. If it does not directly or indirectly generate any benefits for that society, then it is meaningless. “For example, if I state that treatment works for a particular disease, then patients will benefit directly – doctors will use the evidence I provide and perhaps this will improve patient outcomes. By contrast, indirect use of science means that I might do a study that generates a hypothesis. Other scientists will follow up the study, come up with conclusions, and eventually, patients will be the beneficiaries. But it is important to be absolutely sure of what we have discovered because this will determine how patients are treated, how they are cared for, and what prevention programmes we will apply to them.”
According to Ažukaitis, it is also important that research is meaningful and that patients have at least an equal say in what science should investigate. For example, a change in a laboratory parameter during treatment using a certain drug may indeed be important to a doctor, but if it does not prolong the patient’s life, improve the quality of their life or reduce the burden of symptoms, the doctor is not concerned about the change. Often it is even inconvenient because every treatment involves a burden for the patient – they have to take medicines, they sometimes suffer from side effects, and of course, they have to find time for doctor’s appointments and tests.
“It is important that the research is reliable and meaningful for the patient. Otherwise, research that investigates what patients do not care about or what should not be investigated contributes by generating junk science. The science that tries to answer questions that have already been answered, or provides answers that nobody needs, is unnecessary. Sometimes, the problem is that, although the research is meaningful and has been done correctly, the description of the results is presented in such a way that nobody can use it. For example, when you start to look at an article you have read, you realise that you cannot even tell exactly how the patients were treated, whether the treatment was continued, and if so, in what way. If all this is not described, it is of no use to the practitioner,” explained the paediatric nephrologist.
Finally, there are many scientific studies, the results of which, are never published. According to Dr Ažukaitis, research that other scientists or doctors are not aware of is worthless because it has no impact: “The issue of the non-publication of science is very big. Research results that are not liked by the scientists themselves, or by pharmaceutical companies, are simply not published. There was a study looking at the clinical trials of antidepressants. Research studies are recorded in a clinical trials registry: it turned out that of the 74 trials on the effects of antidepressants that were recorded, the results of only a third were ever published. These were mostly studies that showed the positive effects of antidepressants. The question is, what happened to the rest of the studies?” According to Ažukaitis, this shows the information vacuum we are facing, the distorted picture we are seeing due to bias when choosing what to publish and what not to.
Dr Karolis Ažukaitis, Vice-Dean for Research and Innovation of VU Faculty of Medicine, a paediatric nephrologist. Photo: VU MF
What is the extent of the problem of scientific junk?
“The term 'junk science' became widespread after a number of scientific methodologists reviewed a large number of studies in The Lancet in 2009 and tried to quantify where mistakes might have been made, which studies had been conducted in areas that no longer needed to be investigated – because the questions they were asking had already been answered, or the results were not of benefit to patients. Their findings quoted a dramatic figure, suggesting that 85% of science could be classed as junk science that is unusable, incorrectly done, not understood or not even needed,” noted Dr Ažukaitis.
He went on to explain, that although the figure sounds excessive, sometimes it is the only way to draw attention to a problem: “Imagine that over a million publications are generated in the world every year. Inevitably, a large proportion of these become junk science, which means that resources are wasted instead of being allocated effectively. What causes this? I think one of the main reasons that one needs to appreciate, is that a successful career in research is based on bibliometric data. Allegedly, the more publications, and the more citations I produce, the better the scientist I am. This is a particularly acute problem in China, where in hospitals, if you want to be in a management position, you have to have at least some scientific publications.”
Ažukaitis explained that the demand for scientific articles is eagerly exploited by those who are interested in making a financial profit: “There are so-called paper mills, companies that sell authorship of ready-made scientific articles. The peer-review process here is often also based on a scheme where there are reviewers who guarantee positive reviews. A review of the data on articles published in a series of journals, and then retracted, (potentially as many as 100), reveals that they were produced by these profit-driven companies.”
Another problem is that it is in the interest of universities to have as many publications as possible, since this is important for institutional rankings. As the saying goes, when there is demand, there is supply. “There are publishing houses that are starting to publish what are called predatory journals. They either have no peer-review process in principle or a very dismissive peer-review process. They have quite high publication rates, and once a paper is submitted, it is very likely to be published, regardless of its quality. Publishers have an incentive to publish and institutions have an incentive to pay for it – publications determine their rankings, and rankings determine further funding. A closed circle is born – we are living in a distorted world where the outcome of science is no longer important, it is simply a process,” explains Dr Ažukaitis.
Cardiovascular damage: an area of interest for the paediatric nephrologist
As a paediatric nephrologist, Dr Ažukaitis pays particular attention to the quality of his research: “This is a narrow field and at the same time a very broad one. My main area of interest is early cardiovascular damage. My work in this field started when I started working on a study at the University of Heidelberg. It looked at children with chronic kidney disease and their early cardiovascular damage. Although these children, when young, do not have serious cardiovascular problems or serious diseases, cardiovascular disease becomes the dominant cause of death when they reach the age of 30 to 40 years. It seems to me that the whole of paediatrics is now moving towards the fact that we are not only concerned with treating the disease, but also with ensuring that the patient lives as long and as healthy a life as possible. Or in other words, to avoid premature biological ageing and to prevent the processes that start in childhood and will lead to illnesses 20-30 years later.”
The study was carried out in Finland on children from childhood and into adulthood. The results showed that elevated blood pressure in childhood has a direct correlation with cardiac abnormalities at the age of 40. “While childhood hypertension was once a rare event, mostly associated with comorbidities – in the kidney, and endocrinological – we now have an epidemic of obesity and hypertension not only in the adult population, but also in children. So, my research has turned to cardiovascular damage in early childhood, both in children with arterial hypertension and in children with chronic kidney disease”, he said.
The researcher noted that he does not have the luxury of following his patients until they reach age 40, but he is interested in understanding as much as possible about the patient in childhood: “You can observe the child to see how effective the treatment is, if there are small structural changes in the heart – if the ventricle is thickening, if the geometry of the heart is changing, or if there are subtle disturbances in the relaxation of the heart that you can measure. It is also possible to assess the health of the blood vessels, and this has become my major interest. In particular, I can assess whether a child’s atherosclerosis is progressing faster than it should according to normal physiological development, by assessing whether the blood vessels are stiffer than they should be for their age”.
Dr Karolis Ažukaitis, Vice-Dean for Research and Innovation of VU Faculty of Medicine, a paediatric nephrologist. Photo: VU MF
How is arterial stiffness measured?
The question, “How to measure arterial stiffness in children?” is almost a fundamental problem, because it is quite difficult to achieve. According to Dr Ažukaitis, he relies on methods that are used in the adult population, but the changes that occur in children’s arteries are particularly subtle. “This means that if I want to measure them, I have to pay attention to whether I am really getting a number that indicates the stiffness of the arteries, and what is the tolerance of that measuring device. If the error rate in the data is high, then I will no longer be able to detect a subtle change that differs from their peers.”
There are many methodological problems with these studies. First of all, there are many devices available to measure arterial stiffness, but not all of them are suitable for children. “A few years ago, we did a review of these devices, and their validity in the paediatric population, and we saw how differently people use them – how differently they interpret the data. These reasons may be responsible for the fact that many of the methods used in the research misjudge arterial stiffness and sometimes the results do not reflect it accurately at all,” he noted.
According to the researcher, his study with colleagues from the European Childhood Hypertension Network (HyperChildNET) found that arterial stiffness in children depends almost entirely on blood pressure. This prompted the question of whether their measurements of arterial stiffness were indirectly measuring blood pressure. “This is a very difficult question to answer. We know what tests would need to be done to answer it, but they would be unethical, with too much risk of intervention. So, at the moment, we are still measuring pulse wave velocity – how fast the pulse wave travels through the large blood vessels. This can be measured very simply by using a special device to record when the pulse occurs in the neck and when it occurs, for example, in the leg. The distance between the two is then measured, which roughly reflects the course of the aorta, and the faster the pulse travels from the neck to the leg, the stiffer the aorta is.”
In concluding the interview, Dr Ažukaitis pointed out that as children grow up, their height changes every year. It is therefore important to compare the data with the normal values for children of the same age, height and gender. Another challenge is that each machine that measures arterial stiffness, measures a slightly different amount, so each machine needs the same data. Moreover, there can be large errors. “Imagine the number of risks in this process if, instead of measuring arterial stiffness, I were to measure something we sometimes call ‘milk fat’. We do these tests on children where body changes are extremely subtle, the child is growing up, everything is changing all the time, so minimal mistakes, especially if they are made at the first, second, or third stage of the test, will lead to your test results being completely meaningless. With poor quality baseline data, we will never get a reliable result.”