Article Summary
The LongCovid study by the RIVM, announced on Tuesday, has quite a few methodological problems. This calls for great caution in interpretation. It certainly doesn’t give a good idea of the absolute magnitude of the problem.
Read full article: LongCovid research RIVM with serious problems
Reading time: 6 minutes
Various messages
Yesterday morning, news reports appeared in the media about a LongCovid investigation commissioned by the RIVM. The headlines were quite different.
This was the headline of the NOS article. I also saw a headline like this in NRC.
However, the headline on the RIVM website was:
De Volkskrant wrote a completely different message.
I went looking for the research itself. That was hard to find. There was no link in the message on the RIVM site to that study. But the preprint has been here since 15 June. And if you read that research carefully, you realize that – unfortunately – the figures don’t say much. And that there are hardly any hard conclusions to be drawn, no matter how much you would like to. And that the RIVM is much too firm in its reporting on the site.
A good approach to Long Covid research
Let me first say that there is no doubt in my mind that people suffer from LongCovid. Whereby they are – strongly – hindered in their normal lives. And that sometimes they have no prospect of improvement.
But for this Corona crisis, it is a crucial question of how great the risk is from LongCovid, what the complaints are, how long they last and to what extent this differs with the different mutations.
However, a LongCovid study faces major challenges. The two most important are:
- If you use a questionnaire to determine whether people have or have had symptoms and how bad they are, then that is a subjective opinion. It is then not independently determined by a medical expert. A French study revealed that when reporting complaints, people thought they had had Covid, but were not sure. For example, it is assumed that approximately 5% of the patients of general practitioners suffer from hypochondria. It is very likely that in the past year quite a few people have become more afraid of the Sars-2 virus than was necessary on the basis of their age and health situation. This can also influence the description of complaints in a questionnaire.
- It is very important to have a comparable group in addition to Covid-19 patients, who did not have Covid-19, as a control group. Only then can the results of the Covid-19 patients be properly interpreted.
The problems of the investigation
In this study, therefore, use was made of a questionnaire (ie the subjective approach) with all the associated problems (no matter how well the questionnaire was drawn up). In addition to the Covid-19 patients, they also tried to put together a control group, but that was not without problems. As early as January 2022, this publication set out the research design of this study.
The group of Covid-19 patients were people who had taken a positive PCR test or antigen test between May and December 2021 and were willing to complete an extensive questionnaire about their complaints within 7 days. The control group were people who had also been tested and had a negative result and who subsequently completed the questionnaire. Finally, there was a third group. That was a national sample that was approached. If they had or had received Covid-19 during the study period, they would no longer participate in the study.
Initially, there were more than 9100 participants with a positive Covid-19 test, about 1700 with a negative Covid-19 test and about 3700 from the national sample. This group completed the questionnaire within 7 days.
Persons who contracted Covid-19 (again) during the study period were excluded from the study.
Three months later, all participants were given a questionnaire to describe their current health situation. However, more than 4000 of the 14500 did not complete the questionnaire! Of the group this was 23% and of the national sample the national sample was even 34%!
The big question is, of course, why people no longer participated in the study. It is not improbable that this concerned more people who have no or few complaints than the people who did participate. It is also possible that this effect of the non-response is not the same for those three different groups.
This actually makes it impossible to draw firm numerical conclusions and therefore the figures can only be regarded as indications.
These are only intermediate results. Apparently they want to take a questionnaire three more times (every three months). The number of participants who will then participate will decrease even more. What do the results of the remaining participants mean then?
Finally, I also noticed the level of education of the participants in the tables. There was a tripartite division “High”, “Middle” and “Low”. And the “Low” group was only 3% in the entire study and “High” was 62%. With an average age of around 55, these were very remarkable figures. Now it may be the definition of those three classes, but there are also indications that something else is the case. And then it can significantly affect the numbers!
The questionnaire about health complaints was extensive (“one of the participants told me for a long time” and quite spicy (perhaps also the reason that so many people dropped out in the second instance). This makes it “more difficult” for the less educated to complete the questionnaire. The participants knew that the research was about LongCovid and that may also have meant a clear selection of the participants, in which training also played a role on the one hand and how the respondents were concerned with their health and Covid-19 on the other.
Very special was the fact that the group that participated and had themselves tested, but then apparently did not have Covid-19, had by far the highest education on average: 72% and 2% had a low education! This also indicates that the participants are not very well represented among the Dutch.
If I then list everything, I think that this problem makes it impossible to draw firm conclusions on the basis of this research.
A good design of a LongCovid study is more difficult and will cost much more money. In addition, the complaints (at least in part as a check) must also be determined by an expert. Inquiries should also be made with people who drop out along the way.
I can understand why this research is still being conducted in this way. But the fact that RIVM then publishes the results, such as on the website, without any relativization of the results, I think is “very unfortunate” to say the least.
Just read what it says:
No comparison whatsoever. Unfortunately, this is not the first time that there are conclusions on the RIVM site about research, which were put down much more firmly than the material allows.
A little worse
NRC and NOS made it even worse with the headline of the message. Because the research showed that people who had not had Covid also reported the complaints that the Covid-infected had reported in their research after 3 months. The difference between those groups could then be due to Covid. But the absolute numbers don’t mean much. For two of the complaints, I’ll show the numbers for the three groups to show how this works:
This is the percentage that reported feeling tired immediately in the first week and after three months:
| Suffering from fatigue | in first week | after three months |
| Covid-19 positive | 60% | 31% |
| Covid-19 negative | 41% | 19% |
| National sample | 13% | 6% |
| Chest pressure | in first week | after three months |
| Covid-19 positive | 15% | 6% |
| Covid-19 negative | 5% | 3% |
| National sample | 1% | 3% |
Which part of this should now be classified as LongCovid is very difficult if you compare the three groups and have read the previous chapter.
But in fact, the headline of NOS and NRC was only limited to the Covid-19 positive group that still had complaints. But without looking at the two reference groups, those numbers mean nothing.
Two phenomena stand out in the tables of the 41 possible complaints. And those are the loss of smell and the loss of taste. In the first week, this is reported by more than 40% of the Covid-19 infected. And that only by 1% of the sample. Three months later that is 12% and 1% respectively.
Given the explanation of the problems of the study, you cannot say “12% of all Covid-19 patients still suffer from the loss of taste and smell after three months”. That percentage could be even lower. But it is certainly the case that there are quite a few Covid-19 infected from 2020 and 2021 who suffer from this problem for a long time (or still).
This study states that this loss of smell and taste occurs in 44% of the cases with Delta, but fortunately significantly less with Omikron (in 17% of the cases).
In conclusion
This research is certainly not worthless and can teach us more. But it does have significant limitations, which are not easy to overcome.
That is precisely why RIVM should be very careful with the publication of the results and should certainly warn the media against conclusions that are too strong.
Fortunately, at least De Volkskrant made clear comments about the research.
My prediction is that the next three times the survey is questioned, the research will have more and more problems in the response. Hopefully, RIVM and the media will then be a lot more careful in their reporting.
Help our site with an occasional (small) donation. Then click here.