"Whoa! Even Politico, which is typically critical of the Hungarian government, puts Fidesz ahead of the Tisza Party," Géza Sebestyén, head of MCC's economic policy workshop posted on Facebook on Sunday morning. He attached a screenshot from Politico's Poll of Polls page to his post, which featured a timeline of Hungarian public opinion polls over the past year. According to this image, in Politico's summary, Tisza has been much more popular than Fidesz so far this year, but in recent weeks the trend has reversed, and Fidesz is now ahead by one percentage point.
The post was quickly picked up by pro-government figures and those associated with the ruling parties in Hungary. Tamás Menczer, Fidesz's communications director posted about it in the morning, saying, "Politico doesn't like us. But even Politico says we are in the lead." In the afternoon, he was followed by others, including Máté Kocsis, Fidesz's parliamentary group leader, who wrote that "Politico has measured Fidesz's advantage." The news quickly spread through pro-government media outlets, with everyone from Index to Origo and Ripost to Magyar Hírlap reporting on it.
But did Politico really measure a Fidesz advantage?
They don't measure, they just aggregate
What set of data are they referring to? Politico Poll of Polls is a website that aggregates the results of various opinion polls from a given country and then calculates a weighted average from them. As they write in their own description: "One poll can be misleading, so Politico aggregates data from multiple sources to provide a more accurate picture." It is important to note that Politico does not conduct its own polls or measurements, but simply compares the results of other polls and calculates a trend line from them.
In the Poll of Polls, the results of each poll are marked with a dot, with each dot representing the results for each party. So the vertical axis of the graph shows the popularity of each party in percentage terms, while the horizontal axis represents time. The latest results and their averages are shown on the right-hand side.
But which polls are included? According to the website, polls must meet two requirements in order to be considered. The first is transparency, meaning that when publishing results, it must be indicated how, when, by whom, and on how many people the poll was conducted, as well as who paid for the work. The second requirement is that the survey must be based on a sample controlled by the polling agency, meaning that surveys conducted on Facebook or other external websites do not count.
Although Politico's website mentions that there is a rule of thumb according to which surveys based on a sample of at least 800 people are considered good measurements, they also include smaller samples in their graphs. For the sake of accuracy, however, the surveys are weighted by sample size, meaning that surveys conducted with a bigger number of respondents carry more weight than those conducted with a smaller sample size, and more recent surveys are also given preference over older ones. Opinion polls commissioned or paid for by political parties are not included in the average.
Politico then aggregates the results of the various polls using a statistical model called the Kalman filtering, which is used to create a trend line from the poll results. According to Politico, it is not possible to filter out whether the results of a particular party are over- or underestimated by all institutes, but experience shows that the trend line can more accurately predict the expected outcome of an election than individual opinion polls.
The difficult question is: What about researchers affiliated with political parties?
In Hungary, the issue is complicated by the fact that there are pollsters who are closely aligned with the government and others who are independent of the government and also work for the opposition, and they generally publish significantly different research results. And then, sometimes it turns out which institutes are seriously mistaken:
- For example, prior to the 2019 Budapest mayoral election, Nézőpont was wrong by 25 percentage points and Századvég by 15 percentage points in their predictions of the difference in support for the candidates, with both institutes overestimating István Tarlós, who was running with the support of the ruling parties.
- However, in the final polls before the 2022 elections, it was Publicus that made the biggest mistake: 20 percentage points in the difference between the support for Fidesz and the united opposition, slightly more than Republikon and Závecz did, which were off by 16 percentage points (all three institutes overestimated the opposition and underestimated Fidesz);
- and before the 2024 EP elections, Publicus overestimated the DK-MSZP-Párbeszéd list by 6.5 percent and underestimated Tisza's list by 4.2 percent—although they did predict the results for the other parties and the mayoral election relatively accurately.
It’s not easy for the Poll of Polls to analyze Hungarian opinion polls, as its trend line for Hungary is largely determined by which polls are excluded and which ones are included. Another determining factor is which institute publishes more data and which one publishes less.
The overall picture in Hungary is also affected by the recent increase in the number of pro-government research institutes, which, unlike other researchers, measure the advantage of the ruling parties. While previously only Századvég and Nézőpont published their own data, they have now been joined by Real PR 93 Kft., Magyar Társadalomkutató, Alapjogokért Központ, and the XXI. Század Intézet. (The latter should not be confused with the 21 Kutatóközpont.) So if we average the results of the studies without any critique of the sources, then obviously the trend will be determined by whoever publishes the most studies.
But what happened in this specific case? Although Politico's graph does not indicate which company conducted which survey, when we compare them with surveys published in the recent period, it turns out that the Poll of Polls' Sunday standings were most influenced by the latest measurements from Magyar Társadalomkutató and the Alapjogokért Központ. The former measured a 10-point Fidesz lead, while the latter measured a 7-point lead, and these two measurements were sufficient to swing the average by enough to produce a 1-point Fidesz lead.
Surveys conducted by institutes linked to Fidesz have been deleted
And here comes the twist. Around noon on Monday, Politico deleted the surveys conducted by Magyar Társadalomkutató and the Alapjogokért Központ from its summary, but added the latest survey by Publicus, which shows a 9-point lead for Tisza. Accordingly, the aggregate figures show a significant 7-point lead for Tisza, with Tisza at 46 percent and Fidesz at 39 percent. (We asked Politico why they deleted the polls by Magyar Társadalomkutató and the Alapjogokért Központ, and we will report back if they provide a substantive response.)
With the exclusion of the two surveys from the aggregate and the inclusion of Publicus, which had made significant errors in the previous two elections, the difference between the support for the Tisza Party and Fidesz jumped by 8 percentage points. This illustrates quite well why it is not worth taking the trend line shown by the latest surveys at face value.
But there is one thing that can be stated with certainty: Politico did not "measure" a Fidesz lead over the Tisza Party.