A Look Back At The Past, Present, And Future Of UK Elections
On Past UK General Election Results, Polling, Trends, Etc
Given how heavily focused this newsletter will be on a 2020 U.S presidential election post-mortem and the history of U.S presidential elections overall in the near future, plus the fact that another UK general election will probably not take place for another three to four years at best, I decided I would be better off writing a little about the latter while awaiting to get going on the many presidential psephology posts.
UK general elections are of course much different than U.S presidential races, particularly in that its the UK House of Commons in parliament that calls most of the shots, not an executive branch elected official - lead by a Prime Minister that leads his or her own political party's fortunes as well as the country. UK elections as of today (because the rules have changed throughout time) are called whenever a Prime Minister desires, when a vote of no confidence is passed by the opposition on the government, or when a five year deadline for a term in power comes up. Polls close all at once throughout the country on election night (10 PM local time), and a much more accurate than a U.S version of an exit poll is released projecting the final seat count. After that a formal vote declaration is completed one by one in each constituency as the hours pass by into the night and the final actual results come to be known. The leader of the winning party then gets to visit the monarch and declare they have the numbers to form a government and schedule a speech from that said monarch to announce the government's plans for governing. Of course sometimes no one gets a majority of seats, and in that case the UK would end up with a minority or a coalition government heading the commons.
The House of Commons has had elections going back to the days before the United Kingdom even existed, back when it was Great Britain - and even before then when it was merely England. Of course as time has passed, the power of parliament has grown as the monarchy's has almost vanished. However election results going back to the earliest parliaments remain hard to find. Thankfully, since the UK came into being in 1801, each election has had enough recorded results for us to learn a little from - and we have very detailed information for modern era races. Like with the U.S, UK electoral results can be shaped by incumbent popularity, political environments, economic growth, and socio-cultural revolutions. Incumbency of the Prime Minister can also play a major factor, just as with U.S Presidents it is usually a non-elected Prime Minister who loses the majority for his or her own party. Of course sitting Prime Ministers who have won elections before losing their majorities do exist, but they're outliers.
As I said, UK psephology in terms of the next general election at least is sort of in wait and see mode at the moment as the last UK general election was held just last year. Until then, the following chart below was created by me to look at the various size of majorities (or minority governments) that have come into being since the UK began electing its members of parliament. In the last two hundred and twenty odd years, we've seen Tory, Labour, Liberal, Whig, and even national coalition lead governments win the keys to 10 Downing Street - the residence of the Prime Minister. This chart below plays out as a sort of timeline from election to election in which you can compare several sizes of victories to one another.
Now in terms of forecasting future UK general elections for the House of Commons, there are as always various ways one might try their hand at such. Like with the U.S presidential elections, you can use everything from complex fundamentals-based political science models to simple poll aggregating. I have personally found that aggregating national and regional polls, and then applying a swing to them in seats gets you a good picture of the likely results. However given that the average error in UK general election polling is a tad higher than U.S election polling (Though presidential polling in the U.S the last two cycles hasn't been a good example of that), I find that keeping in mind a five point margin of error at the most for each party gives you perspective as well to the range of plausible results. The table below highlights the history of polling average estimates versus final results in past UK general elections.
As you can see though polling error in regard to the final actual margins can at times be dramatic in UK general elections, the national polling leads can still be a good barometer of the likely popular vote winner and the party that will lead the next government. I'll show you how I myself used polling to forecast a good picture of the would be 2019 results.
That all said, I personally have lost a lot of the forecasting data I had collected myself for last year's race that I had posted on my social media, and thus decided to make a simple replication of what I did to forecast that race. Because of this, the example I will show you here doesn't emphasize the range of possible results based on probable poll error - and the final predicted seat margins I show here may even be a tad off from what I had then as I may have missed one or two other data points from what I got when I did this last. Basically, this example of forecasting last year's UK general election may not be a 100% copy of the way I showed it back then - but I can assure you the overall picture seems to be the same one I found last December.
First, how I get my polling aggregate. Like with the U.S presidential forecasting I try, I take a chance on actually trusting the public polling and public MRP estimates, and average the last three dates of final polling field dates or MRP estimates to each survey available for each race. Whenever data is missing, I use historical results to capture the likely trajectory of a race - but for the UK this doesn't really become an issue like it does with heavily partisan and less polled states in the U.S. For the UK, I average polling numbers for the nation as a whole, the region of Scotland, and the region of Wales to get the overall seat swing estimates. With all due respect to Northern Ireland, their results don't get any forecasting from me since their parties won't affect government control directly that much really thanks to the Good Friday agreement. I added an image from an excel polling average below I created real quick to show you how this polling average gives me a final number to get a swing from. This is the averaging I got for national public UK polling so you'll note regional nationalist parties aren't featured in the table below, but I do average their polling in Scotland and Wales aggregates (Not shown here).
The strength to such a method is that it catches late swings as the election comes to an end, catching potentially game changing momentum late that other more complex methods would take time to catch themselves. This method did wonders for me in 2017 when it caught a weakening Tory government as it headed towards an eventual minority government. The weakness however is that such a method leaves this sort of forecasting susceptible to noise in the middle of campaigns, and doesn't weight the quality of data so that it can balance out potentially bad polling from actual good polling - though I should note the UK (tends) to have much better regulations on polling than the U.S does when it comes to public reputations. The point of such a method though is to catch the likely outcome if the election "were held today" more so than a more complex formula that attempts to anticipate future momentum swings in the race. In other words, its more of a "Now-Cast" or snapshot of the race.
Again, I use this method for state by state polling in the U.S presidential elections, but it comes in handy (at least in my experience) with UK general elections as well. Based on the polling averages I got from the UK as a whole, Scotland, and Wales I ended up getting this forecast for last year's race. Note that while not perfect, it captured the outcome of a pretty sizable Tory majority as the most likely (and eventual) result. Compare the projection to the final results (which I also featured down below) and they're really not that much off from one another.
As I warned before, this will likely be the last word from me on the subject of UK general election psephology for a good while now for this newsletter, as again the next UK general election will likely be years away, and my attention will be mostly focused on U.S psephology projects regarding the election of the presidency for the next weeks and months of posts. But until then, this look at historical UK data from electoral outcomes and how my method of poll averaging guided me to a respectable performance in 2019 will do.
Once the 2020 U.S presidential election is fully certified, I will next do a deep dive on a post-mortem on how I did attempting to predict that race using a simple averaging of the public polls and public MRP estimates, as well as a look at forecast error in the race and the swing from 2016 to 2020 in what turned out to be what I’d refer to as 75% of a change election and 25% of a choice election.