Sunday, February 11, 2018

Predictions for 2017 - Analysis

One of the first posts I made on this blog was a set of 50 predictions for 2017, each with probabilities attached. Now it's time to see how I did.

1) Maxime Bernier will win the CPC leadership - 50%
Result: WRONG. Close, but wrong.
2) Kellie Leitch will not win the CPC leadership - 80%
Result: RIGHT. Given how she actually did, I actually feel under-confident here.
3) There will be 5 or fewer candidates on the final CPC leadership ballot - 60%
Result: WRONG. I misunderstood the process, and didn't realize how little reason to drop out there was, or how early you had to drop out.
4) Jason Kenney will win the PCAA leadership - 90%
Result: RIGHT.
5) The PCAA and Wildrose parties will not have begun any official merger process - 70%
Result: WRONG.
6) There will be no Danielle Smith-style defection of multiple Wildrose MLAs to the PCAA(or vice-versa) - 90%
Result: RIGHT.
7) There will be less than 3 candidates who formally register for the federal NDP leadership - 80%
Result: WRONG. This one was somewhat snarky on my part, and I let it mess with my judgement.
8) The expected 2017-18 deficit announced in the 2017 federal budget will be higher than predicted in the 2016 budget($29.0 billion) - 70%
Result: WRONG. It was actually $28.5 billion, so close, but wrong.
9) The expected 2017-18 deficit announced in the 2017 Ontario budget will be higher than predicted in the 2016 budget(balanced) - 70%
Result: WRONG. It seems they might actually run a deficit, but they didn't predict one, so this fails.
10) Approval of the performance of Justin Trudeau will be below 50% in at least one major scientific poll - 80%
Result: RIGHT. It was a squeaker, though.

11) The UK will invoke Article 50 and begin the formal Brexit process - 95%
Result: RIGHT.
12) Theresa May will remain PM of the UK - 99%
Result: RIGHT. That said, this one is case in point for why 99% is the highest certainty allowed in this scheme. This came shockingly close to failing.
13) There will be no second referendum on Brexit, actual or planned - 99%
Result: RIGHT.
14) The UK will initiate public talks with at least five non-EU nations for free trade deals - 80%
Result: I can't find any convenient list of deals underway. There's definitely talks with the US, and seemingly at least some PM-level discussions with Japan, but I can't find an easy way to judge this as either true or false. Which means it's probably false, to be fair.
15) No public process for a CANZUK free trade/free movement agreement will have begun - 80%
Result: RIGHT. A bit fuzzy on my phrasing, but governments haven't made any serious steps that I can see.
16) No deal will be reached between the UK and the EU on the terms of Brexit - 95%
Result: WRONG. They reached a deal in December.
17) No other nation will decide to leave the EU or Eurozone - 80%
Result: RIGHT.
18) Marine Le Pen will make it to the French Presidential runoff election - 60%
Result: RIGHT.
19) Marine Le Pen will not win the Presidency of France - 95%
Result: RIGHT. No surprises here - 30% of the population likes the alt-right and 70% hates them in most countries, and they were always going to unify behind whoever the sane one was.
20) Angela Merkel will be re-elected Chancellor of Germany - 70%
Result: RIGHT. Pretty close, which was expected(hence my comparatively low confidence here), but she made it.
21) There will be at least one explicitly Islamic terrorist attack killing 50+ people in the OECD - 70%
Result: WRONG. There were several nasty attacks, but the highest number killed by any was 39. This is one where I'm glad to be wrong, of course, but wrong I am.
22) There will be no terrorist attack killing 500+ people in the OECD - 90%
Result: RIGHT.
23) ISIS will remain a de facto nation with control of at least some territory - 80%
Result: RIGHT. Not a lot, but there's a few areas in Syria that they still hold(the grey parts on this map).
24) The US will continue to attack ISIS(at least until all their territory is lost) - 90%
Result: RIGHT.
25) The US will not begin any substantial new conflicts(Libya 2011 or larger) - 80%
Result: RIGHT.
26) Approval of the performance of Donald Trump will be above 50% in at least one major scientific poll in the last three months of the year - 80%
Result: WRONG. Again, I let my cynicism get the better of me. The highest I can find is 46%.
27) The US Senate will eliminate the filibuster for at least some votes - 70%
Result: RIGHT. The Gorsuch vote went nuclear.
28) The US will withdraw some form of support from the anti-Russian nations of Eastern Europe - 80%
Result: WRONG. Trump has been better than I thought he would be here.
29) No nuclear weapons will be used against civilian targets - 95%
Result: RIGHT. Thankfully so.
30) At least one world leader will be elected on a populist campaign routinely mentioned alongside Trump and Brexit - 80%
Result: WRONG. I can't think of any examples, at least.

31) Using the same methodology as my discussion of 2015-16 celebrity deaths, at least 10 celebrities will die - 90%
Result: RIGHT. 15 people, average age 62, which actually makes it slightly worse than 2016. (The list is David Cassidy(67), Tom Petty,(66), Malcolm Young(64), Roy Halladay(40), John Heard(71), Chester Bennington(41), Chris Cornell(52), Stephen Furst(53), John Hurt(77), George Romero(77), Gregg Allman(69), Bill Paxton(61), Richard Hatch(71), Jimmy Snuka(73), and Gord Downie(53)).
32) Canada will not have a natural disaster as devastating as the 2016 Aberta wildfires - 80%
Result: RIGHT.
33) No major green energy breakthrough(>50% improvement in output per dollar) - 90%
Result: RIGHT.
34) Global average temperature is higher than 2016 - 80%
Result: WRONG. 0.1 degrees cooler. That said, looking at recent history, I'd say this was pretty well-calibrated.
35) No natural disaster will kill 10,000 or more people - 80%
Result: RIGHT. The deadliest was the South Asian flooding, that killed about 1,200.
36) No natural disaster will kill 100 or more people in the OECD - 90%
Result: WRONG. There was an earthquake in Mexico City that killed 225.
37) There will be at least one computer security breach affecting 50 million or more accounts - 80%
Result: RIGHT. The Equifax breach was 143 million accounts.
38) No film released in 2017 will have a global box-office gross over $1.5B - 50%
Result: RIGHT. The highest was Star Wars: The Last Jedi, at $1.32 billion. (It's still in theatres, but it made $3M last week, so I don't think I need to worry).
39) The MSCI All-Cap World Index index will increase faster than inflation in USD - 60%
Result: RIGHT. Substantially faster, actually - it grew 23.16%.
40) 2017 will be the most prosperous year in human history - 95%
Result: RIGHT. Global real GDP per capita grew 2.5% last year, as it has grown every year since 2009.

41) I will be married - 95%
Result: RIGHT. :D
42) Our wedding will be under budget - 70%
Result: RIGHT. Substantially so, which was really nice.
43) I will have a full-time job - 95%
Result: RIGHT. This was a riskier one at the time than I wanted to really admit to myself, because when I wrote the original list, I didn't. But I was confident of my ability to find one.
44) I will have a permanent full-time job - 80%
Result: RIGHT. And rightly so!
45) I will no longer be doing financial plans as a side job - 70%
Result: RIGHT. Not something you can do when you're employed by a bank.
46) I will still be posting Friday Night Videos - 95%
Result: WRONG. Sorry, folks.
47) I will be making regular blog posts - 70%
Result: WRONG. Clearly, I fell off that wagon.
48) I will have a new desktop computer - 80%
Result: RIGHT. Got it back in August, and it's wonderful.
49) I will have my CFP and CFA - 80%
Result: WRONG. I have the necessary experience, but I've been procrastinating on my paperwork for it.
50) I will complete the design of at least one game - 60%
Result: WRONG. Too distracted by other stuff. But I still have a bunch in the works.

So, to summarize:
50% Confidence: 
1 right (#38), 1 wrong(#1) = 1/2 = 50%. Perfectly calibrated.
60% Confidence: 
2 right(#18 and 39), 2 wrong(#3 and 50) = 2/4 = 50%. Perfect calibration is 2.4 correct, so this is the best result that can be managed with this few guesses.
70% Confidence: 
4 right(#20, 27, 42, and 45), 5 wrong (#5, 8, 9, 21, and 47) = 4/9 = 44%. Overconfident, best result would be 6/9 correct.
80% Confidence: 
12 right(#2, 10, 15, 17, 23, 25, 28, 32, 35, 37, 44, and 48), and 5 wrong (#7, 26, 30, 34, and 49) = 12/17 = 71%. Best result would be 14 correct. (If we treat #14 as wrong instead of null, this is 12/18 = 67%, best result still 14 correct).
90% Confidence: 
#4 right, #6 right, #22 right, #24 right, #31 right, #33 right, #36 wrong,
6 right (#4, 6, 22, 24, 31, and 33), and 1 wrong(#36) = 6/7 = 86%. Still best possible result.
95% Confidence: 
#11 right, #16 wrong, #19 right, #29 right, #40 right, #41 right, #43 right, #46 wrong,
6 right(#11, 19, 29, 40, 41, and 43), 2 wrong (#16 and 46) = 6/8 = 75%. Overconfident, best result would be 8/8.
99% Confidence: 
2 right(#12 and 13), no wrong = 100%. Best result.

Overall, I'm somewhat overconfident, and predicted an average of 39 correct answers, when I only got 33 right. The 70% and 95% brackets in particular were bad for me. I suspect this is still better than most people would manage, but I think I can do better - a few of my mistakes seem obvious in retrospect. Also, I may want to change up what I guess on in order to spread things out a bit - there weren't many data points at either end, and that makes it tougher to know if I did well by luck or skill.

No comments:

Post a Comment