That is, suppose the event is that the Democrats take the Senate and pass a law outlawing healthcare. Here’s one helpful intuition: Your score should be the same whether you predict the pieces of events separately or together. It sounds like maybe there’s no scoring rule that both penalizes you worse for being confidently wrong, and has guessing 50% when you genuinely don’t know neither gain or cost you points? That makes more sense, except that in log odds, guessing 50% always gives you 0, but IIUC this way guessing 50% always gives you -0.69.
On your first prediction you lose ~4.6 points, on your The probability you assigned to the truth (and thus is always I shouldn’t be back to zero unless I make ninety-nine 99% predictions that are right and one that is wrong.Īm I using log odds wrong, or is there some better scoring rule that naturally captures the intuition that 99% failures should count for more than 99% writes: This feels intuitively bad? If I made one 99% prediction that was right, and then another 99% prediction that was wrong, I should be way in the hole. Then if I predict 99% chance the stock market will go up tomorrow, and it doesn’t, I get ln(0.01/0.99) = -4.59. So if I predict 99% chance the sun will rise this morning, and it does, I get ln(0.99/0.01) = 4.59.
My understanding is that the most common proper scoring rule is log odds, where you add the log odds of everything you predicted. Okay, people who understand statistics, help me out.
0 kommentar(er)
