Author: Shaun Roach

In a previous post, I claimed numeric projections should be distributions. This is an example of something that can be built on top of such a projection system. It is an evaluation of a 12 team draft for a 5×5 fantasy baseball league.

Team 3 has a 30% chance of winning the league with this roster:

Evan Gattis Hou – C
Paul Goldschmidt Ari – 1B
Dustin Pedroia Bos – 2B
Kyle Seager Sea – 3B
Carlos Correa Hou – SS
George Springer Hou – OF
Giancarlo Stanton Mia – OF
José Bautista Tor – OF
Todd Frazier CWS – 1B,3B
Jake Lamb Ari – 3B
Carlos Martínez StL – SP
Cole Hamels Tex – SP
Jeurys Familia NYM – RP
Héctor Neris Phi – RP
J.A. Happ Tor – SP
Brandon Kintzler Min – RP
Chris Tillman Bal – SP
Jon Gray Col – SP

Distributions give more traction for calculating probabilities, all the way up the chain from player projections, to team-by-team category projections, and finally, probabilities for winning the league. I am working on a short essay on decisions to expand on these ideas, as well as an automated system for evaluating drafts.

If you want to be notified when this system is up, please submit this form:

*quick note on the points histograms, the labels on the x-axis should be divided by 2

 

Here is a link to some raw projections for the 2017 baseball season:

2017 Major League Baseball Projections

I will add some updates in the next couple days to better handle the distribution tail information, which isn’t quite represented accurately in these histograms, and also to incorporate additional information like injury news and playing time news.

Next week I’m releasing projections for the 2017 Major League Baseball season. These projections differ from others because they are distributions. I believe most projections should include a distribution, range, or some other precision indicator.

To argue why, I need to be a little obtuse and deconstruct the idea of a projection. It is an educated guess about a future metric. It is not, however, a statement of certainty. When someone projects a single number, like 35 home runs, we understand if the total at the end of the season is not exactly 35. We just expect it to be close. 

Single value predictions are useful because they allow you to rank players by their expected output, and make meaningful drafting/bidding decisions based on the lists. We must not, however, disregard the assumptions we know are explicit in projections, which we just acknowledged: these rankings are not certainties. We expect the rankings compiled at the beginning of the season to differ from the ones at the end of the year. We know there will be unexpected injuries, breakouts, and benchings. Some systems make this inexactness explicit, and include ranges or confidence intervals.

If we want to consider uncertainty when valuing players, we have to start with projections that include many possibilities. That is, projections that are not a single value, but also include measures at least for range, and ideally also skew, and extreme upside and downside events.

Here is a link to an example. I hope projections like these will be helpful in constructing draft rankings, and also more complex decision tools.

One of the bases of the quncertain platform is that probabilities are not measures of physical quantities. Rather, they are measures of the limits of your knowledge.

From this point of view, making educated guesses about the future is very similar to making educated guesses about the past. Of course there are a couple of differences.

For one, it is possible to have complete knowledge about certain aspects of the past. This is not true of the future.

Also, even if we have not memorized facts about the past completely, we still have ‘contact knowledge’ about the past. We may have heard about a heat wave in the eastern part of the US, or about a crash in the price of commodities, or a sports teams’ winning streak. These incomplete pieces of knowledge help inform us about specific events in the past.

But, unless you have encyclopedic access to past events, predicting events in the future is the same as predicting events in the past. With an extra dose of uncertainty.

Quncertain uses this equivalence to give immediate feedback on past predictions, allowing users to calibrate the plausibility they lend to their predictions.

 

 

Here are some common investing assumptions. How much plausibility do you lend to each one?

1. Though the stock market will fluctuate, it tends to go higher. An index fund which tracks the US stock market will be worth more in a year. 5 years? 10 years? 20 years?
2. Real Estate is a good investment. A house purchased now will be worth more in a year. 5 years? 10 years? 20 years? Portland market? Detroit market?
3. Apple is the best company in the world, and investing in AAPL will give above average returns.

I hope to supplement this post with an interactive question set for each of the assumptions above. There are times in the past where each of the above statements turned out to be true, and times when each of them turned out to be false. The financial questions on quncertain.com will currently let you do something close — predict the performance of different sectors of the US economy, both in the past and in the future.

You will find that (unless you cheat), you will ‘predict’ the past imperfectly. And predicting the future is even more uncertain. This is not a revelation, but most people feel they have some insight into the market at some point of time. Maybe using this app will help you verify that feeling.

People have taken different strategies in reaction to efficient markets that are almost impossible to predict:

1. Invest passively in index funds
2. Opt out and keep cash
3. Be a contrarian and fade the market when it makes large moves in either direction
4. Be a momentum trader and pile in and out of stocks in proportion to market movements
5. Use computers to model and predict the market
6. Trade other people’s money, and take your guaranteed profit from salary, commissions, and other incentives (in other words, take your own skin out of the game)

Each of these strategies has assumptions, and we should realize that these assumptions may not be true. I hope using this site will help you understand that there is uncertainty in all forward-looking statements. It is wise to prepare for your assumptions to be incorrect. This is called risk management, an area I will write more about. Probabilities are just one of the inputs needed in handling risk optimally.

Quncertain is a platform for making many predictions, in domains you are very knowledgeable and domains where you’re very ignorant. As you make enough predictions you come to realize a couple things:

First, you realize that even in domains where you are very knowledgeable, and about topics where you feel very confident, you will sometimes be wrong.

Second, you realize that even in domains where you are very ignorant, you usually are able to answer at a higher rate than a completely naive guess (assuming you’re not arbitrarily selecting answers and are actually trying to be correct).

In the quncertain platform, the first scenario will manifest in making wrong predictions about things you assign a high plausibility.

The second scenario will manifest in topics where you continually pick an answer, but pick it at only 50% plausibility. Giving with an answer with 50% plausibility is like saying “I don’t know” but being forced to pick an answer anyway. You’ll find that over time you get a higher rate than 50% of these correct.

This is because you are tangentially aware of some knowledge in most domains. You don’t follow the stock market, but recognize certain company names. You know cities in the southern US are generally warmer than cities in the north. You have heard of certain books or authors or athletes, which makes them more likely to be higher on lists than more obscure options.

People might start overconfident, incorrectly answering items they rated with a high plausibility. The solution is to realize it and be more cautious in the amount of plausibility you assign to your answers. Or, they might start underconfident, and see that they are getting answers right at a higher rate than the 50% or 60% plausibilities they’re assigning. You adjust by consciously raising the plausibility you assign to answers you were previously conservative about.

If you push yourself both ways, using the calibration charts for feedback, you’ll have a better idea of how different levels of plausibility ‘feel’.

 
And, importantly, you will know to have contingency plans when you make decisions with real life consequences. Even in things where you’re very certain, prepare a little to be wrong. And even if you’re completely unsure about some outcome, listen for, and have a little faith in your gut feeling.