Leaders need to make strategic decisions in a world full of randomness and uncertainty. Visionary organizations are increasingly using data and statistical inference to make predictions and to inform decision making. Most of the commonly used statistical measures are based on the “bell curve” or “normal” distribution, which assumes mild uncertainty. However, many real world processes have much more extreme uncertainty. This can lead to huge problems when utilizing models or measures that assume bell curve characteristics. The objective of this whitepaper is to help you improve strategic decision making by (A) understanding the characteristics and implications of extreme uncertainty, and (B) providing techniques to improve decisions and outcomes under these conditions.
This paper is organized as follows:
Random processes are pervasive in nature and in our lives, yet they are routinely misunderstood, especially when uncertainty is high. The most commonly used statistical methods are based on the “bell curve”, also known as the “normal” or Gaussian distribution. Popular measures such as standard deviation (or “sigma”), variance, and R-squared all stem from the bell curve.
With the bell curve, the extremes (or “tails”) of the distribution decline to zero exponentially or faster. This means that extreme events become increasingly less likely the larger the event. Such distributions are called “thin tailed”.
Many real-world processes follow a normal distribution. For example, the height and weight of people, the number of people who die each day from heart attacks or car accidents, and the results from throwing a dice or flipping a coin all follow a normal (thin tailed) distribution. This means that even extreme examples will not significantly alter the statistical properties of a reasonably sized sample. If you take 100 randomly chosen people and average their height, even if you were to add someone 8-feet tall, it wouldn’t change the average height by much (less than 1%).
In contrast, there are also many real world processes where the variations are much more explosive. In these cases, extreme events far from the center of the distribution are more likely. Such distributions are called “fat tailed”. With fat tailed processes, a single example can completely dominate the statistical properties of even a large sample. If you take 100 randomly chosen people (assume average net worth of $500K), and you add a billionaire to your sample, you will increase the average by 2000% (20X)!
One of the most common fat tailed distributions is the Pareto distribution (aka the “80/20 rule”), named for Vilfredo Pareto who noticed empirically that 80% of the land in Italy belonged to 20% of the population. Additional processes that have been shown to have fat tails: the distribution of wealth, financial market prices, venture capital returns, the size and cost of wars, casualties from pandemics like COVID-19, damage from natural disasters (such as hurricanes, earthquakes, and floods), sales by customer for a business, movie sales, hospital bills, and popularity of search terms on Google (to name a few).
Applying “normal” statistics in such cases can lead to huge problems.
Processes with fat tails have many important characteristics that distinguish them from those with thin tails.
With fat tails, extreme events are more likely. Moreover, there is no such thing as a typical large deviation. To further illustrate using height and wealth as examples…
Thin tails: If you are told that two random people have a combined height of 14 feet, it is way more likely that you have two 7 footers than one person who is 13 feet tall and another who is 1 foot tall.
Fat tails: If you are told that two people chosen at random have a combined wealth of $50 million, it is much more likely that one is worth ~$50 million and the other is worth close to nothing, than it is that both are worth $25 million.[8]
Means and standard deviations are highly influenced by extreme values. However, it is common for potential extremes, since they are rare, to be absent from the data sample or historical record. In other words, fat tailed distributions often have hidden tails.
This makes traditional statistical tools misleading and potentially dangerous under fat tails. It is easy to calculate the mean and standard deviation of the sample, however these measures will not be representative of the underlying distribution. Other popular techniques such as linear regression, z-test, t-tests, and bootstrapping will perform poorly. Historical data will be unreliable for prediction and can lead to huge underestimation. For example, Cirillo and Taleb conclude that the expected number of deaths from wars are underestimated by at least a factor of three if we base our expectation on historical averages.[2]
Traditional statistical estimation is based largely on two key principles: the Central Limit Theorem (CLT) and the law of large numbers. The CLT states that if you take a sufficiently large sample from a thin-tailed distribution, the distribution of the sample mean will follow a normal distribution, even if the original distribution is heavily skewed or multimodal. This is a very useful property.
However, the Central Limit Theorem does not work with fat tails. The uncertainty in the mean may not converge even with large data samples. And if it does converge, it does so much more slowly, requiring way more data (WAY more). With fat tails, the law of large numbers works too slowly to be applicable in the real world. To illustrate: it takes about 30 observations for the normal distribution (thin tailed) to stabilize. Whereas it takes 10^11 observations (that’s 100 billion) for the Pareto distribution to converge the same amount. [8]
A typical decision framework or cost-benefit analysis considers the possible outcomes (the magnitudes of benefits and/or losses) and their respective probabilities. However, as discussed, “expected values” under fat tails (which are based on historical means) can be very misleading, and the probabilities of rare events are highly uncertain or unknowable. Moreover, extreme events can have effectively infinite costs, which causes the cost-benefit analysis to break down completely.[8]
Non-parametric statistics, such as the median, sign test, chi-square test, and others, are commonly recommended and used for non-normal distributions. If your concern is “typical” values at the center of the distribution, these are helpful measures. If you care about risk or rare events (e.g., in insurance or epidemiology), these measures are meaningless and can make things worse by obscuring the potential extremes.
With the thin-tailed distributions, the mean excess function (the expected value above a threshold, given that the threshold is exceeded) decreases to zero with large thresholds. The ratio of successive record values decreases. However with fat tails, the differences between the largest values are larger than the differences in the smaller values. The expected excess beyond a threshold increases as the threshold increases, assuming the threshold is exceeded. The differences between successively larger observations will increase. In other words, there is no such thing as a typical large deviation. This implies that we should expect the next worst-case example to be significantly worse than the current worst-case example.[3]
Linear regression and other predictive modeling techniques are commonly used to approximate the value of some target variable (dependent variable) based on some predictor variables (independent variables). Models are built based on historical data. We wish to use these models to make predictions on new data, which our model has never seen before.
A central problem in forecasting is that we typically have observed data without a good understanding of the underlying distribution (that is, the generator that's producing the data). This is especially acute under fat tailed processes. In any finite sample (say, for historical data on floods, earthquakes, or stock returns), the average and standard deviation are always finite and easy to compute based on historical data. However, these measures can have negligible relevance to future predictions.
With fat tails, regression coefficients will behave erratically, making them unstable and unreliable. Even with increased sampling (more data), the sample statistics may not converge. If they aren’t converging, the results will whiplash erratically, along with any predictions we would base on these results. R-squared will be uninformative. For extreme fat tails, R-squared will be effectively zero.[6]
When the impacts of a process are local and independent, risk is contained. When processes have high levels of interdependence, risks can cascade and propagate through the system and the impacts can be large and systemic, greatly increasing the risk of ruin. Examples include global financial crashes, the impact of invasive species, and casualties from global pandemics.
Note that the significant increases in global travel have greatly increased interdependence, connecting regions and systems that were previously isolated. As connectivity increases, the risk of extremely severe outcomes from pandemics increases dramatically.[10]
Note also that in the area of risk management, under fat tails, ruin is more likely to come from a single extreme adverse event than from a series of adverse events.
Despite the challenges posed by fat tailed distributions, there is much we can do to improve our decisions and outcomes.
Hopefully by now it’s clear that we should seek to understand the nature of the distributions underlying our data, and in particular, to understand whether we are operating with thin tails or fat tails.
This can be challenging: it may be unclear or indiscernible whether a distribution is thin tailed or fat tailed. As discussed, a key feature of fat tailed distributions is that they have rare, extreme events. Hence, it is common for potential extremes to exist in the underlying distribution that we have not yet observed. A fat tailed distribution can appear like a thin tailed distribution. If we see a distribution with no extreme examples, we cannot be sure it is thin tailed, especially if we have a small dataset. On the other hand, if you see a “10-sigma” event, even with a tiny dataset we can safely conclude that our distribution is NOT thin-tailed (in this case it is far more likely that we have the wrong distribution and the wrong estimate of sigma).[8]
There are several ways of estimating tail thickness based on sample data. See the Appendix for details.
When faced with an “outlier”, you may be tempted to simply throw away the extreme data point. An insurance actuary might see that one or a few very large data points heavily influences their models (say for a hurricane, earthquake, or wildfire), and that by discarding them the resulting dataset looks more “normal.” Yet these departures from normality can provide important clues about the data. The fatter-tailed the distribution, the more statistical information resides in the extreme values than in the rest of the distribution. Unless they are truly false data points (e.g., a data-entry error), discarding them reduces information and is a serious mistake. Don’t automatically discard the extreme values!
The appropriate reaction to risk depends on our degree of exposure. Of course all risks are not created equally. When processes are localized and/or independent, risks are not systemic. In this case, an extreme reaction could do more harm than good. When processes have high interdependence and the ability to propagate and spread afar, the risks can be systemic and disastrous. In this case, an extreme reaction may be necessary.
It is extremely common to see people compare casualties from a thin-tailed process (say deaths from car accidents or heart attacks) to those from a fat-tailed distribution (say deaths from wars or pandemics). This is a false comparison and a common error in decision making and policy making. If conclusions are inferred assuming thin-tailed properties, risk will be underestimated and any conservative reaction will be viewed as an overreaction.
“People quote so-called “empirical” data to tell us we are foolish to worry about Ebola when only two Americans died of Ebola in 2016… Let us think about it in terms of tails. If we were to read in the newspaper that 2 billion people have died suddenly, it is far more likely that they died of Ebola than smoking or diabetes.”[8]
With fat tails, we need to pay much greater attention to the potential for extreme outcomes and their consequences. If we are concerned with floods, or earthquakes, or pandemics, we care about the extremes, not the averages or medians.
“Just like the FAA and the FDA who deal with safety by focusing on catastrophe avoidance, we will throw away the ordinary under the rug and retain extremes as the sole sound approach to risk management.”[8]
It’s obvious that we should be more conservative when risks are severe.
The precautionary principle states that if an action has the potential to cause extreme harm, the action should not be taken unless there is strong evidence confirming its safety. This means we should not accept arguments that “there is no evidence of harm” but rather insist on evidence that there is no harm. The burden of proof in these circumstances falls on the party proposing the action, not on those opposing it. This distinction between “absence of evidence” and “evidence of absence” is a critical one.[6, 9, 10]
The precautionary principle is especially relevant early in a crisis when data and therefore statistical evidence is limited. This is the period when absence of evidence can be paralyzing. It may be tempting to wait until more data (evidence) can be collected, but this can significantly increase the risk of ruin. The evidence may come too late! Instead, we should start out by erring on the conservative side, at least until evidence can be collected.
Consider financial crashes, terrorist actions, or pandemics. Note that evidence about these events typically follows the event rather than preceding it. This is a common feature of the rare, extreme event. We do not wait for evidence of an accident before putting our seat belts on. We do not wait for evidence of a burglar in the neighborhood before locking our doors at night. Similarly, it is rational to react with “paranoia” in response to a pandemic, recognizing that “over-reaction” is responsible risk management in the face of a potentially catastrophic threat. As the saying goes, it is better to mistake a stone for a bear (and look silly) than to mistake a bear for a stone (and get eaten).
As discussed above, it may be unclear whether a process is thin tailed or fat tailed. If in doubt and the risk is systemic, we should be conservative and assume fat tails. Greater uncertainty argues for greater caution and precaution.
As discussed above, forecasting under fat tails is problematic. When forecast models have high uncertainty, we should act with greater caution. While we should be skeptical about such forecasts and any cost-benefit analysis based on them, that does not mean that they are not useful.
For example, in a pandemic, extreme forecasts should be viewed as possibilities, not likely scenarios, and our risk mitigation efforts should consider those possibilities. Precautionary measures should be driven by the potential magnitude of effects rather than on specific predictions or forecasts. And when forecast models have high uncertainty, we should act with greater precaution, not less.
Similarly, in economic forecasting, we can build models based on historical data. However, we need to recognize the limitations of such models.
Econometric methods “capture the ordinary and mask higher order effects. Since blowups are not frequent, these events do not show in data and the researcher looks smart most of the time while being fundamentally wrong.”[8]
As discussed, under fat tails the probabilities of events are largely opaque or incalculable. However, we can gain advantage by concentrating on our exposure to events (the benefits or risks) rather than the probabilities of those events. Recognize the distinction between (a) the probability of events and (b) the outcomes or impacts of those events. If X is our probability distribution, F(X) is the outcome or impact of X. We usually have much greater control over F than we do over X.
“It is vastly more effective to focus on being insulated from the harm of random events than to try to figure them out in the required details… It is more solid, much wiser, more ethical, and more effective to focus on detection heuristics and policies rather than fabricate statistical properties.”[8]
Clearly, we need to minimize or eliminate exposure to fat tailed risks. This means we should avoid things that make us fragile, such as single points of failure or excessive debt. It means we should insure or hedge against catastrophic risk. It means we should be willing to invest in redundancies, even though that creates “inefficiencies” and comes at the expense of “optimization.”
We should also recognize that there are fat tailed processes with positive upside, such as those stemming from innovation. We benefit from exposure to such “positive” fat tails.
For example, innovations and new ventures can be viewed as options, which (if managed well) have low downside and high upside. This illustrates the benefits of bottom-up experiments and tinkering, which allow more options at lower cost, compared to large-scale top-down interventions which provide less optionality at significantly higher cost. This is reflected in the work of venture capitalists, who invest relatively small amounts in early stage companies, and progressively larger amounts as opportunities are validated and risks are reduced.
Note also the value of optionality that comes from having flexible plans, allowing us to adapt to new information and take advantage of serendipity, as opposed to being locked into fixed long-term plans.
Finally, note the value of “serial optionality,” the opportunity to pursue opportunities serially, one after the other. We can always begin anew.
“What I’ve learned, above all, is to keep marching forward because the best news is that since chance does play a role, one important factor for success is under our control: the number of at bats, the number of chances taken, the number of opportunities seized. For even a coin weighted toward failure will sometimes land on success. Or as the IBM pioneer Thomas Watson said, ‘If you want to succeed, double your failure rate.’”[4]
There are many ways to estimate tail thickness based on sample data.
Calculate kurtosis – You can compute the sample's kurtosis, which is a measure of the "tailedness" of the distribution. Kurtosis is also known as the fourth moment (the first moment is the mean, the second moment is the variance, the third moment is the skewness.)
Evaluate convergence of moments – You can estimate tail thickness by evaluating the convergence of moments. For thin tailed distributions, the mean and higher moments will converge according to the Central Limit Theorem and the law of large numbers. However, with fat tails the convergence is considerably slower, if it converges at all, and requires much more data. To illustrate, in over 50 years of S&P 500 performance, a single day represents over 80% of the kurtosis. This extreme sample dependence suggests that the 4th moment of the S&P 500 does not have stability, and we can rule out thin-tailedness for this distribution.[7, 8]
Estimate the tail index (Plot the data on a log-log plot) – Pareto and other power law distributions have the form F(X) = k * X^n. Taking the log of both sides, leaves us with log (F(X)) = log(k) + n*log(X). Hence, if the log-log plot of our distribution follows a straight line, we have a power law relationship and a fat tailed distribution. The slope of the line provides an estimate of the tail index.[1]
Evaluate the mean excess above a threshold – The mean excess function gives the expected amount that a random variable will exceed a given threshold, on the condition that the threshold is exceeded. In the insurance industry, the mean excess function is the expected claim size above a threshold, given that the threshold is exceeded. With thin tailed distributions, the mean excess function decreases with increased threshold magnitude. Conversely, with fat tailed distributions, the mean excess function increases as the threshold increases. The magnitude of this increase relates to the degree of thickness.[3]
Apply Extreme Value Theory – Extreme value theory is concerned with the distribution of extreme events from a process. Extreme values are collected either in blocks (e.g., by extracting annual extrema) or by taking all peak values over a threshold. By evaluating the resulting distribution, and the probability of exceeding a threshold value, tail properties can be inferred.
Leaders need to make strategic decisions in a world full of randomness and uncertainty. Visionary organizations are increasingly using data and statistical inference to make predictions and to inform decision making. Most of the commonly used statistical measures are based on the “bell curve” or “normal” distribution, which assumes mild uncertainty. However, many real world processes have much more extreme uncertainty. This can lead to huge problems when utilizing models or measures that assume bell curve characteristics. The objective of this whitepaper is to help you improve strategic decision making by (A) understanding the characteristics and implications of extreme uncertainty, and (B) providing techniques to improve decisions and outcomes under these conditions.
This paper is organized as follows:
Random processes are pervasive in nature and in our lives, yet they are routinely misunderstood, especially when uncertainty is high. The most commonly used statistical methods are based on the “bell curve”, also known as the “normal” or Gaussian distribution. Popular measures such as standard deviation (or “sigma”), variance, and R-squared all stem from the bell curve.
With the bell curve, the extremes (or “tails”) of the distribution decline to zero exponentially or faster. This means that extreme events become increasingly less likely the larger the event. Such distributions are called “thin tailed”.
Many real-world processes follow a normal distribution. For example, the height and weight of people, the number of people who die each day from heart attacks or car accidents, and the results from throwing a dice or flipping a coin all follow a normal (thin tailed) distribution. This means that even extreme examples will not significantly alter the statistical properties of a reasonably sized sample. If you take 100 randomly chosen people and average their height, even if you were to add someone 8-feet tall, it wouldn’t change the average height by much (less than 1%).
In contrast, there are also many real world processes where the variations are much more explosive. In these cases, extreme events far from the center of the distribution are more likely. Such distributions are called “fat tailed”. With fat tailed processes, a single example can completely dominate the statistical properties of even a large sample. If you take 100 randomly chosen people (assume average net worth of $500K), and you add a billionaire to your sample, you will increase the average by 2000% (20X)!
One of the most common fat tailed distributions is the Pareto distribution (aka the “80/20 rule”), named for Vilfredo Pareto who noticed empirically that 80% of the land in Italy belonged to 20% of the population. Additional processes that have been shown to have fat tails: the distribution of wealth, financial market prices, venture capital returns, the size and cost of wars, casualties from pandemics like COVID-19, damage from natural disasters (such as hurricanes, earthquakes, and floods), sales by customer for a business, movie sales, hospital bills, and popularity of search terms on Google (to name a few).
Applying “normal” statistics in such cases can lead to huge problems.
Processes with fat tails have many important characteristics that distinguish them from those with thin tails.
With fat tails, extreme events are more likely. Moreover, there is no such thing as a typical large deviation. To further illustrate using height and wealth as examples…
Thin tails: If you are told that two random people have a combined height of 14 feet, it is way more likely that you have two 7 footers than one person who is 13 feet tall and another who is 1 foot tall.
Fat tails: If you are told that two people chosen at random have a combined wealth of $50 million, it is much more likely that one is worth ~$50 million and the other is worth close to nothing, than it is that both are worth $25 million.[8]
Means and standard deviations are highly influenced by extreme values. However, it is common for potential extremes, since they are rare, to be absent from the data sample or historical record. In other words, fat tailed distributions often have hidden tails.
This makes traditional statistical tools misleading and potentially dangerous under fat tails. It is easy to calculate the mean and standard deviation of the sample, however these measures will not be representative of the underlying distribution. Other popular techniques such as linear regression, z-test, t-tests, and bootstrapping will perform poorly. Historical data will be unreliable for prediction and can lead to huge underestimation. For example, Cirillo and Taleb conclude that the expected number of deaths from wars are underestimated by at least a factor of three if we base our expectation on historical averages.[2]
Traditional statistical estimation is based largely on two key principles: the Central Limit Theorem (CLT) and the law of large numbers. The CLT states that if you take a sufficiently large sample from a thin-tailed distribution, the distribution of the sample mean will follow a normal distribution, even if the original distribution is heavily skewed or multimodal. This is a very useful property.
However, the Central Limit Theorem does not work with fat tails. The uncertainty in the mean may not converge even with large data samples. And if it does converge, it does so much more slowly, requiring way more data (WAY more). With fat tails, the law of large numbers works too slowly to be applicable in the real world. To illustrate: it takes about 30 observations for the normal distribution (thin tailed) to stabilize. Whereas it takes 10^11 observations (that’s 100 billion) for the Pareto distribution to converge the same amount. [8]
A typical decision framework or cost-benefit analysis considers the possible outcomes (the magnitudes of benefits and/or losses) and their respective probabilities. However, as discussed, “expected values” under fat tails (which are based on historical means) can be very misleading, and the probabilities of rare events are highly uncertain or unknowable. Moreover, extreme events can have effectively infinite costs, which causes the cost-benefit analysis to break down completely.[8]
Non-parametric statistics, such as the median, sign test, chi-square test, and others, are commonly recommended and used for non-normal distributions. If your concern is “typical” values at the center of the distribution, these are helpful measures. If you care about risk or rare events (e.g., in insurance or epidemiology), these measures are meaningless and can make things worse by obscuring the potential extremes.
With the thin-tailed distributions, the mean excess function (the expected value above a threshold, given that the threshold is exceeded) decreases to zero with large thresholds. The ratio of successive record values decreases. However with fat tails, the differences between the largest values are larger than the differences in the smaller values. The expected excess beyond a threshold increases as the threshold increases, assuming the threshold is exceeded. The differences between successively larger observations will increase. In other words, there is no such thing as a typical large deviation. This implies that we should expect the next worst-case example to be significantly worse than the current worst-case example.[3]
Linear regression and other predictive modeling techniques are commonly used to approximate the value of some target variable (dependent variable) based on some predictor variables (independent variables). Models are built based on historical data. We wish to use these models to make predictions on new data, which our model has never seen before.
A central problem in forecasting is that we typically have observed data without a good understanding of the underlying distribution (that is, the generator that's producing the data). This is especially acute under fat tailed processes. In any finite sample (say, for historical data on floods, earthquakes, or stock returns), the average and standard deviation are always finite and easy to compute based on historical data. However, these measures can have negligible relevance to future predictions.
With fat tails, regression coefficients will behave erratically, making them unstable and unreliable. Even with increased sampling (more data), the sample statistics may not converge. If they aren’t converging, the results will whiplash erratically, along with any predictions we would base on these results. R-squared will be uninformative. For extreme fat tails, R-squared will be effectively zero.[6]
When the impacts of a process are local and independent, risk is contained. When processes have high levels of interdependence, risks can cascade and propagate through the system and the impacts can be large and systemic, greatly increasing the risk of ruin. Examples include global financial crashes, the impact of invasive species, and casualties from global pandemics.
Note that the significant increases in global travel have greatly increased interdependence, connecting regions and systems that were previously isolated. As connectivity increases, the risk of extremely severe outcomes from pandemics increases dramatically.[10]
Note also that in the area of risk management, under fat tails, ruin is more likely to come from a single extreme adverse event than from a series of adverse events.
Despite the challenges posed by fat tailed distributions, there is much we can do to improve our decisions and outcomes.
Hopefully by now it’s clear that we should seek to understand the nature of the distributions underlying our data, and in particular, to understand whether we are operating with thin tails or fat tails.
This can be challenging: it may be unclear or indiscernible whether a distribution is thin tailed or fat tailed. As discussed, a key feature of fat tailed distributions is that they have rare, extreme events. Hence, it is common for potential extremes to exist in the underlying distribution that we have not yet observed. A fat tailed distribution can appear like a thin tailed distribution. If we see a distribution with no extreme examples, we cannot be sure it is thin tailed, especially if we have a small dataset. On the other hand, if you see a “10-sigma” event, even with a tiny dataset we can safely conclude that our distribution is NOT thin-tailed (in this case it is far more likely that we have the wrong distribution and the wrong estimate of sigma).[8]
There are several ways of estimating tail thickness based on sample data. See the Appendix for details.
When faced with an “outlier”, you may be tempted to simply throw away the extreme data point. An insurance actuary might see that one or a few very large data points heavily influences their models (say for a hurricane, earthquake, or wildfire), and that by discarding them the resulting dataset looks more “normal.” Yet these departures from normality can provide important clues about the data. The fatter-tailed the distribution, the more statistical information resides in the extreme values than in the rest of the distribution. Unless they are truly false data points (e.g., a data-entry error), discarding them reduces information and is a serious mistake. Don’t automatically discard the extreme values!
The appropriate reaction to risk depends on our degree of exposure. Of course all risks are not created equally. When processes are localized and/or independent, risks are not systemic. In this case, an extreme reaction could do more harm than good. When processes have high interdependence and the ability to propagate and spread afar, the risks can be systemic and disastrous. In this case, an extreme reaction may be necessary.
It is extremely common to see people compare casualties from a thin-tailed process (say deaths from car accidents or heart attacks) to those from a fat-tailed distribution (say deaths from wars or pandemics). This is a false comparison and a common error in decision making and policy making. If conclusions are inferred assuming thin-tailed properties, risk will be underestimated and any conservative reaction will be viewed as an overreaction.
“People quote so-called “empirical” data to tell us we are foolish to worry about Ebola when only two Americans died of Ebola in 2016… Let us think about it in terms of tails. If we were to read in the newspaper that 2 billion people have died suddenly, it is far more likely that they died of Ebola than smoking or diabetes.”[8]
With fat tails, we need to pay much greater attention to the potential for extreme outcomes and their consequences. If we are concerned with floods, or earthquakes, or pandemics, we care about the extremes, not the averages or medians.
“Just like the FAA and the FDA who deal with safety by focusing on catastrophe avoidance, we will throw away the ordinary under the rug and retain extremes as the sole sound approach to risk management.”[8]
It’s obvious that we should be more conservative when risks are severe.
The precautionary principle states that if an action has the potential to cause extreme harm, the action should not be taken unless there is strong evidence confirming its safety. This means we should not accept arguments that “there is no evidence of harm” but rather insist on evidence that there is no harm. The burden of proof in these circumstances falls on the party proposing the action, not on those opposing it. This distinction between “absence of evidence” and “evidence of absence” is a critical one.[6, 9, 10]
The precautionary principle is especially relevant early in a crisis when data and therefore statistical evidence is limited. This is the period when absence of evidence can be paralyzing. It may be tempting to wait until more data (evidence) can be collected, but this can significantly increase the risk of ruin. The evidence may come too late! Instead, we should start out by erring on the conservative side, at least until evidence can be collected.
Consider financial crashes, terrorist actions, or pandemics. Note that evidence about these events typically follows the event rather than preceding it. This is a common feature of the rare, extreme event. We do not wait for evidence of an accident before putting our seat belts on. We do not wait for evidence of a burglar in the neighborhood before locking our doors at night. Similarly, it is rational to react with “paranoia” in response to a pandemic, recognizing that “over-reaction” is responsible risk management in the face of a potentially catastrophic threat. As the saying goes, it is better to mistake a stone for a bear (and look silly) than to mistake a bear for a stone (and get eaten).
As discussed above, it may be unclear whether a process is thin tailed or fat tailed. If in doubt and the risk is systemic, we should be conservative and assume fat tails. Greater uncertainty argues for greater caution and precaution.
As discussed above, forecasting under fat tails is problematic. When forecast models have high uncertainty, we should act with greater caution. While we should be skeptical about such forecasts and any cost-benefit analysis based on them, that does not mean that they are not useful.
For example, in a pandemic, extreme forecasts should be viewed as possibilities, not likely scenarios, and our risk mitigation efforts should consider those possibilities. Precautionary measures should be driven by the potential magnitude of effects rather than on specific predictions or forecasts. And when forecast models have high uncertainty, we should act with greater precaution, not less.
Similarly, in economic forecasting, we can build models based on historical data. However, we need to recognize the limitations of such models.
Econometric methods “capture the ordinary and mask higher order effects. Since blowups are not frequent, these events do not show in data and the researcher looks smart most of the time while being fundamentally wrong.”[8]
As discussed, under fat tails the probabilities of events are largely opaque or incalculable. However, we can gain advantage by concentrating on our exposure to events (the benefits or risks) rather than the probabilities of those events. Recognize the distinction between (a) the probability of events and (b) the outcomes or impacts of those events. If X is our probability distribution, F(X) is the outcome or impact of X. We usually have much greater control over F than we do over X.
“It is vastly more effective to focus on being insulated from the harm of random events than to try to figure them out in the required details… It is more solid, much wiser, more ethical, and more effective to focus on detection heuristics and policies rather than fabricate statistical properties.”[8]
Clearly, we need to minimize or eliminate exposure to fat tailed risks. This means we should avoid things that make us fragile, such as single points of failure or excessive debt. It means we should insure or hedge against catastrophic risk. It means we should be willing to invest in redundancies, even though that creates “inefficiencies” and comes at the expense of “optimization.”
We should also recognize that there are fat tailed processes with positive upside, such as those stemming from innovation. We benefit from exposure to such “positive” fat tails.
For example, innovations and new ventures can be viewed as options, which (if managed well) have low downside and high upside. This illustrates the benefits of bottom-up experiments and tinkering, which allow more options at lower cost, compared to large-scale top-down interventions which provide less optionality at significantly higher cost. This is reflected in the work of venture capitalists, who invest relatively small amounts in early stage companies, and progressively larger amounts as opportunities are validated and risks are reduced.
Note also the value of optionality that comes from having flexible plans, allowing us to adapt to new information and take advantage of serendipity, as opposed to being locked into fixed long-term plans.
Finally, note the value of “serial optionality,” the opportunity to pursue opportunities serially, one after the other. We can always begin anew.
“What I’ve learned, above all, is to keep marching forward because the best news is that since chance does play a role, one important factor for success is under our control: the number of at bats, the number of chances taken, the number of opportunities seized. For even a coin weighted toward failure will sometimes land on success. Or as the IBM pioneer Thomas Watson said, ‘If you want to succeed, double your failure rate.’”[4]
There are many ways to estimate tail thickness based on sample data.
Calculate kurtosis – You can compute the sample's kurtosis, which is a measure of the "tailedness" of the distribution. Kurtosis is also known as the fourth moment (the first moment is the mean, the second moment is the variance, the third moment is the skewness.)
Evaluate convergence of moments – You can estimate tail thickness by evaluating the convergence of moments. For thin tailed distributions, the mean and higher moments will converge according to the Central Limit Theorem and the law of large numbers. However, with fat tails the convergence is considerably slower, if it converges at all, and requires much more data. To illustrate, in over 50 years of S&P 500 performance, a single day represents over 80% of the kurtosis. This extreme sample dependence suggests that the 4th moment of the S&P 500 does not have stability, and we can rule out thin-tailedness for this distribution.[7, 8]
Estimate the tail index (Plot the data on a log-log plot) – Pareto and other power law distributions have the form F(X) = k * X^n. Taking the log of both sides, leaves us with log (F(X)) = log(k) + n*log(X). Hence, if the log-log plot of our distribution follows a straight line, we have a power law relationship and a fat tailed distribution. The slope of the line provides an estimate of the tail index.[1]
Evaluate the mean excess above a threshold – The mean excess function gives the expected amount that a random variable will exceed a given threshold, on the condition that the threshold is exceeded. In the insurance industry, the mean excess function is the expected claim size above a threshold, given that the threshold is exceeded. With thin tailed distributions, the mean excess function decreases with increased threshold magnitude. Conversely, with fat tailed distributions, the mean excess function increases as the threshold increases. The magnitude of this increase relates to the degree of thickness.[3]
Apply Extreme Value Theory – Extreme value theory is concerned with the distribution of extreme events from a process. Extreme values are collected either in blocks (e.g., by extracting annual extrema) or by taking all peak values over a threshold. By evaluating the resulting distribution, and the probability of exceeding a threshold value, tail properties can be inferred.
