際際滷

際際滷Share a Scribd company logo

Model Lookback Presentation

G
Graeme Keith

Lookback analysis provided by Stochastic ApS for oil and gas exploration departments. The company, wells and dates are (obviously) made up.

1 of 11
Downloaded 10 times
Lookback analysis of exploration assessments covering the 鍖ve year period from 1973 to 1977 Prepared for Hubris Enterprises inc. by Graeme Keith, Stochastic ApS
Accumulated sequence plot of success rate - Each point in the 鍖gure corresponds to a sequence of wells. The 鍖rst, leftmost, point corresponds to the 鍖rst well, the second to the 鍖rst two wells, and so on to the last, rightmost, point which shows the full sequence of 75 wells. - For each point, the actual success rate up to that point is plotted together with the expected, P90 and P10 success rate predicted by the pre-drill assessments of the probabilities of success of the wells in the sequence up to that point. - Actual success rate is comfortably (but not suspiciously) within the predicted uncertainty range
Sliding window plot of success rate - Each point in the 鍖gure corresponds to a sequence of 25 wells. The 鍖rst, leftmost, point corresponds to wells 1 to 25 in the sequence, the second to wells 2 to 26. The last, rightmost, point corresponds to wells 51 to 75. - For each window, the actual success rate for the wells in that windown is plotted together with the expected, P90 and P10 success rate predicted by the pre-drill assessments of the probabilities of success of the wells in the window. - The constant sequence length in the sliding window plot gives a clearer, less ambiguous picture of the evolution of the 鍖delity of predictions over time (later points uncontaminated by early statistical aberrations) - Actual success rate is comfortably within the uncertainty range and the uncertainty range corresponds well to the actual sequence variability.
Probability plot for success rate - The full sequence is divided into 鍖ve groups based on assessed pre-drill probability of success: 0 to 20%, 20% to 40%, 40% to 60%, 60% to 80% and 80% to 100% - Actual success rates are shown as columns together with the expected, P90 and P10 success rates predicted by pre-drill assessments of the prospects in the interval. - The plot is designed to reveal polarization - the tendency to overestimate the signi鍖cance of evidence and assess probabilities too high or too low - and the converse baselining - a tendency not to move probabilities far enough on the basis of evidence. - Polarization leads to over-prediction in the lower intervals and under-prediction in the upper intervals. - Baselining leads to under-prediction in the lower intervals and over-prediction in the upper intervals. - The plot is inconclusive. The pattern of deviation is inconsistent. - Actuals are broadly in line with predictions, except for the middle interval. However, the probability that all four intervals lie in the 80% con鍖dence interval is only 41%, so it is not unusual to see an outlier.
Inferred probability bias parameters - Stochastic ApS has developed a methodology that quanti鍖es the amount by which probabilities are systematically over- or under- estimated (optimism, pessimism) and the amount by which probabilities are moved away from or towards a base-rate (polarization, baselining). - These biases are captured in two numbers, the probability bias parameters, that can be used to compare performance across regions and trap types as well as with other companies and industries. - The analysis uses a Bayesian model to mimic the effects of the different biases. The parameters are then inferred from predictions and outcomes using Bayesian inversion. - The Bayesian analysis returns a probability distribution on the parameters revealing not only the most likely combination of biases, but also the uncertainty that necessarily arises from short sequences. - Hubris Industries bias footprint reveals a pronounced tendency to polarize probabilities, although predictions are sound on average.
Visualization of probability biases - Using the probability bias parameters presented in the previous slide, it is possible to reconstruct the relationship of the probabilities as given to the probabilities as they ought to have been. - The polarization can clearly be seen with low probabilities being pushed down from their correct values (red) for low probabilities, and pushed up for high.
Accumulated sequence plot of discovery resources - Each point in the 鍖gure corresponds to a sequence of discoveries. The 鍖rst, leftmost, point corresponds to the 鍖rst discovery, the second to the 鍖rst two discoveries, and so on to the last, rightmost, point which shows the full sequence of 25 discoveries. - For each point, the actual cumulative discovered volume up to that point is plotted together with the expected, P90 and P10 cumulative volume predicted by the pre-drill assessments of the volume distributions of the wells in the sequence up to that point. - Although delivered volumes appear to exceed predictions throughout the sequence, this is largely an artefact of an early successful phase. - The changing sequence length makes it very dif鍖cult to draw robust conclusions regarding the rest of the sequence.
Sliding window plot of discovery resources - Each point in the 鍖gure corresponds to a sequence of 15 discoveries. The 鍖rst, leftmost, point corresponds to wells 1 to 15 in the sequence, the second to wells 2 to 16. The last, rightmost, point corresponds to wells 16 to 25. - For each window, the actual average volume per discovery in that windown is plotted together with the expected, P90 and P10 volume per discovery predicted by the pre-drill assessments of the volume distributions of the wells in the window. - The constant sequence length in the sliding window plot gives a clearer, less ambiguous picture of the evolution of the 鍖delity of predictions over time (later points uncontaminated by early statistical aberrations) - The early success phase is much more clearly seen. Nonetheless, delivered volumes lie comfortably in the uncertainty range throughout the sequence.
Percentile plot - Discoveries are split into ranges according to the distribution percentile corresponding to the discovered volume - The proportion of the total number of discoveries that fall into each group is plotted in the percentile plot. - The distribution of percentiles is uniform so we should expect to see roughly 20% of discoveries falling in each percentile interval - However, there will be a considerable amount of statistical variance around this 20% value, as shown the uncertainty range, which is a function of the number of discoveries. - The plot is designed to reveal over-con鍖dence - a tendency to predict ranges that are too small - and vagueness - a tendency to predict unecessarily large uncertainty ranges. - Over-con鍖dence leads to a higher proportion of discoveries landing in the lowest and highest percentile intervals. - Vagueness leads to a higher proportion of discoveries landing in the middle percentile intervals. - The plot is inconclusive. The pattern of deviation is inconsistent. - Actuals are broadly in line with predictions. The probability that all four intervals lie in the 80% con鍖dence interval is only 41%, so it is not unusual to see this pattern of outliers.
Resource uncertainty bias parameters - Stochastic ApS has developed a methodology that quanti鍖es the amount by which probability distributions are systematically over- or under-estimated (optimism, pessimism) and the amount by which ranges are over- or under-estimated (over-con鍖dence, vagueness). - These biases are captured in two numbers, the volume bias parameters, that can be used to compare performance across regions and trap types as well as with other companies and industries. - The analysis uses a Bayesian model to mimic the effects of the different biases. The parameters are then inferred from predictions and outcomes using Bayesian inversion. - The Bayesian analysis returns a probability distribution on the parameters revealing not only the most likely combination of biases, but also the uncertainty that necessarily arises from short sequences. - Hubris Industries volume bias footprint also reveals a pronounced tendency to over-con鍖dence, although predictions are sound on average.
Empirical distribution plot - An alternative way to illustrate the deviation from uniformity of the prediction of percentiles is by plotting the actual attained percentiles in an ordered list from least to greatest. - Theory predicts that - as always, within a certain range of uncertainty - these percentiles should fall on a straight line according to their uniform distribution. - The empirical distribution of percentiles can also be predicted based on the resource bias parameters presented in the previous slide. - The plot reveals both optimism and pessimism, as well as over-con鍖dence and vagueness. - Optimism leads to percentiles that are consistently under the no-bias curve. Pessimism, conversely, leads to an upward lift on the empirical curve - Over-con鍖dence leads to a steepening of the curve as prospects come in with unexpectedly low and high percentiles. - Conversely vagueness leads to a 鍖attening of the curve, as everything comes in around the middle of the wide ranges - Both the empirical distribution and the predicted distribution based on the bias parameters show a tendency to underestimate uncertainty.

More Related Content

Model Lookback Presentation

  • 1. Lookback analysis of exploration assessments covering the 鍖ve year period from 1973 to 1977 Prepared for Hubris Enterprises inc. by Graeme Keith, Stochastic ApS
  • 2. Accumulated sequence plot of success rate - Each point in the 鍖gure corresponds to a sequence of wells. The 鍖rst, leftmost, point corresponds to the 鍖rst well, the second to the 鍖rst two wells, and so on to the last, rightmost, point which shows the full sequence of 75 wells. - For each point, the actual success rate up to that point is plotted together with the expected, P90 and P10 success rate predicted by the pre-drill assessments of the probabilities of success of the wells in the sequence up to that point. - Actual success rate is comfortably (but not suspiciously) within the predicted uncertainty range
  • 3. Sliding window plot of success rate - Each point in the 鍖gure corresponds to a sequence of 25 wells. The 鍖rst, leftmost, point corresponds to wells 1 to 25 in the sequence, the second to wells 2 to 26. The last, rightmost, point corresponds to wells 51 to 75. - For each window, the actual success rate for the wells in that windown is plotted together with the expected, P90 and P10 success rate predicted by the pre-drill assessments of the probabilities of success of the wells in the window. - The constant sequence length in the sliding window plot gives a clearer, less ambiguous picture of the evolution of the 鍖delity of predictions over time (later points uncontaminated by early statistical aberrations) - Actual success rate is comfortably within the uncertainty range and the uncertainty range corresponds well to the actual sequence variability.
  • 4. Probability plot for success rate - The full sequence is divided into 鍖ve groups based on assessed pre-drill probability of success: 0 to 20%, 20% to 40%, 40% to 60%, 60% to 80% and 80% to 100% - Actual success rates are shown as columns together with the expected, P90 and P10 success rates predicted by pre-drill assessments of the prospects in the interval. - The plot is designed to reveal polarization - the tendency to overestimate the signi鍖cance of evidence and assess probabilities too high or too low - and the converse baselining - a tendency not to move probabilities far enough on the basis of evidence. - Polarization leads to over-prediction in the lower intervals and under-prediction in the upper intervals. - Baselining leads to under-prediction in the lower intervals and over-prediction in the upper intervals. - The plot is inconclusive. The pattern of deviation is inconsistent. - Actuals are broadly in line with predictions, except for the middle interval. However, the probability that all four intervals lie in the 80% con鍖dence interval is only 41%, so it is not unusual to see an outlier.
  • 5. Inferred probability bias parameters - Stochastic ApS has developed a methodology that quanti鍖es the amount by which probabilities are systematically over- or under- estimated (optimism, pessimism) and the amount by which probabilities are moved away from or towards a base-rate (polarization, baselining). - These biases are captured in two numbers, the probability bias parameters, that can be used to compare performance across regions and trap types as well as with other companies and industries. - The analysis uses a Bayesian model to mimic the effects of the different biases. The parameters are then inferred from predictions and outcomes using Bayesian inversion. - The Bayesian analysis returns a probability distribution on the parameters revealing not only the most likely combination of biases, but also the uncertainty that necessarily arises from short sequences. - Hubris Industries bias footprint reveals a pronounced tendency to polarize probabilities, although predictions are sound on average.
  • 6. Visualization of probability biases - Using the probability bias parameters presented in the previous slide, it is possible to reconstruct the relationship of the probabilities as given to the probabilities as they ought to have been. - The polarization can clearly be seen with low probabilities being pushed down from their correct values (red) for low probabilities, and pushed up for high.
  • 7. Accumulated sequence plot of discovery resources - Each point in the 鍖gure corresponds to a sequence of discoveries. The 鍖rst, leftmost, point corresponds to the 鍖rst discovery, the second to the 鍖rst two discoveries, and so on to the last, rightmost, point which shows the full sequence of 25 discoveries. - For each point, the actual cumulative discovered volume up to that point is plotted together with the expected, P90 and P10 cumulative volume predicted by the pre-drill assessments of the volume distributions of the wells in the sequence up to that point. - Although delivered volumes appear to exceed predictions throughout the sequence, this is largely an artefact of an early successful phase. - The changing sequence length makes it very dif鍖cult to draw robust conclusions regarding the rest of the sequence.
  • 8. Sliding window plot of discovery resources - Each point in the 鍖gure corresponds to a sequence of 15 discoveries. The 鍖rst, leftmost, point corresponds to wells 1 to 15 in the sequence, the second to wells 2 to 16. The last, rightmost, point corresponds to wells 16 to 25. - For each window, the actual average volume per discovery in that windown is plotted together with the expected, P90 and P10 volume per discovery predicted by the pre-drill assessments of the volume distributions of the wells in the window. - The constant sequence length in the sliding window plot gives a clearer, less ambiguous picture of the evolution of the 鍖delity of predictions over time (later points uncontaminated by early statistical aberrations) - The early success phase is much more clearly seen. Nonetheless, delivered volumes lie comfortably in the uncertainty range throughout the sequence.
  • 9. Percentile plot - Discoveries are split into ranges according to the distribution percentile corresponding to the discovered volume - The proportion of the total number of discoveries that fall into each group is plotted in the percentile plot. - The distribution of percentiles is uniform so we should expect to see roughly 20% of discoveries falling in each percentile interval - However, there will be a considerable amount of statistical variance around this 20% value, as shown the uncertainty range, which is a function of the number of discoveries. - The plot is designed to reveal over-con鍖dence - a tendency to predict ranges that are too small - and vagueness - a tendency to predict unecessarily large uncertainty ranges. - Over-con鍖dence leads to a higher proportion of discoveries landing in the lowest and highest percentile intervals. - Vagueness leads to a higher proportion of discoveries landing in the middle percentile intervals. - The plot is inconclusive. The pattern of deviation is inconsistent. - Actuals are broadly in line with predictions. The probability that all four intervals lie in the 80% con鍖dence interval is only 41%, so it is not unusual to see this pattern of outliers.
  • 10. Resource uncertainty bias parameters - Stochastic ApS has developed a methodology that quanti鍖es the amount by which probability distributions are systematically over- or under-estimated (optimism, pessimism) and the amount by which ranges are over- or under-estimated (over-con鍖dence, vagueness). - These biases are captured in two numbers, the volume bias parameters, that can be used to compare performance across regions and trap types as well as with other companies and industries. - The analysis uses a Bayesian model to mimic the effects of the different biases. The parameters are then inferred from predictions and outcomes using Bayesian inversion. - The Bayesian analysis returns a probability distribution on the parameters revealing not only the most likely combination of biases, but also the uncertainty that necessarily arises from short sequences. - Hubris Industries volume bias footprint also reveals a pronounced tendency to over-con鍖dence, although predictions are sound on average.
  • 11. Empirical distribution plot - An alternative way to illustrate the deviation from uniformity of the prediction of percentiles is by plotting the actual attained percentiles in an ordered list from least to greatest. - Theory predicts that - as always, within a certain range of uncertainty - these percentiles should fall on a straight line according to their uniform distribution. - The empirical distribution of percentiles can also be predicted based on the resource bias parameters presented in the previous slide. - The plot reveals both optimism and pessimism, as well as over-con鍖dence and vagueness. - Optimism leads to percentiles that are consistently under the no-bias curve. Pessimism, conversely, leads to an upward lift on the empirical curve - Over-con鍖dence leads to a steepening of the curve as prospects come in with unexpectedly low and high percentiles. - Conversely vagueness leads to a 鍖attening of the curve, as everything comes in around the middle of the wide ranges - Both the empirical distribution and the predicted distribution based on the bias parameters show a tendency to underestimate uncertainty.