ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo

Tim Bates Mba Technology Conference 2007 Data Mining Presentation

?
?
T
timbates2

Presentation given at the 2007 Mortgage Bankers Association Technology Conference- topic was the Operational Process of Data Mining.

1 of 16
Download to read offline
The Operational Process of Effective Data Mining Tim Bates Washington Mutual Corporate Credit Risk Management 1
Overview ? Defining and measuring the effectiveness of a data mining function ? The concept of the Decision Management Life Cycle in data mining ? Execution as a competitive advantage 2
How Do You Measure The Effectiveness of Data Mining? What are we typically measuring? ? ? %/$ Delinquency ? Net Chargeoff ? ROA/RAROC These are the right things to watch, but do they ? really measure what makes data mining effective? 3
How Do Should Data Mining Effectiveness Be Measured? Effective data mining provides a framework to ? measure and manage risk/reward opportunities In a fact-based environment, this is accomplished ? via execution of a decision management life cycle ? Develop, execute, measure, and analyze ? The more quickly an organization can learn, the more effective its strategy execution will be over time Measured goal becomes the ability to compress ? learning into a smaller time frame 4
Organizations that measure and manage their operational effectiveness and speed of execution in decision management can gain a competitive advantage in the form of advanced strategy execution and speed-to-market. 5
Why Is Decision Management Important? ? Rate of change of the world is increasing ? ¡°Obsolescence of experience¡± ? Product life cycles are shrinking ? Organizations that acknowledge and embrace this as a foundational world view can begin to understand the importance of decision management execution 6
Data Mining and the Feedback Loop Concept ? Decision making (in Develop Analyze any business) consists Strategy of a basic four-step Results process ? Concept of iterative learning: in order to execute more effective Decision Infrastructure decision making, full ? Systems execution of the loop is ? Attributes ? Definitions required ? Processes ? The longer the loop ? Standards takes to execute, the less effective the decision making process Measure Execute Results Strategy 7
Data Mining and the Feedback Loop Concept Develop Strategy ? Starts with hypothesis based on previous experience or data: ? Auto-approve loans with 680+ Fico, Custom Score > 500, < 55% Debt Ratio, LTV < 80%, and Loan Amount < $650k 8
Data Mining and the Feedback Loop Concept ? Codifying strategy/policy for execution ? Identify business rules to execute policy ? Deployment systems: ? Automated Underwriting System/Decision Engine ? Change control/ implementation process: development, test/QA, implementation Execute Strategy 9
Data Mining and the Feedback Loop Concept ? Conventional reporting ? Largely descriptive, tactical ? MIS-based (as opposed to forward- looking) ? Focused on tracking of results as behavior is being observed Measure Results 10
Data Mining and the Feedback Loop Concept ? Beyond measurement of results to interpretation for Analyze re-formulation of credit strategy Results ? Two forms: ? Descriptive analysis of existing scores/ attributes ? Inferential analysis: development of predictive model based on performance results 11
Data Mining and the Feedback Loop Concept ? Successful execution Develop of any single element Analyze Strategy has dependencies Results upon other elements: ? Analysis of results dependent upon effective results measurement and data ? Great strategy development means nothing without equally great execution capabilities ? Weaknesses in one area diminish the effectiveness of the Measure Execute overall system Results Strategy ? System only as strong as its weakest link 12
Keys to Data Mining Execution View execution of data mining/decision strategy as a ? production process ?Identify bottlenecks and add capacity at the bottlenecks ?Focus on efficiencies in the linkages between phases/components This is not the same as speed of execution via blunt ? instruments: Brute Force (e.g. headcount) ? Technology ? $$$ ? 13
The Glue that holds this together is¡­ The pieces that are in the middle. ? Systems ? Attributes ? Definitions ? Processes ? Develop Analyze Strategy Results Standards ? Elements of a common lexicon Decision Infrastructure: ? Systems ? Attributes Key point: By managing these ? ? Definitions ? Processes ? Standards elements and assuring consistency across all the phases, we can streamline the execution of the Measure Execute feedback loop cycle Results Strategy 14
Standards In data mining, where information is currency, rigorous ? standards and definitions are a necessary control point Avoids misleading interpretations and process breakdown ? of the steps supporting strategy development ? Analytic software ? Processes ? Procedures Examples include: ? ? Scoring system monitoring standards ? Reporting methods ? Loss definitions 15
In Summary¡­ Organizations that manage the execution of the ? decision management cycle can achieve sustained competitive advantage through more proactive execution of credit strategy Rather than brute force expenditure on technology or ? headcount, focus on the elements that are common links across all the elements Common elements and consistency throughout the ? cycle minimize translations that slow down execution of the process 16

More Related Content

Tim Bates Mba Technology Conference 2007 Data Mining Presentation

  • 1. The Operational Process of Effective Data Mining Tim Bates Washington Mutual Corporate Credit Risk Management 1
  • 2. Overview ? Defining and measuring the effectiveness of a data mining function ? The concept of the Decision Management Life Cycle in data mining ? Execution as a competitive advantage 2
  • 3. How Do You Measure The Effectiveness of Data Mining? What are we typically measuring? ? ? %/$ Delinquency ? Net Chargeoff ? ROA/RAROC These are the right things to watch, but do they ? really measure what makes data mining effective? 3
  • 4. How Do Should Data Mining Effectiveness Be Measured? Effective data mining provides a framework to ? measure and manage risk/reward opportunities In a fact-based environment, this is accomplished ? via execution of a decision management life cycle ? Develop, execute, measure, and analyze ? The more quickly an organization can learn, the more effective its strategy execution will be over time Measured goal becomes the ability to compress ? learning into a smaller time frame 4
  • 5. Organizations that measure and manage their operational effectiveness and speed of execution in decision management can gain a competitive advantage in the form of advanced strategy execution and speed-to-market. 5
  • 6. Why Is Decision Management Important? ? Rate of change of the world is increasing ? ¡°Obsolescence of experience¡± ? Product life cycles are shrinking ? Organizations that acknowledge and embrace this as a foundational world view can begin to understand the importance of decision management execution 6
  • 7. Data Mining and the Feedback Loop Concept ? Decision making (in Develop Analyze any business) consists Strategy of a basic four-step Results process ? Concept of iterative learning: in order to execute more effective Decision Infrastructure decision making, full ? Systems execution of the loop is ? Attributes ? Definitions required ? Processes ? The longer the loop ? Standards takes to execute, the less effective the decision making process Measure Execute Results Strategy 7
  • 8. Data Mining and the Feedback Loop Concept Develop Strategy ? Starts with hypothesis based on previous experience or data: ? Auto-approve loans with 680+ Fico, Custom Score > 500, < 55% Debt Ratio, LTV < 80%, and Loan Amount < $650k 8
  • 9. Data Mining and the Feedback Loop Concept ? Codifying strategy/policy for execution ? Identify business rules to execute policy ? Deployment systems: ? Automated Underwriting System/Decision Engine ? Change control/ implementation process: development, test/QA, implementation Execute Strategy 9
  • 10. Data Mining and the Feedback Loop Concept ? Conventional reporting ? Largely descriptive, tactical ? MIS-based (as opposed to forward- looking) ? Focused on tracking of results as behavior is being observed Measure Results 10
  • 11. Data Mining and the Feedback Loop Concept ? Beyond measurement of results to interpretation for Analyze re-formulation of credit strategy Results ? Two forms: ? Descriptive analysis of existing scores/ attributes ? Inferential analysis: development of predictive model based on performance results 11
  • 12. Data Mining and the Feedback Loop Concept ? Successful execution Develop of any single element Analyze Strategy has dependencies Results upon other elements: ? Analysis of results dependent upon effective results measurement and data ? Great strategy development means nothing without equally great execution capabilities ? Weaknesses in one area diminish the effectiveness of the Measure Execute overall system Results Strategy ? System only as strong as its weakest link 12
  • 13. Keys to Data Mining Execution View execution of data mining/decision strategy as a ? production process ?Identify bottlenecks and add capacity at the bottlenecks ?Focus on efficiencies in the linkages between phases/components This is not the same as speed of execution via blunt ? instruments: Brute Force (e.g. headcount) ? Technology ? $$$ ? 13
  • 14. The Glue that holds this together is¡­ The pieces that are in the middle. ? Systems ? Attributes ? Definitions ? Processes ? Develop Analyze Strategy Results Standards ? Elements of a common lexicon Decision Infrastructure: ? Systems ? Attributes Key point: By managing these ? ? Definitions ? Processes ? Standards elements and assuring consistency across all the phases, we can streamline the execution of the Measure Execute feedback loop cycle Results Strategy 14
  • 15. Standards In data mining, where information is currency, rigorous ? standards and definitions are a necessary control point Avoids misleading interpretations and process breakdown ? of the steps supporting strategy development ? Analytic software ? Processes ? Procedures Examples include: ? ? Scoring system monitoring standards ? Reporting methods ? Loss definitions 15
  • 16. In Summary¡­ Organizations that manage the execution of the ? decision management cycle can achieve sustained competitive advantage through more proactive execution of credit strategy Rather than brute force expenditure on technology or ? headcount, focus on the elements that are common links across all the elements Common elements and consistency throughout the ? cycle minimize translations that slow down execution of the process 16