MReport October 2018

TheMReport — News and strategies for the evolving mortgage marketplace.

Issue link:

Contents of this Issue


Page 40 of 67

TH E M R EP O RT | 39 FEATURE Bringing ML Power to the Field Safely and Effectively O ur goal should be to combine the strengths of ML-only models (discovering subtle predic- tive patterns in the data) with the strength of multi-scorecard models (highly predictive and easy to explain). Our team executes this "best-of-both-worlds" approach using a two-phase development strategy: • Develop the best ML-only model quickly in the lab, using inherent highly automated processes. • Closely approximate that model with a system of segmented scorecards. Human credit scor - ing experts remain in control, imposing constraints on the scorecards to ensure explain- ability and palatability. At FICO, our mission is to innovate new tools that enable lenders to safely expand consumer access to credit and fuel economic growth. Machine learning is an exciting technology that we have used for more than 30 years to enhance FICO ® Scores, and we remain at the forefront of explor- ing how ML can be applied to quantify credit risk and identify key drivers of default, fueling new discoveries in the analytic leap from correlation to causation. But our research also reveals the fallibility and adverse potential of ML-only scoring models: • ML-only models are not a FICO ® Score R&D ML Lift Over Scorecard Approach is Measuarable, But Modest Figure 2: FICO researchers tested the efficacy of the FICO ® Score model and two variations of an ML-only credit scoring model, SGB stands for "Stochastic Gradient Boosting." All models were developed to predict defaults (payments more than 90 days past due) on bankcard accounts using the same training data and their performance was evaluated on an indepen- dent test data set. Figure 3: In testing both a scorecard model and an ML-only model against unscorable consumer credit files with derogatory information, removing palatability constraints from the scorecard model produced predictive performance nearly identical to the ML-only model. cure-all for a lack of data. • ML-only models can produce potentially biased predictions and underestimate default rates in traditionally unscorable populations. • ML has limited predictive upside over a well-constructed system of scorecards. • ML-only models potentially lack transparency and palatability. Unleashing ML-only models into the broad lending market would almost certainly usher in systemic risk, market confu - sion, and lack of transparency for consumers. Thus, innovations in ML must be combined with domain expertise and should be complemented with the provision of relevant new data sources, an approach that continues to drive the proven safety, soundness, and innovation of FICO ® Scores. ETHAN DORNHELM is a VP in the Scores and Predictive Analytics unit at FICO. Dornhelm is currently responsible for the analytic development of FICO® Scores globally, as well as the research and development of new products and services within the Scores organization. He spent much of his 17 years at FICO supporting the technical redevelopment, maintenance, and analytic support of FICO ® Scores in North America. Prior to joining FICO, Dornhelm served as a Director in the Risk Management Group at American Express. He graduated from U.C. San Diego with a B.S. in manage - ment science/operations research. DR. GERALD FAHNER is Senior Principal Scientist in FICO's Scores division. He specializes in innovative algorithms that turn data and domain knowledge into superior insights, predictions, and decisions. Fahner is also responsible for the core algorithms underlying FICO's Scorecard develop - ment platform. He holds patents in the areas of marketing analytics and causal modeling. Prior to joining FICO in 1996, he served as a researcher in artificial intelligence, neural networks, and robotics at the International Computer Science Institute in Berkeley and earned his Computer Science doctorate from the University of Bonn.

Articles in this issue

Archives of this issue

view archives of TheMReport - MReport October 2018