MReport October 2018

TheMReport — News and strategies for the evolving mortgage marketplace.

Issue link:

Contents of this Issue


Page 37 of 67

36 | TH E M R EP O RT FEATURE E very day, new technol- ogy innovations are hatched in incubators, corporate research labs, universities, and government in- stitutions. But no matter where it originates, successful new technology follows a predictable evolution, migrating over time from nascence to mainstream maturity. Early technologies that are unleashed too quickly for widespread use—such as Google Glass or hoverboards—can have dangerous repercussions. In the world of credit risk assessment, machine learning (ML) is one of these technologies. There's a lot to marvel about ML's effectiveness, but the technology is only as smart as the data it consumes. It is not a replacement for new data sources. In the last decade or so, new players in the credit-scoring market have promoted models built entirely or predominantly with machine learning as "more modern," in - novative, and effective bases for fair, inclusive credit decisions, particularly for underbanked and "unscorable" populations. These assertions are overstatements. Testing shows that no ML tech- nique alone can overcome the fundamental lack of credit data available for these consumers. Furthermore, overreliance on unfettered "ML-only" models can actually obscure risks and shortchange consumers by picking up harmful biases and behaving counterintuitively. Such models could underestimate default risk or deny consumers improvements to their credit scores as they lower their debt. This lack of explain - ability makes "black box," ML-only models difficult to operationalize at any scale and, in turn, unpalat- able to lenders and consumers, particularly those who are inexpli- cably denied credit. Weak analytic accountability thus risks creating market confusion, lender losses, and consumer exploitation. Compare and Contrast: Machine Learning Vs. Standard Models A lthough both the latest machine learning algo- rithms and the FICO ® Score analytic model can produce a credit risk score, their underly- ing technology is very differ- ent. We tested cutting-edge ML-only techniques, including multilayer neural networks and gradient-boosted decision trees, to examine their applicability to credit scoring. A typical ML-only process uses training data containing predictors and the outcomes the model is trying to predict. In this case, the outcome is whether a consumer will miss payments on credit obligations during a time period after the predictors were observed while the predictors are comprised of thousands of credit bureau characteristics developed by FICO over 25 years of develop - ment of the FICO ® Score. Figure 1 (following page) il- lustrates how this data is used to build thousands of "trees" that segment the population during an iterative process leading to better and better predictions; the result- ing trees are combined to produce the output: a machine learning- driven credit score. This can have significant predictive benefits, capturing nuanced patterns in the data au- tomatically and effectively, which other model-building techniques not based on the latest ML devel- opments don't automatically do. But this approach also makes it much more difficult to determine exactly which variables drive particular predictive outcomes and how. Unlike credit score models built solely or predominantly with ML, FICO produces a system of seg- mented scorecard models that are: • Engineerable: Constraints can be applied to test and refine each scorecard to ensure palatability and to overcome potential data weaknesses. • Transparent: How the variables combine with each other to impact the score is very clear and explainable. This system of segmented score - cards allows us to capture nuances in risk patterns and data interre- lationships, typically considered a machine-learning strength; FICO ® Score 9 utilizes 13 different credit risk-scorecards tuned to distinctly different population segments. Designing the Perfect Match An evolution in machine-learning innovations, along with traditional scorecard modeling, is creating a more predictive credit score that aids both lenders and consumers. By Ethan Dornhelm and Dr. Gerald Fahner

Articles in this issue

Archives of this issue

view archives of TheMReport - MReport October 2018