TheMReport — News and strategies for the evolving mortgage marketplace.
Issue link: http://digital.themreport.com/i/1367362
20 | M R EP O RT FEATURE O ver the past several months, record refinanc- ing volumes—driven by the lowest mortgage rates in history—have led to an industrywide shortage of underwriters and overworked fulfillment teams, challenges that were exacerbated by the COVID-19 pandemic and the shift to remote work. With rates now rising, staff resources will need to be redistrib- uted to address a growing share in a more labor-intensive purchase and servicing transfer market and cross-trained to enable greater business flexibility. Unfortunately, the hiring frenzy of 2020 may result in a resource surplus for 2021. Now is the time to get serious about right-sizing high-value human skills and achieving scal- ability through automation, maxi- mizing both time and money. Machine learning is a critical technology that is enabling both for the industry. Measuring Confidence W hat tasks can automation perform better than a human? Where is human over- sight needed? How can the two work together to manage risk and optimize expense? In the mort- gage industry, machine learning is successfully being used to achieve the right balance. Some tasks that are always done the same way—especially keystroke- and rules-driven—are ripe for automation. There is no grey area about whether the technology is performing the task correctly. Other tasks that are based on patterns and subject to changing variables are more challenging. This is the space where machine learning has been the breakthrough technology for automation. What does machine learning have that other technologies don't? Confidence. The degree to which the machine thinks it has "gotten it right" is known and quantifi- able. That metric enables busi- nesses to apply human oversight based on the importance and risk associated with the task. But before we get too far, let's review some definitions. First, it bears mentioning that confidence is not the same as ac- curacy. Within the context of loan automation technologies, accuracy refers to how often an automated process is correct in its outcomes. For example, if you run a pool of 100 documents through automated document recognition (ADR) technology, and ADR recognizes 89 documents correctly, that's 89% accuracy. Confidence, on the other hand, is the probability the result provided is in fact correct. In our example, the machine might have been 95% confident it got all 100 documents correctly classified. The 5% margin for error accounts for the misclassification of 11 documents. If getting all 100 documents correctly classified is critical to a business, a human would need to do a quick review to verify each classification. On the other hand, if 95% confidence is good enough, a business can decide to "trust" the technology and eliminate human review all together. The application of a confidence score becomes a useful guide for deter- mining whether the results are likely to be right or wrong, and if that probability is acceptable for a given task. Confidence scores are also helpful when determining if data being extracted from loan docu- ments is correct. Here, algorithms can be applied to derive confi- dence scores of individual char- acters, and subsequently entire words or phrases. For example, if the data field is a street name, such as "Parker," how confident is the machine that each charac- ter is what is reported, and how confident is the machine that the reported value of the entire street name is in fact "Parker"? With this basic understanding of machine learning confidence scoring, let's discuss in a bit more detail how lenders can apply it to their threshold settings to reflect their "comfort level" for data ac- Removing Costs with Confidence Learn how confidence scores can help improve quality and decrease costs. By Terrell C. Cassada