MReport May 2021

TheMReport — News and strategies for the evolving mortgage marketplace.

Issue link:

Contents of this Issue


Page 23 of 67

22 | M R EP O RT FEATURE curacy. The importance of each document and data element in a loan file varies across the loan lifecycle. The thresholds in new loan production help to ensure the manufacture of a quality loan. However, the same thresholds might not be applicable when sell- ing the asset or servicing rights. It would be expensive and time-consuming to review every document classification and data extraction again and again as a loan moves through production and due diligence reviews. Market participants rely on each other to implement the appropriate loan quality checks in each stage of the loan lifecycle, which in many instances is a regulatory require- ment. However, we all know mis- takes do happen. This is where machine learning and confidence scoring can more efficiently enable the mortgage industry to manage loan quality holistically with less manual intervention. While machine learning can automate all classification and data extraction at each stage, confidence scoring can enable each entity in the loan lifecycle to prioritize document and data ac- curacy through confidence scoring thresholds. The scores themselves can be used to help lenders control exceptions by dialing thresholds up or down, based on their specific requirements. In particular, confidence scores can really help lenders gauge the quality of critical documents, such as loan applications or disclosures. By applying a higher confidence threshold to these documents, lenders can achieve greater safe- guards for their accuracy. The Value Behind the Scores B y combining confidence scores with automated document classification and data extraction tools, lenders are better able to focus their resources on documents and data that demand greater scrutiny. This gets to the heart of value behind confidence scoring—it allows lenders to better control costs by minimizing the amount of human intervention. For instance, a lender using ADR with confidence scoring may decide that all closing disclosure documents must have a score of 90% or higher in order to "pass," and that those documents do not need to be reviewed by a staff person. Basically, that's money saved. Meanwhile, only the disclo- sures that fall below that threshold would trigger a review by a human processer or closer to confirm the document classification. In the production space, con- fidence thresholds might be set high for key documents contain- ing data used to underwrite the loan. Often these are first-genera- tion documents, and the machine learning classification confidence is quite high, accurately classify- ing and extracting data with no human intervention. In loan acquisition, whether acquiring closed loans or mort- gage servicing rights (MSRs), loan file documents may be second or third generation in nature and much lower quality. This increases the likelihood that some critical documents evaluated in pre-purchase reviews may be clas- sified but with low confidence. Setting a confidence threshold to "always fail" critical documents can ensure human review and greater transparency, which is important for due diligence. Beware of Wild Claims T he proliferation of automated technology offerings in the mortgage industry has yielded a lot of unfounded claims by differ- ent providers. Many companies, for example, are fond of making blanket statements that their tech- nologies deliver 95% (or whatever percent) accuracy. But what does this really mean? Does that per- centage apply to all the different documents and document types that lenders encounter? Does it apply to all data fields, regard- less of which documents they are found in? To what depth and to what breadth of mortgage life cycle use cases does this apply? These grandiose claims over accuracy have proliferated for several years now. It's easy to make such statements. But when you start peeling back the onion, you'll find that most often ac- curacy only applies to a small set of common loan documents, not the vast library of documents in use today, and it becomes a very diminishing statement. As already mentioned, the "accuracy" of document processing technol- ogy is also highly dependent on the quality of the documents being processed—first, second, or third generation—which varies by channel. Yet another consideration is what fail-safe measures—if any— have been put in place for han- dling exception scenarios. In other words, if a document cannot be classified, what method exists to review and update them? What is needed is some combination of exception management built into the technology that can comple- ment human workflow to achieve the degree of accuracy required. This is where the flexibility behind confidence scoring can truly make an impact. The abil- ity to adjust confidence scoring thresholds for different docu- ment types and data points gives lenders the ability to reduce the "noise" created by differ- ent scenarios while maintaining high productivity and deploying staff resources in a more efficient manner. The possibilities with confi- dence scoring are endless, but it's important to remember nothing is 100% correct 100% of the time. Like any technology, confidence scoring is simply a tool that en- ables better decision making. But when it comes to saving time and money, it's a tool that can deliver high returns. . TERRELL C. CASSADA is the Chief Product Architecture & Innovation Officer at LoanLogics. For instance, a lender using ADR with confidence scoring may decide that all closing disclosure documents must have a score of 90% or higher in order to "pass," and that those documents do not need to be reviewed by a staff person. Basically, that's money saved.

Articles in this issue

Archives of this issue

view archives of TheMReport - MReport May 2021