At Capital One, machine learning has become an important part of the company’s business, with the financial firm adopting a standardized process for developing models and sponsoring research to help define its strategy.
The McLean, Va.-based financial services company inserts ML in multiple use cases that build on its cloud-based data ecosystem. Support for ML goes all the way up to the C-suite: Rich Fairbank, CEO of Capital One, mentioned ML seven times during an analyst call last year, citing the use of ML to monitor the economic environment. CapitalOne is now pursuing the new practice with ML operations (MLops)essentially, DevOps for ML, to further institutionalize the technology.
Zachary Hanif
“Our AI ML capabilities are absolutely central to how we build our products, and even more than that, they’re very important to the way we actually run our business,” said Zachary Hanif, vice president and head of machine learning models and platforms. at Capital One. “Over several years, we have leveraged machine learning capabilities across the company in various ways.”
These ways include using ML to empower fraud detection, deliver more personalized customer experiences and improve business planning. On the latter, “we’re making sure we have a better understanding of emerging market conditions and our place in the larger economy,” Hanif said.
Machine learning in banking
Capital One is not alone in its pursuit of ML. Big the banks lead the waycreate their own infrastructure to spin up applications.
But smaller financial institutions are also looking to leverage ML by using third-party platforms and services rather than building in-house capabilities.
“I think it’s important for all banks right now,” said Joe Davey, a partner in the technology practice at West Monroe, a consulting firm headquartered in Chicago. “Banks have generally tried to leverage technology to reduce their efficiency ratios,” he said, referring to the ratio of operating expenses to income. “[ML] is just another piece in the automation puzzle.”
Building an ML platform
Capital One’s current ML initiative stems from a decade-long technology transformation — a program that included restructuring its data environment.
The resulting cloud-based platforms — Capital One uses Snowflake’s data warehousing and engineering platform, for example — provides the basic infrastructure on which developers can build and deploy models.
“Infrastructure allows your teams to focus on the problem at hand without thinking about all the necessary components needed to support solving it,” Hanif said. “Developers spend more of their time focusing on the material that matters most to the business problem.”
He said the platform approach also promotes the computer science concept of accessibility, which aims to make data and applications understandable and accessible to users and developers.
“Accessibility is incredibly important,” Hanif said. “If you can’t make a piece of software accessible to your users—meaning they’re able to understand it, can think about how they can apply it, and can see a use for it inside their environment—it has essentially failed to deliver on its promise and potential.”
Platforms become important as organizations seek to expand AI and ML beyond early experiments and pilots. In the pharmaceutical industry, Eli Lilly created an Enterprise Data Program and centralized analytics platforms to help scale AI across the enterprise.
Most companies have two or fewer years of ML experience, but methods like MLOps could help them scale.
Accelerating MLOs: Challenges and Benefits
Scale is a matter of method as well as technology. In that way, MLOps provides an approach to running an enterprise-wide ML program. Hanif said Capital One has “fully adopted MLOps processes” and is among the early adopters, particularly in the financial sector.
“We see MLOps as the foundational framework to be able to create teams for success in machine learning, to deploy their capabilities at scale, and to ensure that we are able to create an end-to-end environment, ” said Hanif. The goal: to provide a consistent environment to design, deploy and manage ML models, iteratively and at scale.
We see MLOps as the foundational framework to be able to set up teams for success in machine learning, to deploy their capabilities at scale, and to ensure that we are able to create an end-to-end environment.
Zachary HanifVice President and Head of Machine Learning Model and Platforms at Capital One
Unmanageable data is a barrier to MLOps and ML at scale. Organizations can have data stored in a number of different places, making it difficult to discover, Hanif said. “The first challenge you always have to engage with is data,” he noted. Another challenge, Hanif said, is establishing an ML workflow that development teams can follow.
Organizations that overcome these obstacles can potentially see a significant increase in ML effectiveness. A white paper from Harvard Business Review Analytic Services, citing data from Cnvrg.io, noted that previous users of MLOps reported as much as a 10-fold increase in productivity and 5 times faster model training.
That report, which Capital One sponsored, also suggested that most ML models exist outside of a structured process to manage them. The report cited IDC research that claims 90% of ML models are not implemented in production.
Such models can actually find application in organizations and influence business decisions. But they are not implemented inside a standard release pipeline with large-scale automated testing and validation monitoring, Hanif noted.
“You have data scientists developing hundreds or thousands of ML models that never really see the light of day,” he said. “They exist in a kind of shadow state.”
That said, Hanif said he believes more companies are now exploring MLOps to create a well-managed framework for the ML lifecycle.
This structured way of managing ML comes as more financial institutions recognize the potential of the technology.
“The banks are starting to understand these use cases better than they did a few years ago,” said West Monroe’s Davey.
He pointed to anomaly detection and credit risk as typical financial services applications, noting that all large banks and many, if not most, medium-sized banks are pursuing these applications. Document processing and onboarding, meanwhile, are new use cases on the operational side, he added.
Investment in learning
Another aspect of Capital One’s ML strategy is sponsored research, the HBR Analytic Services whitepaper provides an example. This report, released in October 2022, built a case for the MLOps practices that Capital One follows: “Companies without mature MLOps programs could find their competitors outperforming them in using ML,” the white paper said.
Capital One, also last year, commissioned a Forrester Research report on ML challenges. For that research project, Forrester surveyed 150 data governance decision makers in North America. The report highlighted anomaly detection as top ML use case and pointed out the importance of cooperation with third parties to advance enterprise ML strategies.
The investment in research informs Capital One’s ML methods and technology platforms.
“We provide and we develop a whole lot of learning to make sure we’re leveraging best practices,” Harif said.
