Background
Our client is a consumer credit risk advisory firm for customers ranging from top 10 banks to financial technology startups. Their credit experts use data analysis and modeling techniques to deliver superior economic outcomes and sustainable competitive advantages for their clients.
Despite their credit model expertise, our client faced two key problems with distributing the models to customers. First, although they delivered their credit models as a single packaged unit of Python code, it often took up to 6 months for their customers to provision all the components to get the model running and delivering real business value. Some customers even paid for models they were never able to get running. Second, this lengthy delay eroded the model’s accuracy due to data changes over time, reducing the value delivered to their customers.
These problems were leading to unhappy customers and lost sales opportunities. The client reached out to SingleStone to engage in “how might we” deliver models to our customers faster and with a better user experience.
Response
Our team quickly launched into our Discovery phase to dive deep into the problem space and uncover the issues preventing them from achieving their outcomes. We conducted in-depth interviews with leaders and observed data scientists doing their work to understand their current processes and friction points. Next, we identified options for what could be accomplished within the timeframe and budget while remaining focused on the outcomes.
To minimize disruption to the existing data science practices, we focused on model deployment rather than model development. We re-imagined the model delivery process and eliminated the need for our client’s customers to deploy any infrastructure. Trained models can now be immediately integrated into consumer-facing applications, generating instant business value.
Under the covers, our Model-as-a-Service (MaaS) platform operates within a secure and scalable AWS Foundation that we had previously built. Using an account-per-customer tenancy model to support our client’s data security and regulatory requirements, we designed, built, and deployed this solution using services including AWS Amplify with React, API Gateway, AWS Cognito, DynamoDB, ECS Fargate, Amazon S3, and Lambda.
The new Model-as-a-Service (MaaS) platform hosts trained models and makes them available to customers via a web browser and API. Alongside the new platform, we delivered code pipelines using GitHub Actions to automate the customer provisioning, platform infrastructure deployment, and model deployment processes. These and other MLOps capabilities dramatically reduce the time to set up new customers and deploy changes while also improving quality with fast feedback loops.
On top of this rock-solid platform, our design team created a streamlined user experience that makes it intuitive for customers to:
• Perform batch inference via web UI
• Execute multiple batch inference jobs in parallel
• Download batch inference results
• Perform real-time batch inference via an API
Results
With a new MaaS platform, our client has deployed new business services to engage their customers throughout the entire model lifecycle. Instead of thinking about model delivery as an email attachment with a homework assignment, our client now offers a full lifecycle service resulting in happier customers and recurring revenues. With a focus on outcomes, our client has seen trained model delivery time reduced from months to minutes and built a new revenue stream while providing customers with an intuitive and frictionless user experience.