DATA INGESTION IN FINANCE

By : BBInsight

OVERVIEW

A consumer credit reporting agency expanded globally and needs a modern, performant platform for ingesting new customer data. BBI is building automated provisioning for the client’s existing Ab Initio setup on AWS. This new framework will ensure that the organization is cloud-ready for future development, and able to scale up quickly as they continue to expand.
Machine_Learning_Medium_new

CHALLENGE

Our client is a leading consumer credit reporting agency undertaking a global expansion. They need an efficient, seamless data acquisition and ingestion system for the new customer data. This data comes from multiple sources (finance telecom, retail, etc.) and is critical, as it feeds into analytics products that provide customer insights. The existing system had several challenges, including:

01

Outdated Technologies

02

Lack of agility to scale server size and disk space

03

Slow time to market for innovations

SOLUTION

BBI's project team took ownership of new development to ensure the solution is cloud-ready. We began by creating AWS resources and installing Ab Initio products manually. We then created two frameworks:

  • A Terraform framework to facilitate the automated setup of AWS infrastructure.
  • An Ansible framework for automated setup of Ab Initio products in the cloud. Cloudplatform agnostic.

We also implemented a Lambda function to generate alert when the Amazon Machine Image (AMI) is about to expire to support new EC2 creation. The overall solution was integrated with key cloud services to manage security, log monitoring, data storage and more. With this foundation, we are further building out the setup to include EKS containerization and craft a deployment process.

OUTCOME

01
Alignment with delivery of business outcomes.
02
Automated AWS infrastructure setup and Ab Initio products installation.
03
Ability to scale out and in quickly, and as needed.
03
Documented standards that can be used across client regions to support migration to the cloud.
 
Interested in learning more about our services? Let’s Talk.