DataWorks Beyond UC data Goldeneye

DataWorks is the Data & Analytics plat-form for Universal Credit data. It provides analytical capability directly for 300+ users across the data science and analyst community. It further supports 3000+ users with access to UC data.

DWP has a new AWS platform that utilises native AWS big data technologies; EMR & Hbase and streaming data ingest via Kafka. The platform is fully built using infrastructure as code via Terraform and we run an automated deployment pipeline in Concourse CI. Other technologies in use include: Java, EC2, Lamba, Cloud HSMv2 and SageMaker.

DWP require services delivered while working within the Data & Analytics department and with Universal Credit, that include data quality, data governance, data APIs, bringing in new data sets, and the build and deployment of machine learning infrastructure. This work will be a continuation and enhancement of work that is already underway by an existing team that is ramping down, and as such there will be requirements for handover.