Data Platform Engineer – Cloud

Senior Data Platform Engineer – Cloud Platform Migration Location – Hybrid Leeds of London 3 days Drive a major cloud data modernisation programme, migrating legacy Databricks pipelines and workloads to a modern AWS infrastructure. Working closely with Architects and stakeholders, you will build automated, large-scale data solutions to secure and transform critical healthcare data assets. Requirement: (SC Clearance or eligibility is mandatory). Key Responsibilities Data Engineering & Migration Support migration of data pipelines, notebooks, and data processing workloads between cloud environments. Analyse existing data engineering solutions and identify migration requirements. Develop and maintain data processing pipelines using modern cloud data platform technologies. Support testing, validation, and reconciliation of migrated datasets. Work with structured and semi-structured data stored in cloud environments. Troubleshoot and resolve data processing issues across development, test, and production environments. Cloud Data Platform Engineering Support the operation and enhancement of cloud-based data platforms. Assist with onboarding datasets, users, and workloads onto target platforms. Collaborate with platform and architecture teams to implement scalable data solutions. Support platform optimisation, monitoring, and operational readiness activities. Infrastructure as Code Develop and maintain Terraform code supporting data platform deployments. Support deployment of data platform components, jobs, permissions, and environment configurations. Assist with automation of deployment and release processes. Maintain infrastructure consistency across environments. Platform Modernisation Support cloud migration initiatives and platform transformation programmes. Contribute to platform standards, engineering practices, and reusable deployment patterns. Produce technical documentation and implementation guides. Essential Skills Data Engineering * Python * PySpark * SQL * Data Pipeline Development * ETL / ELT Design * Data Modelling fundamentals * Data Quality and Validation Cloud Platforms * AWS * S3 * IAM * Cloud-based Data Platforms Infrastructure as Code * Terraform * Infrastructure Automation * Git Engineering Practices * CI/CD concepts * Agile Delivery * DevOps principles Desirable Skills * Databricks * Palantir Foundry * Foundry Pipeline Builder * Ontology concepts * Delta Lake * Healthcare or NHS data experience * Secure Data Environment (SDE) experience * Cloud migration programmes Experience 5+ years of Data Engineering experience. Experience delivering cloud-based data platforms. Experience supporting migration of data workloads between environments. Experience working with Infrastructure as Code using Terraform. Experience working within regulated or enterprise-scale environments. Role Senior Data Platform Engineer – Cloud Platform Migration Type – Hybrid Leeds or London – 3 days a week £70,000 to £80,000 ++ depending on experience Candidates must be SC Clearance or have SC eligibility is mandatory SC eligibility - Security Check (SC) clearance in the UK, candidates cannot apply independently; the process must be sponsored by an employer or government department. You must be aa UK citizen or have the right to work in the UK and have resided in the UK for at least 5 years