Senior Data Engineer

Senior Data Engineer Role (Python / Spark / Databricks) - Hybrid - Contract - Banking Role - Senior Data Engineer Rate - £850 p/d (Inside IR35) Duration - 6 months with very likely extension Location - Hybrid / Liverpool Street (London) - 3 days per week in a Liverpool Street office About the Role Global financial institution seeking a highly skilled Senior Data Engineer to join a growing data engineering function responsible for delivering critical data platforms and analytics capabilities across the business. This is a genuinely hands-on engineering role, requiring strong expertise in Python, Databricks, Spark/PySpark and SQL. The successful candidate will be actively involved in the design, development, optimisation and support of enterprise-scale data platforms, working alongside engineers, architects and business stakeholders to deliver robust and scalable solutions. The role offers the opportunity to work on complex data challenges within a modern cloud-based environment, leveraging the latest data engineering technologies and best practices. Key Responsibilities Data Engineering & Development Design, develop and maintain scalable data pipelines using Python, Databricks and PySpark Build and optimise ETL/ELT solutions processing large-scale datasets Develop high-quality, reusable code and engineering frameworks Deliver robust, performant and maintainable data solutions Troubleshoot and resolve complex production and performance issues Support the ongoing enhancement and modernisation of enterprise data platformsDatabricks & Data Platform Engineering Develop and support Databricks-based data solutions and Lakehouse architectures Utilise Delta Lake and Spark technologies to deliver scalable data processing capabilities Optimise distributed data processing workloads for performance and reliability Implement data quality controls and monitoring solutions Contribute to platform stability, scalability and operational excellenceSolution Delivery Work closely with business and technology stakeholders to understand requirements and deliver effective solutions Participate in Agile delivery processes including sprint planning, refinement and estimation Support the delivery of strategic data initiatives across multiple business areas Collaborate with architects, engineers and product teams to deliver high-quality outcomes Identify and resolve technical risks and implementation challengesEngineering Best Practice Promote software engineering best practices including testing, code reviews and version control Support CI/CD adoption and deployment automation Contribute to coding standards and engineering governance Drive continuous improvement across development and support processes Mentor junior engineers and contribute to knowledge sharing within the teamRequired Skills & Experience Strong commercial experience as a Data Engineer or Senior Data Engineer Expert-level Python development skills Extensive hands-on experience with Databricks in enterprise environments Strong Apache Spark and PySpark expertise Advanced SQL development and optimisation skills Experience designing and building large-scale ETL and data integration solutions Strong understanding of distributed computing concepts and data processing frameworks Experience working with modern cloud platforms including Azure, AWS or GCP Knowledge of software engineering principles, testing and code quality practices Experience working within Agile development environments Excellent problem-solving and troubleshooting capabilitiesDesirable Experience Experience within investment banking, financial services or other highly regulated environments Exposure to risk, trading, treasury or regulatory data domains Experience with Kafka, streaming technologies or real-time data processing Databricks certifications Experience with Airflow, Databricks Workflows or similar orchestration tools CI/CD pipeline development and DevOps practices Docker and Kubernetes Experience supporting data science or machine learning workloadsThis role will suit a Senior Data Engineer who enjoys remaining close to the technology and has strong recent experience building production-grade solutions using Python, Databricks, Spark/PySpark and SQL in enterprise environments. GCS is acting as an Employment Business in relation to this vacancy