Intermediate Data Engineer

SQL

CI/CD pipelines

Hadoop

SQL MS

Data engineering

Data warehousing

Python, Java, or Scala

Analytical

Machine Learning

Problem Solving

Responsibilities

Data Platform Engineering:

• Build, maintain, and improve batch and near-real-time data pipelines that support analytics and operational use cases.
• Contribute to ingestion, transformation, storage, and serving layers within the enterprise data platform.
• Develop reliable and maintainable data processing logic using SQL, Python, and related tooling.
• Help modernise legacy data workflows into more scalable and supportable patterns.

Lakehouse Architecture & Scalable Processing

• Contribute to the implementation and improvement of data lake or lakehouse components.
• Support the preparation of data for efficient analytical use across warehouse and large-scale storage environments.
• Assist with data layout, partitioning, and optimisation practices that improve platform performance and maintainability.
• Work with senior engineers to apply platform standards and architectural patterns consistently.

Data Quality, Reliability & Governance

• Build and maintain data quality checks, reconciliation logic, and pipeline validation routines.
• Help ensure datasets meet expectations for freshness, completeness, and accuracy.
• Support metadata, lineage, and documentation practices that improve trust and transparency.
• Troubleshoot pipeline failures, data discrepancies, and performance issues in production.

Collaboration & Delivery

• Partner with BI, analytics, software engineering, product, and business stakeholders to understand and support data use cases.
• Translate business and platform requirements into scalable, well-engineered technical solutions.
• Participate in planning, estimation, implementation, testing, and deployment activities.
• Contribute to code reviews, documentation, and shared engineering standards.

Continuous Improvement & Innovation

• Identify opportunities to improve automation, reliability, scalability, and maintainability across the platform.
• Stay current with emerging data engineering practices, tools, and patterns.
• Grow technical capability in modern data platform engineering through hands-on project delivery.

Tech Environment

The platform may include a combination of established and modern technologies such as: SQL Server, Python, Spark, Flink, Airflow, Object storage, Open format Tables, Kafka / Redpanda, ClickHouse or similar columnar analytical stores, Git and CI/CD tooling, Kubernetes, OpenShift, or similar runtime environments.

Qualifications

• Degree or diploma in IT, Computer Science, Engineering, or a related technical discipline.
• 2 to 4 years of experience in data engineering, ETL/ELT development, or data platform engineering.
• Hands-on SQL experience, including query optimisation, indexing, and performance tuning.
• Experience building and maintaining batch data pipelines in production environments.
• Exposure to data platform concepts such as data warehouse, data lake architecture, or object storage.
• Experience with orchestration platforms such as Airflow, Dagster, or similar.

Living the Spirit

• Engages in cross-functional collaboration and problem solving while contributing to an inclusive team culture.
• Supports a culture of adaptability and shared accountability across the department and wider business.
• Shows up authentically and contributes to team success by working effectively with diverse colleagues and perspectives.
• Approaches challenges as opportunities to learn, improve, and help others grow.