Data Engineer

Nambiti Technologies

We are sourcing for a Data Engineer role.

Contribute to the establishment and operation of the AI Centre of Excellence (AI CoE)

Collaborate with executives to develop strategies that align with organisational imperatives.
Partner with AI Champions, ICT, Data Scientists and AI/ML Engineers to integrate AI into operational workflows smoothly.

Build scalable batch and streaming pipelines from internal and external sources (APIs, CDC, files, events).
Implement schema evolution, late-arriving data handling, and idempotent ingestion.
Set up connectors and jobs across cloud services, message queues (Kafka/Kinesis/Event Hubs), and integration tools.

Design and maintain curated datasets (bronze/silver/gold) using ELT/ETL.
Implement dimensional models, data vault, and/or lakehouse patterns as appropriate.
Use transformation frameworks (e.g., dbt, Spark) with robust testing and documentation.
Data Platform Engineering
Build and operate data lakes / warehouses / lake houses (e.g., Delta Lake, BigQuery, Synapse).
Manage storage formats and optimizations (Parquet/ORC, partitioning, compaction, etc).
Implement cataloguing, lineage, and discovery (e.g., Unity Catalog, Purview, Data Catalogue).

Reliability, Observability & Quality

Implement data testing (unit, expectations), monitoring (freshness, volume, schema, distribution), and alerting (e.g., Great Expectations, Soda, Monte Carlo).
Establish SLAs/SLOs and incident response for data pipelines.
Ensure reproducibility, versioning, and backfills with orchestration (e.g., Airflow, Dagster, Prefect).

Security, Governance & Compliance

Apply data access controls (RBAC/ABAC), masking, tokenization, encryption at rest/in transit.
Enforce data retention, lineage, and regulatory compliance (POPIA or industry-specific).
Collaborate with Data Governance and Security teams on policies and audits.

Monitor compute/storage spend and optimize partitioning, caching, spot/preemptible usage.
Right-size warehouses/clusters and tune jobs for performance and cost efficiency.

Partner with Data Scientists/Analysts on feature engineering, model-ready tables, and MLOps handoffs.
Create self-service patterns, templates, and documentation to accelerate downstream teams.
Contribute to platform standards, reusable components, and developer experience.
Provide mentorship to emerging data engineers

Education, Skills and Experience

A diploma or bachelor’s degree in computer science, Information Systems, Information Technology, Engineering, or a related field (or equivalent experience) is required.
6+ years in Data Engineering or similar roles (Software Engineer with data focus, ETL Developer).
Experience with Lakehouse platforms (Databricks, Snowflake, BigQuery, Synapse/OneLake).
Hands-on with Terraform/Bicep/CloudFormation for infrastructure-as-code.
Familiarity with MLOps (feature stores, model registries, batch/stream scoring integration).
Proven experience building and operating production-grade data pipelines