Description

Lead enterprise data architecture, migrations, and warehouse implementations.

Responsibilities

• Architect enterprise-scale data migrations, transformations, and data mart builds.
• Design and implement complex streaming data pipelines and batch systems.
• Maintain GitLab CI/CD pipelines and monitor systems using CloudWatch.
• Debug and resolve complex technical issues within big data applications.
• Brief stakeholders and senior management on technology solutions and constraints.

Required Skills

• 8+ years of IT experience with a focus on enterprise data architecture and management.
• Expertise in Conceptual, Logical, and Physical Data Modelling, including Relational and Dimensional models.
• Hands-on experience with Databricks, Spark Scala, Structured Streaming, Delta Lake, and Delta Live Tables.
• Advanced SQL skills including windowing functions, CTEs, joins, and RDBMS schema performance optimization.
• Deep understanding of ETL/ELT processes and incremental data load strategies like tumbling/sliding windows and high watermarks.
• Experience managing data lake concepts such as time travel, schema evolution, and optimization.
• Proficiency with AWS services including S3 and Lambda for data processing and configuration.
• Technical skills in Spark, S3, Glue, AWS Redshift, and Python.

Preferred Skills

• Experience with Great Expectations or similar data quality frameworks.
• AWS architecture experience and familiarity with StreamSets.

Key Skills

Education

Any graduate