Description

Design and implement enterprise-scale data architectures and large-scale big data applications.

You will lead and architect enterprise-wide initiatives including system integration, data migration, and data warehouse builds.

Responsibilities

• Design and implement data lakes, data marts, and transformation pipelines.
• Manage and maintain GitLab CI/CD pipelines and monitor systems via CloudWatch.
• Debug, troubleshoot, and resolve complex technical issues within data systems.
• Partner with stakeholders to brief on the benefits and constraints of proposed technical solutions.

Required Skills

• 8+ years of IT experience focusing on enterprise data architecture and management.
• Hands-on experience with Databricks, Structured Streaming, Delta Lake, and Delta Live Tables.
• Proficiency in Spark programming using Scala and Java.
• Advanced SQL skills, including joins, aggregations, windowing functions, CTEs, and RDBMS schema optimization.
• Expertise in Conceptual, Logical, and Physical Data Modeling, including Relational and Dimensional modeling.
• Deep understanding of ETL/ELT processes, incremental data loads (tumbling/sliding windows, high watermark), and schema evolution.
• Experience with indexing, partitioning strategies, and large-scale big data application deployment.
• Knowledge of AWS services including S3 and Lambda for data processing and configuration.
• Familiarity with GitLab, CloudWatch, Schema Registry, and message formats like Avro and ORC.

Preferred Skills

• Experience with Great Expectations or similar data quality validation frameworks.
• Architecture experience within an AWS environment.

Key Skills

Education

Not specified