Description

You will design and implement enterprise-scale data architectures, focusing on big data deployment and system integration. You own the full lifecycle from data modeling to CI/CD pipeline maintenance.

Responsibilities

• Architect enterprise-wide data initiatives, including migration, transformation, and data lake implementations.
• Design and optimize streaming data pipelines, managing late data, windowing, and data freshness.
• Develop and maintain CI/CD pipelines using GitLab and monitor performance via CloudWatch.
• Debug and resolve complex technical issues within big data environments.
• Communicate technical constraints and benefits to partners and senior management.

Required Skills

• 6-8 years of IT experience in enterprise data architecture and management.
• Proficiency in Java, Scala, and Python.
• Advanced SQL expertise: joins, aggregations, windowing functions, CTEs, and RDBMS schema optimization.
• Experience with Databricks, Structured Streaming, Delta Lake, and Delta Live Tables.
• Hands-on AWS experience with S3, Glue, and Lambda.
• Strong understanding of ETL/ELT processes and Data Migration Service.
• Expertise in Conceptual, Logical, and Physical Data Modeling (Relational and Dimensional).
• Knowledge of Data Lake concepts: time travel, schema evolution, and optimization.
• Familiarity with Schema Registry and message formats like Avro and ORC.

Preferred Skills

• Experience with Great Expectations or similar data quality frameworks.
• Architectural experience specifically within AWS environments.

Key Skills

Education

Bachelor's degree