Description

You will design and implement enterprise-scale data architectures and streaming pipelines within an AWS environment.

Responsibilities

• Architect and lead enterprise-wide initiatives including data migration, system integration, and data lake implementations.
• Design and optimize complex ETL/ELT processes and dimensional data models.
• Build and maintain streaming data pipelines using structured streaming and windowing strategies.
• Debug and troubleshoot technical issues within large-scale big data application deployments.
• Brief stakeholders and senior management on the benefits and constraints of proposed technology solutions.

Required Skills

• 5+ years of IT experience in enterprise data architecture and management.
• Proficiency in Java, Scala, and Python.
• Advanced SQL skills including windowing functions, CTEs, and RDBMS schema optimization.
• Hands-on experience with Databricks, Spark Scala, and Delta Lake concepts.
• Expertise in AWS services including S3, Glue, and Lambda.
• Strong understanding of relational and dimensional data modeling.
• Experience with Gitlab for CI/CD pipeline maintenance and CloudWatch.
• Knowledge of streaming concepts such as tumbling/sliding windows and high watermarks.
• Familiarity with Schema Registry and message formats like Avro or ORC.

Preferred Skills

• Experience with Great Expectations or similar data quality frameworks.
• Direct experience with Delta Live Tables and AWS Data Migration Service.

Key Skills

Education

Any gradudate