Description

You will design and implement large-scale enterprise data architectures and big data applications.

Responsibilities

• Architect enterprise-wide initiatives including system integration, data migration, and data warehouse builds.
• Design and implement streaming data pipelines and manage the transition between batch and streaming systems.
• Debug, troubleshoot, and resolve complex technical issues within big data deployments.
• Brief stakeholders and senior management on the benefits and constraints of proposed technology solutions.
• Maintain GitLab CI/CD pipelines and monitor system performance via CloudWatch.

Required Skills

• 6+ years of IT experience focusing on enterprise data architecture and management.
• Proficiency in Java, Scala, and Python.
• Extensive experience with AWS services including S3, Glue, and Lambda.
• Expertise in Databricks, Structured Streaming, Delta Lake, and Delta Live Tables.
• Advanced SQL skills including window functions, CTEs, and RDBMS schema design optimization.
• Deep understanding of Conceptual, Logical, and Physical Data Modeling.
• Experience with Data Lake concepts such as time travel, schema evolution, and partitioning strategies.
• Practical knowledge of ETL/ELT processes and incremental data loads like tumbling and sliding windows.
• Experience with GitLab for CI/CD and CloudWatch for monitoring.
• Familiarity with Schema Registry and message formats such as Avro or ORC.

Preferred Skills

• Experience with Great Expectations or other data quality frameworks.
• Advanced AWS architecture experience.

Key Skills

Education

Any Graduate