Description

You will design and implement enterprise-scale data architectures, managing large-scale big data application deployments and complex system integrations.

Responsibilities

• Architect and lead enterprise-wide initiatives including data migration, transformation, and data lake implementations.
• Design and implement streaming data pipelines, managing windowing definitions, late data handling, and data freshness.
• Develop and optimize ETL/ELT processes, data warehouses, and data marts.
• Debug, troubleshoot, and resolve complex technical issues within big data environments.
• Communicate the benefits and constraints of technical solutions to stakeholders and senior management.

Required Skills

• 8+ years of IT experience focusing on enterprise data architecture and management.
• Expertise in Conceptual, Logical, and Physical Data Modeling, including Relational and Dimensional modeling.
• Proficiency in Spark Scala and Java programming.
• Deep knowledge of Databricks, Structured Streaming, Delta Lake, and Delta Live Tables.
• Advanced SQL skills, including joins, aggregations, windowing functions, CTEs, and RDBMS schema optimization.
• Experience with Data Lake concepts such as time travel, schema evolution, and optimization.
• Hands-on experience with S3, AWS Lambda, and CI/CD pipelines using GitLab and CloudWatch.
• Strong understanding of indexing, partitioning strategies, and incremental data loads like tumbling and sliding windows.
• Experience with Schema Registry and message formats such as Avro or ORC.

Preferred Skills

• Experience with Great Expectations or other data quality frameworks.
• AWS architecture experience and familiarity with AWS tools for massive-scale data processing.

Key Skills

Education

Any Graduate