Description

Design and implement large-scale enterprise data architectures, data lakes, and streaming pipelines using AWS and Java.

Responsibilities

• Architect enterprise-wide data initiatives, including migration, system integration, and data lake implementations.
• Design and build data warehouses, data marts, and streaming data pipelines.
• Implement ETL/ELT processes and manage incremental data loads using windowing strategies.
• Debug, troubleshoot, and resolve complex technical issues within big data deployments.

Required Skills

• 8+ years of IT experience focusing on enterprise data architecture and management.
• Proficiency in Java and Scala programming.
• Extensive experience with AWS services including S3, Glue, Redshift, and Lambda.
• Advanced SQL skills including joins, aggregations, windowing functions, and CTEs.
• Expertise in Conceptual, Logical, and Physical Data Modeling, including Relational and Dimensional modeling.
• Hands-on experience with Databricks, Structured Streaming, Delta Lake, and Delta Live Tables.
• Deep understanding of Data Lake concepts such as time travel, schema evolution, and optimization.
• Experience with Gitlab and CloudWatch for supporting CI/CD pipelines.
• Knowledge of indexing, partitioning strategies, and RDBMS schema design optimization.

Preferred Skills

• Familiarity with Great Expectations or similar data quality validation frameworks.
• Experience with Schema Registry and message formats like Avro or ORC.

Key Skills

Education

Any Graduate