Description

You will design and implement enterprise-scale data architectures and streaming pipelines for the Trusted Computing project.

Responsibilities

• Architect and lead enterprise-wide initiatives including system integration, data migration, and data lake implementations.
• Design and implement complex technical solutions for data warehouse and data mart builds.
• Develop and maintain CI/CD pipelines using GitLab to support automated deployments.
• Debug, troubleshoot, and resolve complex technical issues within large-scale big data applications.
• Brief technology partners and senior management on the benefits and constraints of proposed technical solutions.

Required Skills

• 10+ years of IT experience in enterprise data architecture and management.
• Proficiency in Scala and Java programming.
• Advanced SQL expertise including joins, aggregations, windowing functions, and CTEs.
• Extensive experience with Databricks, Structured Streaming, Delta Lake, and Delta Live Tables.
• Expertise in Conceptual, Logical, and Physical Data Modeling using Relational and Dimensional models.
• Deep understanding of ETL/ELT processes and incremental data loading strategies like tumbling and sliding windows.
• Hands-on experience with AWS services including S3, Glue, Redshift, and Lambda.
• Strong knowledge of RDBMS schema design, indexing, and partitioning optimization.
• Experience with Gitlab and CloudWatch for pipeline support and monitoring.

Preferred Skills

• Experience with Great Expectations or similar data quality frameworks.
• Familiarity with Schema Registry and message formats like Avro or ORC.

Key Skills

Education

Any Graduate