Description

Designs, develops, and maintains data architecture, data pipelines, and data warehousing systems. Implements ETL processes, data modeling, and database design. Collaborates with cross-functional teams to ensure data accuracy and availability.

·       Modernize legacy ETL pipelines: Lead the transformation of SSIS/SSRS workloads into modular, high-performance pipelines using Databricks, dbt, Fivetran, and Airflow.

·       Architect reusable data design patterns: Define and implement standardized frameworks for ingestion, transformation, curation, and consumption layers across the Lakehouse.

·       Develop and lead POCs/POVs: Experiment with new technologies (e.g., Delta Live Tables, Iceberg, streaming ingestion, AI-driven observability) to validate architecture choices and influence the enterprise roadmap.

·       Leverage AI to accelerate engineering: Use AI-enabled tools like Databricks Assistant, Cursor AI, GitHub Copilot, and dbt Mesh AI tests for code generation, automated testing, documentation, and pipeline optimization.

·       Apply ML for operational intelligence: Integrate predictive models to detect pipeline anomalies, data drift, and optimize compute and scheduling.

·       Enforce engineering excellence: Drive CI/CD, version control, peer reviews, and observability practices across the data platform.

·       Collaborate cross-functionally: Partner with data architects, platform enginee analysts, and business product owners to translate business needs into technical solutions.

·       Mentor data engineers: Provide technical guidance, foster continuous learning, and help the team adopt modern data engineering best practices.

·       Optimize performance and cost: Continuously tune Spark workloads, storage tiers, and orchestration logic across Azure and GCP environments

Qualifications

·       8+ years of experience in data engineering or related technical fields, with at least 3+ years in a lead or senior role.

·       Proven experience designing and implementing data design patterns (e.g., CDC, SCD, Medallion, Data Vault, streaming, and batch patterns).

·       Deep expertise with Databricks, Apache Spark, dbt, Fivetran, Census, Airflow, and Kafka. Solid experience across Azure and/or GCP (e.g., Synapse, Data Factory, BigQuery, Pub/Sub).

·       Hands-on experience modernizing legacy ETL (SSIS/SSRS) workloads into cloud-native pipelines.

·       Demonstrated ability to build POCs and POVs that validate new tools, frameworks, or architectures.

·       Working knowledge of AI-assisted engineering tools for development, observability, or optimization.

·       Proficiency in SQL and one programming language (Python, Scala, or Java). Strong problem-solving, architectural thinking, and collaboration skills. Excellent communicator with the ability to translate technical topics to business stakeholder

Key Skills

Education

Any Graduate