Description

You will design and maintain scalable data pipelines and infrastructure to support real-time analytics and AI/ML initiatives.

This role is hybrid.

Responsibilities

• Build and manage resilient streaming and batch data pipelines for real-time analytics.
• Design and optimize data lakes, warehouses, and marts to meet reporting requirements.
• Implement lake-house architecture and maintain data infrastructure for reliability and security.
• Automate data workflows and perform integrity validations to ensure data accuracy.
• Profile data and apply transformations to prepare raw data for AI/ML and BI consumption.

Required Skills

• 10+ years of experience building enterprise-level data applications on distributed systems.
• Expert proficiency in SQL and Python for data processing and automation.
• Experience building scalable ELT pipelines using DBT, Azure, Airflow, or Fivetran.
• Hands-on experience with streaming platforms such as Kafka or Spark.
• Proficiency with cloud-based data platforms like Snowflake.
• Experience implementing CI/CD pipelines using GitHub Actions, Jenkins, or AWS CodePipeline.
• Experience with containerization using Docker and Kubernetes.

Preferred Skills

• Knowledge of data governance practices including RBAC, RLS, data masking, and lineage.
• Experience implementing observability using Prometheus, Grafana, or AWS CloudWatch.

Key Skills

Education

Any Graduate