Description

You will design and engineer cloud-based big data solutions, building a modern analytics lake and maintaining batch and stream processing pipelines.

Responsibilities

• Develop and maintain data pipelines using modern cloud or open-source ETL/ELT tools.
• Lead the design of distributed, high-throughput, low-latency data processing systems.
• Gather requirements, identify data sources, and resolve data quality issues in collaboration with business and technical teams.
• Implement CI/CD and DevOps practices, including unit testing and UAT support.
• Create data views and APIs from big data stores to feed analysis and visualization engines.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in Python, PySpark, and SQL.
• Hands-on experience with Azure Data Factory, Databricks, and Azure Functions.
• Strong understanding of Data Lake architecture and data pipeline development.
• Experience implementing data governance, lineage metrics, and security standards.
• Ability to build monitoring tools for server-side components.
• Experience working in distributed, India-wide teams.

Preferred Skills

• Experience building data marts, warehouses, and ad-hoc data exploration tools.
• Familiarity with data discovery and consumption models.

Key Skills

Education

Any Graduate