Description

You will build and maintain data pipelines and infrastructure.

This role is on-site.

Responsibilities

• Develop and optimize data transformations using Python and PySpark.
• Design and implement data ingestion workflows using Kafka and Airflow.
• Troubleshoot complex data issues across microservices and external systems using logging tools.
• Apply AI Augmented development practices, leveraging Copilot and LLMs throughout the SDLC.
• Perform performance optimization on data pipelines.

Required Skills

• 5+ years of experience in data engineering.
• Expert proficiency in Python and advanced SQL.
• Experience with distributed processing frameworks (Spark, Hadoop).
• Hands-on experience with data storage solutions (MongoDB, Cassandra).
• Familiarity with big data orchestration tools (Databricks, Kafka).
• Experience with CI/CD and Git workflows.
• Experience with monitoring and logging tools like Kibana.

Preferred Skills

• Proficiency in Scala.

Key Skills

Education

Any Gradute