Description

You will design, build, and maintain scalable data pipelines and processing systems.

This role is remote.

Responsibilities

• Design and maintain scalable data pipelines using Spark, PySpark, and Databricks.
• Develop ETL/ELT workflows and scripts using Python and SQL.
• Implement DataOps practices to streamline development and deployment.
• Ensure data quality through rigorous testing, validation, and monitoring.
• Collaborate with Data Scientists and Analysts to deliver data solutions.

Required Skills

• 5+ years of experience in a Data Engineering role.
• Strong expertise in Apache Spark and PySpark.
• Solid proficiency in Python for data processing.
• Advanced SQL skills for complex queries and optimization.
• Practical knowledge of DataOps principles and automation.
• Experience with version control (Git) and CI/CD pipelines.
• Familiarity with cloud platforms (AWS, Azure, or GCP).

Preferred Skills

• Experience with Databricks ecosystem.
• Background in ETL/ELT concepts.

Key Skills

Education

Any Graduate