Description

Build and optimize large-scale data pipelines using PySpark, SQL, and Databricks.

This role is remote.

Responsibilities

• Build and maintain scalable data pipelines using PySpark and SQL.
• Optimize performance of data workflows and queries for large datasets.
• Ensure data quality, consistency, and reliability across all pipelines.
• Troubleshoot and resolve data-related issues in production environments.
• Collaborate with data analysts and scientists to deliver analytics use cases.

Required Skills

• Strong experience in PySpark and SQL.
• Hands-on expertise in building and optimizing data pipelines.
• Understanding of big data processing and distributed computing.
• Ability to work with large datasets and optimize performance.
• Strong problem-solving and analytical skills.
• B.E., B.Tech, or M.E. in Computer Science, Engineering, or related field.
• 5+ years of relevant experience.

Preferred Skills

• Experience with Databricks for scalable data processing.
• Knowledge of data engineering best practices, including code quality and documentation.

Key Skills

Education

Any Graduate