Description

You will build and maintain scalable data solutions and ETL/ELT pipelines.

Responsibilities

• Develop and optimize ETL/ELT pipelines using PySpark and SQL.
• Write efficient PySpark scripts for data transformation and large-scale processing.
• Design data models, schemas, and partitioning strategies for large datasets.
• Optimize SQL queries, stored procedures, and indexing to enhance performance.
• Implement error handling, logging, and monitoring within data pipelines.

Required Skills

• 5+ years of experience in data engineering or related roles.
• Strong proficiency in Python for data engineering tasks.
• Deep expertise in PySpark for large-scale data processing.
• Advanced SQL skills including joins, window functions, CTEs, and query optimization.
• Experience with cloud platforms such as AWS, Azure, or GCP.
• Hands-on experience with data warehousing tools like Snowflake, Redshift, or BigQuery.
• Knowledge of orchestration tools including Airflow or Apache Oozie.
• Experience with version control using Git and CI/CD pipelines.
• Familiarity with Azure DevOps.

Key Skills

Education

Any Gradute