Description

You will design, build, and maintain high-performance ETL/ELT data pipelines.

Responsibilities

• Design, build, and maintain high-performance ETL/ELT data pipelines using Python and PySpark.
• Apply Python libraries (Pandas, NumPy) for complex data manipulation, cleansing, and transformation.
• Develop and manage data processing jobs using PySpark for distributed computing on large datasets.
• Implement DevOps practices (Docker, Jenkins, Terraform, CloudFormation) for automated deployment and orchestration.
• Utilize AWS or other cloud services (S3, Glue, EMR, Snowflake) to architect cloud-based data ecosystems.
• Write and optimize complex SQL queries for data extraction, integrity checks, and performance tuning.

Required Skills

• 5+ years of experience in Data Engineering or a related technical field.
• Expert proficiency in Python, including Pandas and NumPy.
• Solid hands-on experience with PySpark for scalable data workflows.
• Strong background in DevOps principles and tools for deploying Python applications (CI/CD).
• Experience with AWS and associated data services (S3, Glue, Lambda, Snowflake).
• Advanced knowledge of SQL and database performance tuning.
• Familiarity with distributed data processing technologies (Apache Spark, Hadoop).

Key Skills

Education

Any Gradute