Description

You will design, build, and maintain data pipelines for large-scale data processing.

Responsibilities

• Design, build, and maintain high-performance ETL/ELT data pipelines using Python and PySpark.
• Develop and manage data processing jobs leveraging PySpark for distributed computing across large-scale datasets.
• Implement DevOps practices and tooling (Docker, Jenkins, Terraform, CloudFormation) for automated deployment and orchestration.
• Collaborate with data scientists and analysts to ensure data quality and consistency for advanced modeling.
• Utilize AWS or other cloud services (S3, Glue, EMR, Snowflake) to architect cloud-based data ecosystems.

Required Skills

• 5+ years of experience in Data Engineering or a related technical field.
• Expert-level proficiency in Python, including Pandas and NumPy.
• Solid hands-on experience with PySpark for scalable data workflows.
• Strong background in DevOps principles and tools for deploying Python applications.
• Experience with cloud platforms, specifically AWS, and services like AWS Glue, S3, and Snowflake.
• Advanced knowledge of SQL for data extraction and performance tuning.
• Familiarity with distributed data processing technologies like Apache Spark and Hadoop.

Key Skills

Education

Any Gradute