Description

Key Skills: PySpark, Python, AWS, Azure, Spark, Data Pipelines, ETL/ELT, Data Processing

Roles & Responsibilities:

• Develop and optimize PySpark data processing jobs using Python for batch data pipelines.
• Build and maintain data workflows on AWS for data ingestion and transformation.
• Implement data quality checks and performance tuning for large-scale Spark workloads.
• Design scalable data pipelines for efficient data processing.
• Collaborate with stakeholders to understand data requirements and deliver effective solutions.
• Support deployment, troubleshooting, and operational improvements of data jobs.

Experience Required:

• 6-10 years of relevant experience in data engineering and pipeline development.
• Strong experience with PySpark and Python.
• Hands-on experience with AWS cloud services.
• Experience in building scalable data processing pipelines using Spark.
• Knowledge of ETL/ELT processes and data workflow development.
• Good to have experience with Azure cloud services.

Education: Any Graduation

Key Skills

Education

Any Graduate