Description

You will build and maintain data pipelines for large-scale data processing using Python and AWS services.

This role is on-site.

Responsibilities

• Develop data processing pipelines using Python and Apache Spark.
• Manage data workflows across distributed systems.
• Implement ETL processes and data warehousing solutions.
• Utilize AWS services to manage and process data assets.

Required Skills

• 7+ years of professional experience in data engineering.
• Strong proficiency in Python for data processing.
• Hands-on experience with Apache Spark (PySpark preferred).
• Solid experience with AWS services: S3, Glue, EMR, Redshift, Athena, Lambda.
• Proficiency in SQL and working with relational/non-relational databases.
• Knowledge of data modeling and ETL frameworks.
• Experience with large-scale, distributed data systems.
• Familiarity with Git and CI/CD pipelines.

Preferred Skills

• Experience with Airflow or workflow orchestration tools.
• Knowledge of Kafka, Kinesis, or streaming data platforms.
• Experience with Docker/Kubernetes.

Key Skills

Education

Any Graduate