Description

You will design, develop, and maintain scalable data pipelines and solutions on the AWS cloud platform.

Responsibilities

• Design, build, and maintain scalable ETL/ELT data pipelines using PySpark, SQL, and AWS Glue.
• Optimize data models, implement partitioning strategies, and perform performance tuning on large datasets.
• Manage and deploy data solutions using AWS services including Glue, EMR, S3, Lambda, Redshift, and Athena.
• Use Terraform, CloudFormation, or CDK for infrastructure-as-code development and management.
• Develop CI/CD pipelines to support automated data engineering workflows.
• Provide technical leadership and mentorship to junior engineers.

Required Skills

• 6+ years of experience in data engineering.
• Proficiency in PySpark and Apache Spark for distributed computing.
• Strong expertise in SQL and Python.
• Experience with AWS Glue and AWS cloud services.
• Hands-on experience with Terraform or CloudFormation.
• Ability to write clean, maintainable code in Shell Scripting.
• Knowledge of IAM, KMS, and Lake Formation for data security and governance.
• Experience with distributed computing concepts and Hadoop.

Preferred Skills

• Experience with Snowflake, Databricks, or Apache Airflow.
• Implementation of real-time streaming using Kinesis, Kafka, or Flink.

Key Skills

Education

Any Graduate