Description

You will build and manage data pipelines using big data technologies and AWS services.

Responsibilities

• Manage and transform existing ETL logic and pipeline workflows.
• Develop data processing solutions using SparkSQL, Impala, and Hadoop.
• Implement real-time streaming data architectures.
• Execute CI/CD processes using Jenkins, GitHub, and SonarQube.
• Maintain data infrastructure across AWS and on-premise environments.

Required Skills

• 10+ years of experience in data engineering.
• Proficiency in SparkSQL, Impala, Hadoop, and HDFS.
• Hands-on experience with AWS services including Lambda, EC2, S3, Redshift, Glue, Kinesis, EMR, and Aurora RDS.
• Strong command of Python, Java, Oracle SQL, and Unix shell scripting.
• Experience with Kafka, Airflow, and Elastic Search for streaming and orchestration.
• Knowledge of DevOps tools including Jenkins, GitHub, Spinnaker, SonarQube, and AIM.
• Degree in any graduate field.

Key Skills

Education

Any Graduate