Description

You will design and implement large-scale data processing pipelines and streaming architectures.

This role is on-site.

Responsibilities

• Build and maintain data pipelines using Apache Spark, Kafka, and Hadoop.
• Develop streaming applications and manage data workflows within AWS EMR.
• Implement scalable data storage solutions using Hive and NoSQL databases.
• Automate infrastructure and data tasks using UNIX shell and Python scripting.

Required Skills

• 12+ years of experience in big data engineering.
• Proficiency with Apache Spark, Kafka, Hadoop, and Hive.
• Hands-on experience with AWS services including S3, Lambda, EMR, ECS, ECR, Step Functions, SNS, SQS, EC2, CloudFormation, API Gateway, and DynamoDB.
• Strong programming skills in Java, Python, Kotlin, and Scala.
• Experience with NoSQL and RDBMS such as Cassandra and ScyllaDB.
• Knowledge of REST API and SOAP web services.
• Competency with build tools like Maven and Gradle.
• Proficiency with version control using Git, GitHub, Bitbucket, or GitLab.
• Experience using IntelliJ IDEA, PyCharm, or VS Code.

Preferred Skills

• Experience writing and optimizing code in Java, Python, Kotlin, or Scala.

Key Skills

Education

Any Graduate