Description

Design and maintain large-scale data processing pipelines and AWS-based microservices.

This role is remote.

Responsibilities

• Develop REST API and SOAP webservices using Scala, PySpark, and Java.
• Build and optimize batch applications using Spark, Hadoop, and EMR.
• Implement AWS batch and API microservices using EMR, EC2, ECS, S3, Airflow, and Step Functions.
• Write Linux shell scripts and Python for system automation and performance tuning.
• Establish CI/CD pipelines using Maven, Gradle, Jenkins, Artifactory, and CloudFormation.

Required Skills

• 5+ years of experience in Scala, PySpark, and Java.
• Strong proficiency with AWS services: EMR, EC2, ECS, S3, API Gateway.
• Experience with Airflow and Step Functions for workflow orchestration.
• Solid Linux administration and shell scripting skills.
• Familiarity with Git, GitHub, and Bitbucket for version control.
• Experience creating CI/CD pipelines with Maven, Gradle, or Jenkins.

Preferred Skills

• DevOps experience with GitLab, IntelliJ IDEA, PyCharm, and AWS Code Commit.
• Background in batch application performance tuning.

Key Skills

Education

Any Graduate