Description

You will design, develop, and deploy data pipelines and lakes to support large-scale data processing.

Responsibilities

• Build and maintain ETL pipelines using Java and Python to extract, transform, and load data.
• Design and manage data models to ensure efficient storage and retrieval of large datasets.
• Deploy data lakes using AWS native services including S3, Glue, IAM, and Athena.
• Implement Infrastructure as Code using Terraform, Ansible, or CloudFormation.
• Troubleshoot complex data processing issues, performance bottlenecks, and system anomalies.

Required Skills

• 5+ years of experience in data engineering or related roles.
• Proficiency in Java and Python for data pipeline implementation.
• Hands-on experience with AWS services: S3, Glue (Crawler, ETL, Catalog), IAM, and Athena.
• Experience with Infrastructure as Code using Terraform, Ansible, or CloudFormation.
• Knowledge of data pipeline orchestration platforms.
• Experience implementing monitoring, logging, and alerting using Prometheus or Grafana.
• Proficiency in distributed Linux environments.
• Experience with IT Service Management, including Incident and Problem management.
• Bachelor's degree in Computer Science.

Preferred Skills

• Familiarity with data streaming technologies like Kafka or Kinesis.
• Experience with AWS OpenSearch or Splunk.
• Knowledge of container and orchestration technologies.

Key Skills

Education

Bachelor's degree in Computer Science