Description

You will build and maintain data pipelines leveraging AWS services and Python.

Responsibilities

• Implement real-time and batch data ingestion using AWS Lambda and Fargate in Python, managing data flows from various sources.
• Design and develop solutions for data cloud efforts, focusing on scalability, quality, and performance.
• Own data ingestion processes, solution design, use case development, and post-production support.
• Develop and tune SQL queries against data storage platforms like MongoDB and Postgres.
• Work within an Agile development environment to deliver features and support enhancements.

Required Skills

• 5+ years of professional experience in software development.
• Expert proficiency in Python programming.
• Extensive experience with AWS services including S3, RDS, EMR, ECS, and Athena.
• Hands-on experience implementing ETL data movement using Lambda and Fargate.
• Strong understanding of data storage platforms, including MongoDB and Postgres.
• Experience with big data technologies such as HDFS.
• Proficiency in problem-solving and algorithm design relevant to data cloud development.

Key Skills

Education

Any Graduate