Description

You will build and manage scalable data infrastructure, focusing on near-real-time data ingestion and synchronization.

This role is remote.

Responsibilities

• Develop services to ingest and synchronize data, ensuring near-real-time updates.
• Design and implement event-driven architectures using AWS EventBridge, Kafka, or SNS/SQS for real-time streaming.
• Design, implement, and maintain data pipelines integrating on-prem and AWS cloud environments.
• Develop efficient Python scripts to process large datasets, including transitioning pipelines to MSSQL server.
• Monitor data workflows, troubleshoot issues, and optimize system performance.

Required Skills

• 5+ years of professional experience in data engineering.
• Proficiency in Python and ETL processes.
• Experience designing with NoSQL databases including MongoDB, Cassandra, and DynamoDB.
• Familiarity with cloud-native development and AWS services (SNS, AWS EventBridge).
• Experience working with Kafka.
• Ability to design and document target data architectures and pipelines.
• Experience with MSSQL.

Key Skills

Education

Any Gradute