Description

You will build and manage cloud data ingestion and processing pipelines to support an enterprise data lake for analytics and business intelligence.

Responsibilities

• Build and manage automated data pipelines to capture IoT, healthcare, system operations, and third-party data.
• Develop cloud-based processing pipelines using Spark, SQL, Python, AWS Lambda, and REST APIs.
• Create data models across different layers to support analytics and business intelligence.
• Collaborate with application developers to prioritize and deliver data solutions.
• Implement verification testing to ensure data accuracy and quality.

Required Skills

• 5+ years of experience as a data engineer.
• Extensive experience with Python, SQL, and Apache Spark/PySpark.
• Hands-on experience with AWS services including S3, IAM, Lambda, Secrets Manager, and Event Bridge.
• Proficiency with Databricks and Snowflake.
• Experience building data integrations using REST APIs, JDBC, and Kafka Streams.
• Competency in deploying cloud infrastructure and pipelines using Terraform.
• Experience building stream processing pipelines with Kafka and Structured Spark.
• Strong understanding of data modeling and modern data architecture.
• Solid foundations in software engineering and cloud infrastructure.

Preferred Skills

• Experience working with IoT and healthcare data.
• Experience working within an onsite/offshore model.

Key Skills

Education

Any Graduate