Description

You will build and optimize data platforms on AWS, focusing on data pipelines, infrastructure-as-code, and integration with Snowflake and Databricks.

This role is remote.

Responsibilities

• Develop and enhance data pipelines and ETL processes using CDAP on AWS infrastructure.
• Create new plugins for CDAP using Java and Spark.
• Build data integration flows to migrate large datasets into Snowflake.
• Implement AWS infrastructure-as-code solutions for deployment automation.
• Work with data scientists to optimize workflows and models on Databricks.

Required Skills

• 5+ years of experience in backend development or data engineering.
• Proficiency in Java, Spark, Kafka, SQL, and CDAP.
• Strong programming ability in Python and Java with unit testing.
• Hands-on experience with AWS services including S3, EC2, EKS, and EMR.
• Experience building scalable ETL processes and workflows.
• Expertise with infrastructure-as-code and CI/CD pipelines.
• Bachelor's degree.

Preferred Skills

• Experience with Snowflake, Databricks, GCP, or Azure.
• Knowledge of streaming data architectures.
• Experience with Machine Learning Operations (MLOps).

Key Skills

Education

Bachelor's degree