Description

Build and maintain secure, scalable data pipelines on AWS, integrating Generative AI capabilities into ingestion, transformation, and migration workflows.

This role is remote.

Responsibilities

• Develop ingestion and transformation pipelines using Python, SQL, Snowflake, and Redshift.
• Architect and deploy infrastructure as code using Terraform and CloudFormation.
• Implement advanced IAM roles, policies, and security controls for VPC and compute services.
• Build and monitor LLM data pipelines using Snowflake Cortex AI and observability tools.
• Automate CI/CD processes via GitHub Actions, Jenkins, or AWS CodePipeline.

Required Skills

• 5+ years of hands-on experience in AWS Cloud Engineering.
• Deep proficiency with S3, Lambda, Kinesis, EMR, Step Functions, VPC, and Security Groups.
• Strong background in Data Engineering, specifically building migration and transformation pipelines.
• Expertise in Python and SQL for data manipulation and pipeline logic.
• Advanced knowledge of IAM, including designing roles, policies, and cloud monitoring.
• Practical experience with Snowflake or Redshift for data warehousing.
• Proficiency in Infrastructure as Code tools: Terraform and CloudFormation.
• Experience with CI/CD automation tools such as GitHub Actions, Jenkins, or CodePipeline.

Preferred Skills

• 2+ years working with Generative AI data pipelines and LLM observability (e.g., Arize).
• Experience with structured extraction and evaluation metrics for AI models.

Key Skills

Education

Any Gradute