Description

Build and maintain secure, scalable data pipelines on AWS using Python and SQL. You will own the full lifecycle of ingestion, transformation, and migration, integrating Generative AI capabilities into your workflows.

Responsibilities

• Develop robust ingestion and transformation pipelines using Python, SQL, Snowflake, and Redshift.
• Architect and deploy infrastructure as code using Terraform and CloudFormation.
• Implement advanced IAM roles, policies, and security controls for VPC and compute services.
• Build and monitor LLM data pipelines using Snowflake Cortex AI and observability tools.
• Automate CI/CD processes via GitHub Actions, Jenkins, or AWS CodePipeline.

Required Skills

• 5+ years of hands-on experience in AWS Cloud Engineering.
• Deep proficiency with S3, Lambda, Kinesis, EMR, Step Functions, VPC, and Security Groups.
• Strong background in Data Engineering, specifically building migration and transformation pipelines.
• Expertise in Python and SQL for data manipulation and pipeline logic.
• Advanced knowledge of IAM, including designing roles, policies, and cloud monitoring.
• Practical experience with Snowflake or Redshift for data warehousing.
• Proficiency in Infrastructure as Code tools: Terraform and CloudFormation.
• Experience with CI/CD automation tools such as GitHub Actions, Jenkins, or CodePipeline.

Preferred Skills

• 2+ years working with Generative AI data pipelines and LLM observability (e.g., Arize).
• Experience with structured extraction and evaluation metrics for AI models.

Key Skills

Education

Any Gradute