Description

You will build and maintain scalable data pipelines and AI-powered automation solutions using cloud-native tools.

This role is remote.

Responsibilities

• Design and develop ETL and data ingestion pipelines using Databricks, Apache Spark, and PySpark.
• Implement Data Lake and Medallion Architecture patterns for efficient data storage and retrieval.
• Build and consume REST APIs using FastAPI for data access and integration.
• Manage cloud infrastructure and data services across AWS, Azure, and GCP, including BigQuery, Athena, and Cosmos DB.
• Maintain code quality and deployment workflows using Git and CI/CD practices.

Required Skills

• 8+ years of Data Engineering experience.
• Strong proficiency in Python and SQL.
• Hands-on experience with Databricks, Apache Spark, and PySpark.
• Experience with cloud platforms: AWS, Azure, and GCP.
• Knowledge of BigQuery, AWS Athena, Azure Data Lake, and Cosmos DB.
• Proficiency with Git and CI/CD workflows.

Preferred Skills

• Experience with LangChain, LangGraph, or Agentic AI frameworks.
• Knowledge of cloud cost optimization or FinOps concepts.

Key Skills

Education

Bachelor's degree