Description

Design, develop, and maintain scalable data pipelines and cloud-native data platforms on AWS.

This role is on-site.

Responsibilities

• Build and optimize data lakes, data warehouses, and ETL/ELT workflows using PySpark and Databricks.
• Develop data processing solutions using Python, SQL, and cloud-native technologies.
• Integrate data from multiple sources and ensure high data quality, reliability, and performance.
• Implement CI/CD pipelines, version control, and containerized deployments using Git, Docker, and Kubernetes.
• Support AI/ML initiatives by developing data pipelines for LLM-based and RAG-powered applications.

Required Skills

• 5+ years of experience in Data Engineering.
• Strong expertise with PySpark, Python, SQL, and Databricks.
• Experience designing and implementing scalable data pipelines, data lakes, and data warehouses.
• Knowledge of AWS services including S3, Glue, Athena, Redshift, EC2, EKS, and SageMaker.
• Experience with databases and data warehouses such as Snowflake, Oracle, Starburst Trino, and MS SQL Server.
• Knowledge of Git, CI/CD pipelines, Docker, and Kubernetes.
• Experience building APIs and microservices.
• Understanding of LLM-powered solutions, Prompt Engineering, and Retrieval-Augmented Generation (RAG).

Preferred Skills

• Experience with Java or Scala.
• Knowledge of Power BI or Tableau.
• AWS certifications (e.g., AWS Certified Data Engineer or AWS Certified Solutions Architect).

Key Skills

Education

Any Graduate