Description

Key Skills: Pyspark, AWS, SQL, Python, Data Engineer, Databricks

Roles and Responsibilities:

• Design, develop, and maintain scalable data pipelines and cloud-native data platforms on AWS.
• Build and optimize data lakes, data warehouses, and ETL/ELT workflows using PySpark and Databricks.
• Develop data processing solutions using Python, SQL, and cloud-native technologies.
• Integrate data from multiple sources and ensure high data quality, reliability, and performance.
• Leverage AWS services such as S3, Glue, Athena, Redshift, EC2, EKS, and SageMaker for data engineering solutions.
• Build and maintain APIs and microservices to support data integration and analytics.
• Collaborate with cross-functional teams to deliver scalable data and analytics solutions.
• Implement CI/CD pipelines, version control, and containerized deployments using Git, Docker, and Kubernetes.
• Support AI/ML initiatives by developing data pipelines for LLM-based and RAG-powered applications.
• Create technical documentation and troubleshoot production data pipeline issues.

Skills Required:

• Strong experience in AWS Data Engineering.
• Hands-on expertise with PySpark, Python, SQL, and Databricks.
• Experience designing and implementing scalable data pipelines, data lakes, and data warehouses.
• Strong knowledge of AWS services including S3, Glue, Athena, Redshift, EC2, EKS, and SageMaker.
• Experience with databases and data warehouses such as Snowflake, Oracle, Starburst Trino, and MS SQL Server.
• Knowledge of Git, CI/CD pipelines, Docker, and Kubernetes.
• Experience building APIs and microservices.
• Understanding of LLM-powered solutions, Prompt Engineering, and Retrieval-Augmented Generation (RAG).
• Strong analytical, communication, stakeholder management, and problem-solving skills.

Good to Have:

• Experience with Java or Scala.
• Knowledge of Power BI or Tableau.
• Exposure to machine learning and AI-based data platforms.
• AWS certifications (e.g., AWS Certified Data Engineer or AWS Certified Solutions Architect).

Education: Bachelor's Degree in related field

Key Skills

Education

Any Graduate