Description

Job Description

We are looking for an experienced AWS Glue Developer + Databricks Data Engineer with 6–8 years of experience in designing, developing, and maintaining scalable data engineering solutions. The ideal candidate should have strong hands-on expertise in AWS Glue, Databricks, PySpark, SQL, and AWS data services.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using AWS Glue and Databricks.
• Develop and optimize data processing solutions using PySpark and Python.
• Work with AWS services such as S3, Glue, Lambda, Redshift, Athena, and CloudWatch.
• Build data ingestion and transformation pipelines for structured and semi-structured data.
• Develop and optimize Databricks notebooks, jobs, workflows, and Delta Lake solutions.
• Write complex and optimized SQL queries for data transformation and analysis.
• Implement data quality checks, validation, error handling, and monitoring across pipelines.
• Optimize Spark jobs for performance, scalability, and cost efficiency.
• Collaborate with data architects, analysts, developers, and business stakeholders to understand data requirements.
• Troubleshoot production issues and ensure timely resolution.
• Follow best practices for data security, governance, version control, and deployment.

Required Skills

• 6–8 years of experience in Data Engineering.
• Strong hands-on experience with AWS Glue.
• Strong experience with Databricks and Apache Spark.
• Excellent knowledge of PySpark and Python.
• Strong SQL development and query optimization skills.
• Good understanding of AWS S3, Redshift, Athena, Lambda, and CloudWatch.
• Experience building end-to-end ETL/ELT pipelines.
• Experience working with Delta Lake and large-scale data processing.
• Good understanding of data warehousing and data lake concepts.
• Experience with Git and CI/CD is preferred.
• Strong analytical, troubleshooting, and communication skills.

Preferred Qualifications

• Experience with AWS Glue Data Catalog and Crawlers.
• Experience implementing incremental data loads and partitioning strategies.
• Knowledge of data modeling and dimensional modeling.
• Exposure to AWS IAM, security, and data governance.
• Experience working in Agile/Scrum environments

Key Skills

Education

Any Graduate