Description

Key Skills: AWS, Data Bricks, ETL

Roles and Responsibilities:

• Develop and maintain ETL pipelines and batch data workflows using Databricks notebooks on AWS.
• Build data processing jobs using PySpark and Spark SQL, including joins, aggregations, filtering, and validation.
• Read from files and write to Databricks Delta tables (and vice versa), applying Delta Lake versioning and time travel.
• Use managed and external tables in the Databricks workspace and implement reusable notebooks and utility components.
• Ingest and process data using AWS services such as S3, AWS Glue, AWS Lambda, and Amazon Redshift; troubleshoot failures and performance issues under guidance.

Skills Required:

• 5 - 11 years of relevant experience in data engineering and ETL development
• AWS and hands-on Databricks on AWS for building data workflows.
• PySpark and Spark SQL for batch processing and transformations.
• Delta Lake fundamentals including managed vs. external tables, versioning, and time travel.
• ETL fundamentals and AWS data services integration (S3, AWS Glue, AWS Lambda, Amazon Redshift) with basic IAM, plus Git/Jenkins and CI/CD basics.

Good to Have:

• ETL

Education: Any Graduation

Key Skills

Education

Any Graduate