Description

You design and develop data pipelines using AWS services and Databricks, managing MS SQL Server and MySQL databases.

This role is remote.

Responsibilities

• Design and develop scalable data pipelines using AWS services and Databricks.
• Manage and optimize MS SQL Server and MySQL databases.
• Utilize Python for data processing and automation tasks.
• Collaborate with cross-functional teams to define data requirements and deliver solutions.
• Monitor and troubleshoot data workflows to ensure optimal performance and data integrity.

Required Skills

• 5+ years of experience as a Data Engineer.
• Strong expertise in AWS cloud ecosystem (S3, Glue, EMR, Lambda).
• Hands-on experience building scalable data pipelines using Databricks.
• Proficiency in Python for data engineering and automation.
• Experience with PySpark for big data processing.
• Strong knowledge of MS SQL Server and MySQL database management.
• Experience in ETL/ELT pipeline development and optimization.
• Understanding of data lake and data warehouse concepts.
• Ability to work with large-scale distributed data systems.

Key Skills

Education

Any Graduate