Description

Key Skills: Data Engineer, AWS, Python, AWS-Amazon Web Services, MySQL, Ms Sql, MS SQL Server, Pyspark

Roles and Responsibilities:

• Design and develop data pipelines using AWS services and Databricks.
• Manage and optimize databases including MS SQL Server and MySQL.
• Utilize Python for data processing and automation tasks.
• Collaborate with cross-functional teams to understand data requirements and deliver solutions.
• Ensure data quality and integrity throughout the data lifecycle.
• Monitor and troubleshoot data workflows to ensure optimal performance.

Skills Required:

• Strong expertise in AWS cloud ecosystem (S3, Glue, EMR, Lambda, etc.)
• Hands-on experience in building scalable data pipelines using Databricks
• Strong proficiency in Python for data engineering and automation
• Experience with PySpark for big data processing (nice-to-have)
• Strong knowledge of MS SQL Server and MySQL database management
• Experience in ETL/ELT pipeline development and optimization
• Understanding of data lake and data warehouse concepts
• Ability to work with large-scale distributed data systems
• Strong debugging and performance tuning skills
• Experience working in Agile environments and cross-functional teams

Education: B.Tech, M. Tech, B.E.

Key Skills

Education

Any Graduate