Description

You will design and build cloud-based data solutions, owning the migration of data from MSSQL to Postgres and implementing high-volume ETL pipelines.

This role is remote.

Responsibilities

• Design and implement data models in Postgres DB on Azure.
• Create and maintain optimal data pipeline architecture.
• Develop transformation pipelines in Databricks to migrate data from MSSQL to Postgres.
• Deploy high-volume ETL pipelines for complex and near-real-time data collection.
• Mentor team members and automate manual tasks to improve internal processes.

Required Skills

• 5+ years of experience as a Data Engineer.
• Expertise in Azure Data Engineering, including Databricks, ADF, and Azure Functions.
• Strong proficiency with Spark, PySpark, and Scala for ETL and data processing.
• Experience with high-volume ETL pipeline development and optimization.
• Proficiency in MSSQL to Postgres DB migration.
• Familiarity with Azure DevOps and Agile development methodologies.
• Ability to collaborate with cross-functional teams to optimize data flow.

Key Skills

Education

Any Graduate