Description

You design, build, and maintain scalable data pipelines using Azure Databricks and Python.

This role is remote.

Responsibilities

• Build and manage ETL workflows and data lakes on Azure Databricks.
• Collaborate with data scientists and analysts to define data requirements and deliver quality solutions.
• Implement data engineering best practices to ensure data quality and reliability.
• Monitor, debug, and optimize data workflows for performance and efficiency.

Required Skills

• 5+ years of experience in data engineering with a focus on Azure Databricks and Spark.
• Strong proficiency in Python for data processing, automation, and manipulation.
• Hands-on experience with Databricks workspace, clusters, notebooks, and jobs.
• Proficiency in SQL for querying, data validation, and warehouse interactions.
• Knowledge of big data processing frameworks, data lakes, and data warehouses.
• Experience with cloud-based data platforms, preferably Azure.
• Strong debugging, optimization, and performance tuning skills.

Preferred Skills

• Ability to work effectively in a hybrid environment with cross-functional teams.

Key Skills

Education

Any Graduate