Description

You will design and manage enterprise-level data pipelines and warehouse solutions within a cloud environment.

Responsibilities

• Build and optimize data pipelines for ingestion, transformation, and loading from structured, semi-structured, and unstructured sources.
• Develop and deploy Spark programs in a Databricks environment to process large-scale data volumes.
• Design and implement dimensional data models and enterprise data warehouse solutions.
• Optimize Databricks jobs for performance, scalability, and bottleneck resolution.
• Perform code reviews to ensure optimal execution patterns and adherence to engineering standards.

Required Skills

• 8+ years of experience in data engineering roles.
• 5+ years of professional Python coding experience.
• 5+ years of SQL Server-based development managing large datasets.
• 5+ years of experience developing and deploying ETL pipelines using Databricks and PySpark.
• Proficiency in SQL, including stored procedures and complex query optimization.
• Strong knowledge of Data Warehouse concepts including OLTP, OLAP, Dimensions, Facts, and Dimensional modeling.
• Experience with Snowflake, Databricks, or other cloud data warehouses like Big Query or Redshift.
• Expertise in PySpark coding and Spark program deployment.
• Experience with Azure Data Factory (ADF).

Preferred Skills

• Cloud certifications.
• Experience with event-based or streaming technologies.

Key Skills

Education

Any Graduate