Description

You will design, build, and optimize enterprise data pipelines to ensure high-quality data ingestion, transformation, and loading across systems.

Responsibilities

• Design, develop, and deploy Spark programs within Databricks environments to process large data volumes.
• Build and optimize ETL pipelines using PySpark and Delta Lake for structured, semi-structured, and unstructured data.
• Implement dimensional data modeling and design enterprise data warehouse solutions.
• Monitor and troubleshoot Databricks jobs to resolve performance bottlenecks and ensure scalability.
• Perform code reviews to ensure adherence to execution patterns and established standards.

Required Skills

• 5+ years of Python coding experience.
• 5+ years of SQL Server based development for large datasets.
• 5+ years of experience developing and deploying ETL pipelines using Databricks and PySpark.
• Proficiency in SQL, including stored procedures and complex query writing.
• Experience with Snowflake, Synapse, BigQuery, or Redshift.
• Expertise in data warehousing concepts including OLTP, OLAP, Dimensions, and Facts.
• Experience with cloud-based data architectures, messaging, and analytics.
• Experience leading enterprise-wide cloud data platform migrations.
• Background in working with event-based and streaming technologies.

Preferred Skills

• Experience with Airflow.
• Cloud certifications.

Key Skills

Education

Any Graduate