Description

You will design and build data solutions on Databricks, owning the development of scalable ETL pipelines and lakehouse architectures.

This role is on-site.

Responsibilities

• Develop and maintain robust data pipelines using Databricks, Apache Spark, and PySpark.
• Write efficient SQL queries and implement complex data transformations within Delta Lake environments.
• Deploy and manage code using Git, CI/CD, and Agile workflows.
• Collaborate with stakeholders to solve complex data engineering challenges.
• Ensure data security and governance best practices are integrated into pipeline design.

Required Skills

• 10+ years of software engineering experience with a Bachelor's degree.
• Strong hands-on experience with Databricks and Delta Lake.
• Expert-level Python programming and Pyspark skills.
• Deep expertise in Apache Spark and Spark SQL.
• Proficient in SQL and data transformation techniques.
• Familiarity with cloud platforms: AWS, Azure, or GCP.
• Experience with Git, CI/CD, and Agile methodologies.

Preferred Skills

• Experience with Airflow, Azure Data Factory, or Databricks Workflows.
• Knowledge of Kafka or streaming frameworks.
• Familiarity with MLflow or analytics platforms.

Key Skills

Education

Bachelor's degree