Description

You will design, develop, and maintain scalable data pipelines and architectures, owning the performance, scalability, and reliability of ETL processes across cloud environments.

Responsibilities

• Build and maintain data pipelines using Databricks and Spark.
• Optimize pipeline performance for scalability and reliability.
• Implement ETL processes and data warehouse solutions compliant with security policies.
• Collaborate with data scientists and analysts to define data requirements.
• Troubleshoot data issues, perform root cause analysis, and create validation tools.

Required Skills

• 5+ years of experience in data engineering roles.
• Proficiency in SQL and Python.
• Hands-on experience with Databricks and Spark.
• Experience with cloud platforms such as Azure or AWS.
• Familiarity with Java or Scala.
• Experience with big data technologies including Kafka, Hadoop, or Hive.
• Knowledge of data warehouse and data lake architectures.

Preferred Skills

• Familiarity with visualization tools such as Power BI or Tableau.
• Strong analytical and problem-solving skills.

Key Skills

Education

ANY GRADUATE