You will design, develop, and optimize data pipelines and analytics solutions using Databricks on cloud platforms.
Responsibilities
- Design and implement ETL/ELT pipelines using Databricks and Apache Spark.
- Develop and optimize data workflows for large-scale data processing.
- Integrate data from multiple sources into data lakes and data warehouses.
- Collaborate with data architects and analysts to deliver scalable solutions.
- Implement data quality checks, monitoring, and performance tuning.
Required Skills
- 8+ years of experience as a Databricks Developer.
- Expert proficiency in Spark, Python, and/or Scala.
- Strong command of SQL for data manipulation and analysis.
- Hands-on experience with Databricks and Apache Spark.
- Experience with Delta Lake and Lakehouse architecture.
- Familiarity with cloud platforms including AWS, GCP, or Azure.
- Knowledge of CI/CD pipelines and DevOps practices.
- Understanding of data modeling, data governance, and security.