Description

You will lead the migration of a data warehouse from Oracle and SAP Data Services to the Databricks platform.

Responsibilities

• Develop automated data pipelines from heterogeneous sources, handling file processing, data validation, and exception reporting.
• Collaborate with data analysts and engineers to translate business requirements into scalable ETL solutions.
• Optimize ETL workflows for performance, scalability, and reliability to ensure accurate data delivery.
• Maintain comprehensive documentation for ETL processes, including data mappings, transformations, and workflow diagrams.
• Monitor ETL jobs, troubleshoot issues, and implement enhancements to improve data quality and system efficiency.

Required Skills

• 7-10 years of experience in ETL pipeline development.
• 3-5 years of experience with the Databricks platform.
• 3-5 years of experience with SAP Data Services development.
• Proficiency in SQL and Python for data manipulation and automation.
• Experience designing and deploying ETL workflows for JSON and Parquet files.
• Experience with AWS and big data technologies like Spark.
• Knowledge of relational databases including Oracle, SQL Server, and PostgreSQL.
• Strong understanding of data warehousing concepts and data modeling techniques.

Preferred Skills

• Bachelor's degree in Computer Science or Information Systems.

Key Skills

Education

Bachelor's degree in Computer Science, Information Systems