Description
You will design and build scalable data pipelines and data lake/warehouse solutions on Azure and Databricks.
Responsibilities
- Design and build scalable data pipelines and data lake/warehouse solutions using Azure and Databricks.
- Develop and maintain ETL/ELT processes using Azure Data Factory, Talend, or Informatica.
- Implement dimensional data modeling and schema design using SQL.
- Manage and optimize data storage and retrieval using Azure Synapse, Azure SQL, Snowflake, Redshift, or BigQuery.
- Process big data using Spark, PySpark, and Spark SQL.
Required Skills
- 5+ years of experience in data engineering, data warehousing, or data lake technologies.
- Extensive experience with the Azure cloud platform.
- Expertise in SQL, data modeling, and data warehouse architecture.
- Hands-on experience with Databricks, Spark, and PySpark/Spark SQL.
- Proficiency with ETL/ELT tools such as Azure Data Factory (ADF), Talend, or Informatica.
- Experience with Azure Synapse, Azure SQL, Snowflake, Redshift, or BigQuery.
- Any Graduate degree.
Preferred Skills
- Knowledge of Azure Event Hub, IoT Hub, Stream Analytics, Cosmos DB, or Azure Analysis Services.
- Familiarity with SAP ECC, S/4HANA, or HANA data sources.
- Intermediate skills in Power BI, Azure DevOps, CI/CD pipelines, and cloud migration strategies.