Description

Lead the design and implementation of big-data solutions and scalable ETL processes within the Azure ecosystem.

Responsibilities

• Craft low-level designs for data warehousing solutions on Azure cloud.
• Construct scalable data pipelines and ETL processes.
• Implement big-data solutions including Data Lakes.
• Validate data pipelines using established testing methodologies and data quality assurance practices.
• Monitor system performance and data integrity using Azure Monitor.

Required Skills

• Minimum 2 years of hands-on expertise in PySpark, including Spark job performance optimization.
• Proficiency in Python and PySpark.
• Hands-on experience with Azure ADF or Databricks.
• Minimum 2 years of hands-on experience with Azure Cloud services.
• Experience with Azure Batch, Azure Function, Storage account, KeyVault, Snowflake/Synapse, SQLMI, and Azure Monitor.
• Knowledge of data modeling, integration, and design principles.
• Practical experience with data security services.
• Working knowledge of DevOps environments.
• Any graduate degree.

Key Skills

Education

Any graduate