Description

Design, develop, and maintain large-scale data engineering solutions on Microsoft Azure. You will own the data pipeline lifecycle, from ingestion to analytics, ensuring robust performance and governance.

This role is on-site.

Responsibilities

• Build and optimize data lakes and warehouses using Azure Data Factory, Databricks, and Azure Blob Storage.
• Develop scalable data processing scripts using Python, Spark, and SQL.
• Implement CI/CD pipelines using Azure DevOps, GitHub, Jenkins, and Terraform.
• Integrate with diverse data platforms including Teradata, Cassandra, MongoDB, and Snowflake.
• Ensure data governance and privacy using Informatica Axon, EDC, and BigID.

Required Skills

• 5-7+ years of software solution development using agile and DevOps practices.
• 5-7+ years of data analytics experience using SQL.
• 5-7+ years of cloud development and data lake experience, preferably on Microsoft Azure.
• Expertise in Azure services: Data Factory, Databricks, Blob Storage, Data Lake, Event Hub, Functions, and Power Apps.
• Proficiency in scripting with Python, Spark, and Unix.
• Experience with data platforms: Teradata, Cassandra, MongoDB, Oracle, SQL Server, ADLS, Snowflake, or Azure Data Explorer.
• Strong understanding of Data Warehousing concepts and data governance.

Preferred Skills

• Azure Development Track Certification or Spark Certification.
• Experience with BI tools: Cognos, Tableau, Power BI, Alteryx, Denodo, or Grafana.
• Background in the Airline Industry.

Key Skills

Education

Bachelor's degree