Description

You will build and maintain scalable analytics solutions, data lakes, and modern data platforms on Microsoft Azure.

Responsibilities

• Design and build metadata-driven data ingestion pipelines for batch and streaming datasets.
• Execute high-performance data processing for structured and unstructured data, including data harmonization.
• Schedule, orchestrate, and validate data pipelines while implementing exception handling and log monitoring.
• Collaborate with consulting, data science, and application development teams to deliver analytical solutions.
• Support BI and data science teams in consuming data securely and governed manner.

Required Skills

• 4 to 9 years of total IT experience with 2+ years in big data engineering and Microsoft Azure.
• Hands-on experience with Azure Data Factory (ADF), PySpark, Databricks, ADLS, and Azure SQL Database.
• Working knowledge of Azure Synapse Analytics, Event Hub, Streaming Analytics, Cosmos DB, and Purview.
• Proficiency in writing modular, scalable code using SQL, Python, or Scala/Java with strong debugging skills.
• Experience with big data technologies including Hadoop, Spark, Airflow, NiFi, Kafka, Hive, Neo4J, and Elastic Search.
• Understanding of file formats such as Delta Lake, Avro, Parquet, JSON, and CSV.
• Knowledge of building and designing REST APIs within Data Lake or Lakehouse projects.

Preferred Skills

• Microsoft Azure Data Engineering certification (DP-203) or Databricks Certified Developer (DE).

Key Skills

Education

ANY GRADUATE