Description

You will build and optimize data pipelines and warehousing solutions to support large-scale business intelligence requirements.

Responsibilities

• Design and develop enterprise software for large-scale data processing and information retrieval.
• Implement ETL processes and streaming data processing pipelines using Apache NiFi and ADF.
• Write and optimize complex SQL queries to manage data warehousing architectures.
• Develop automation scripts for data manipulation and system tasks.
• Apply Test Driven Development and CI/CD practices to ensure engineering quality.

Required Skills

• 5+ years of enterprise software design and development experience.
• Proficiency in SQL with expertise in complex query optimization.
• Practical programming experience using Spark, Scala, or Python.
• Technical expertise in Azure, Big Data, Apache NiFi, Hadoop, HDInsight, and ADF.
• Hands-on experience with data warehousing and distributed computation.
• Experience with scripting languages such as PowerShell or Bash.
• Ability to analyze and process very large real-world datasets.
• Knowledge of Cloud Computing platforms and implementation.
• Experience working within SCRUM, DevOps, or Agile methodologies.

Preferred Skills

• Experience with reporting and visualization tools like Power BI or Tableau.
• Familiarity with automation tools such as Visual Studio.

Key Skills

Education

Any Graduate