Description

You will design and develop enterprise-scale data pipelines and warehousing solutions.

Responsibilities

• Write and optimize complex SQL queries for large-scale datasets.
• Develop data processing workflows using Spark, Scala, and Python.
• Implement ETL processes and streaming data pipelines.
• Automate data manipulation and system tasks using PowerShell or Bash scripting.
• Apply Test Driven Development and CI/CD practices to data engineering workflows.

Required Skills

• 5+ years of enterprise software design and development experience.
• Proficiency in SQL and complex query optimization.
• Hands-on experience with Spark, Scala, and Python.
• Technical expertise in Azure, Big Data, Apache NiFi, Hadoop, HDInsight, and ADF.
• Strong understanding of data warehousing and distributed computation.
• Experience with streaming data processing and ETL.
• Practical knowledge of Cloud Computing platforms and implementation.
• Experience using Power BI, Tableau, or similar visualization tools.
• Background in Agile methodologies such as SCRUM or DevOps.

Preferred Skills

• Experience with automation tools like Visual Studio.
• Familiarity with telemetry and information retrieval.

Key Skills

Education

Any Gradute