Description

You will design and maintain cloud-based data architectures and pipelines for Azure PaaS and IaaS environments. Your work centers on building reliable, scalable data solutions using SQL, PySpark, and Python.

Responsibilities

• Develop and document technical solution designs for data ingestion, transformation, and storage.
• Build and manage release pipelines using Azure DevOps Pipelines and Release Manager.
• Implement DevSecOps practices, including automation of build, deployment, and environment management.
• Design and optimize ETL processes and data pipelines using PySpark and Python.
• Manage source code using Azure DevOps and Git, ensuring adherence to Agile methodologies.

Required Skills

• 5+ years of experience with SQL Server database design, development, and relational/dimensional modeling.
• Strong proficiency in Python scripting, including data engineering libraries.
• Hands-on experience with PySpark for distributed computing and performance tuning.
• Familiarity with Azure Data Factory, Azure Data Lake Gen 2, and Delta Lake architectures.
• Experience with NoSQL databases such as CosmosDB or MongoDB.
• Proficiency in scripting languages including PowerShell, Bash, DAX, or M-language.
• Knowledge of big data file formats (Parquet, Avro, ORC) and technologies like Hadoop or Hive.
• Ability to work with reporting tools such as SSRS, Tableau, or Power BI.

Key Skills

Education

Any Gradute