Description

You will engineer and maintain scalable Big Data processing applications and platforms.

This role is remote.

Responsibilities

• Develop highly scalable and extensible Big Data platforms for data collection, storage, modeling, and analysis.
• Build Big Data applications utilizing Spark and Scala or Python.
• Implement data pipelines using Airflow and manage container orchestration with Kubernetes.
• Manage infrastructure provisioning using Infrastructure as Code tools like Terraform.
• Develop and refine ETL processes, ensuring strong adherence to SQL and Data Warehousing concepts.

Required Skills

• 4-6 years of experience developing Big Data applications on Azure Databricks.
• Proficiency in Spark, Scala, and Python for Big Data development.
• Experience with Databricks components and Infrastructure as Code (Terraform).
• Working knowledge of Airflow and Kubernetes.
• Strong command of SQL programming and ETL Data Warehousing concepts.
• Familiarity with CI/CD processes, Git, and Jenkins.

Preferred Skills

• Any Graduate degree or equivalent training.

Key Skills

Education

Any Graduate