Description

You will design, develop, and implement ETL processes within Azure and AWS cloud environments.

Responsibilities

• Build and implement ETL processes using Databricks and PySpark.
• Design solutions for multiple large-scale data warehouses using cluster and parallel architecture.
• Write optimized SQL queries to improve data workflow performance.
• Collaborate with onshore teams, business analysts, and other engineering teams to align on requirements and best practices.
• Manage large volumes of data using distributed RDBMS and high-scale architectures.

Required Skills

• 8+ years of proven experience in data engineering.
• Expertise in Databricks, Spark, and Python.
• Advanced SQL knowledge with strong database fundamentals, performance tuning, and schema design.
• Hands-on experience with RDBMS and distributed systems.
• Proficiency with Azure and AWS cloud platforms.
• Experience using Git and Azure DevOps.
• Ability to interpret and write custom shell scripts.
• Bachelor's or Master's degree in Computer Science.

Preferred Skills

• Experience handling real-time data, Change Data Feed (CDF), and unstructured data.
• Understanding of CI/CD frameworks and Python scripting.

Key Skills

Education

Bachelor's or Master's degrees