Description

You will build and support scalable data engineering pipelines for batch and streaming data.

Responsibilities

• Build and maintain scalable ETL processes for batch and streaming data workloads.
• Design and implement data models for big data infrastructure.
• Optimize data loading and ingestion processes to improve system efficiency.
• Develop and manage real-time monitoring dashboards and alerting systems.
• Work within distributed systems to support large-scale data processing.

Required Skills

• 3+ years of experience in data engineering.
• Proficiency in Python, PySpark, and SQL.
• Hands-on experience with Spark and relational databases like Postgres.
• Experience working with distributed systems software development.
• Familiarity with GitHub, Docker, and VSCode.
• Experience working on public cloud platforms such as AWS or Azure.
• Strong analytical skills for data exploration.
• Degree in any graduate field.

Preferred Skills

• Experience with orchestration tools including Airflow, Kubeflow, or Microsoft Fabric.

Key Skills

Education

Any Graduate