Data Engineer

You will build and manage data pipelines using modern orchestration tools.

Responsibilities

• Develop and maintain data pipeline orchestration using Apache Airflow.
• Implement data processing logic using Python scripting for DAGs.
• Design and manage data flows within various environments including cloud and on-premises.
• Execute tests, analyze data, and resolve defects in production systems.

Required Skills

• 5+ years of hands-on experience with Apache Airflow.
• Proficiency in Python scripting for data workflows.
• Experience with data pipeline orchestration best practices.
• Familiarity with at least one major cloud platform (AWS, GCP, or Azure).
• Understanding of CI/CD pipelines and version control (Git).
• Experience with SQL and data warehousing concepts.
• Exposure to Apache Kafka, topics, and partitions.
• Ability to develop, execute tests, and troubleshoot data issues.

Preferred Skills

• Experience with Apache NiFi.
• Exposure to Data Lakes.
• Familiarity with Kafka Connect, Schema Registry, or Kafka Streams.