Description

You will design and develop data ingestion frameworks, real-time processing solutions, and data transformation pipelines using open source tools.

Responsibilities

• Design and develop data ingestion frameworks and real-time processing solutions.
• Operationalize open source data-analytic tools for enterprise use.
• Implement and validate data lineage, quality checks, and data classification to ensure governance.
• Follow the software development life cycle to develop, deploy, and deliver solutions.
• Participate in a support rotation to provide system support.

Required Skills

• 6+ years of experience in software engineering or a related field.
• Proficiency in Python, Scala, and Java.
• Hands-on experience with Apache Spark, PySpark, and Big Data technologies.
• Experience with GCP tools including Cloud Composer, BigQuery, and DataProc.
• Strong SQL skills and experience with Hive and Hadoop platforms.
• Experience with orchestration tools like Airflow, Oozie, or Kafka.
• Understanding of the Software Development Life Cycle (SDLC).
• Bachelor's degree in Computer Science, CIS, or a related field.

Key Skills

Education

Any Graduate