Description

You will build, develop, and manage data systems, tools, and pipelines to process and organize data.

Responsibilities

• Evaluate business needs to structure data systems and pipelines.
• Combine raw information from disparate sources.
• Export and import data into HDFS, HBase, and Hive using Sqoop.
• Migrate data from RDBMS systems to GCP and Azure cloud platforms.
• Perform unit testing for data validation and deploy pipelines via CI/CD.

Required Skills

• 5+ years of experience in data engineering.
• Master's degree in Computer Science, Engineering, Technology, Management, Business Analytics, or related field.
• Proficiency with HDFS, HBase, and Hive.
• Familiarity with Sqoop for data movement.
• Hands-on experience with GCP and Azure.
• Experience migrating data from RDBMS systems.
• Experience with CI/CD processes.

Key Skills

Education

Master's degree