Description

Build and maintain scalable data pipelines and infrastructure to handle large volumes of data. Collaborate with cross-functional teams to onboard data into the Big Data ecosystem. Monitor data quality, perform root cause analysis, and resolve data discrepancies. Design and optimize data schemas and workflows for efficiency and reliability. Assist in rationalization and cleanup efforts in Databricks prior to migration.

Responsibilities

• Build and maintain scalable data pipelines and infrastructure.
• Collaborate with cross-functional teams to onboard data into the Big Data ecosystem.
• Monitor data quality, perform root cause analysis, and resolve data discrepancies.
• Design and optimize data schemas and workflows for efficiency and reliability.
• Assist in rationalization and cleanup efforts in Databricks prior to migration.

Required Skills

• 5+ years of engineering experience in data systems.
• Strong knowledge of relational SQL and NoSQL databases such as Postgres and Cassandra.
• Experience with cloud-based ecosystems, particularly AWS services like EC2, S3, RDS.
• Proficiency in scripting and programming languages including Python, Java, Scala, and R.
• Bachelor’s or Master’s degree in Computer Science or a related field.

Key Skills

Education

Any Graduate