Description

Manage Big Data ecosystems and lead data migration efforts between Hadoop and Cloudera Data Platform (CDP). This role is on-site.

Responsibilities

• Administer Hortonworks Hadoop clusters and Cloudera Data Platform (CDP).
• Develop data processing and ingestion workflows using Spark, Scala, and PySpark.
• Execute database migrations to MS SQL Server and Hadoop while optimizing OLTP and OLAP performance.
• Build and maintain CI/CD pipelines for database changes using GitHub, Jenkins, and Liquibase.
• Automate tasks through Batch, Shell, and Python scripting.

Required Skills

• 8+ years of experience in Big Data and Hadoop ecosystems.
• Proficiency with Spark, Scala, and PySpark.
• Hands-on experience with Hadoop (Hortonworks) and CDP.
• Strong knowledge of MS SQL Server 2019.
• Experience with Jenkins, GitHub, and Liquibase for CI/CD.
• Proficiency in Python, Shell, and Batch scripting.
• Experience working within Agile and Scrum methodologies using Jira.
• Bachelor's degree.

Key Skills

Education

Bachelor's degree