Description

You will own data domain expertise, platform integrity, and end-to-end ETL pipeline development within an agile team.

This role is on-site.

Responsibilities

• Create functional and technical specifications, including epics, user stories, process flows, and mapping documents.
• Execute migrations from Hadoop to AWS using pipelines and EMR.
• Develop, enhance, and test ETL pipelines and interfaces.
• Manage DevOps efforts covering CI/CD, code scanning, performance testing, and test coverage.
• Establish and enforce guidelines to ensure data asset consistency, quality, and completeness.

Required Skills

• 10+ years of database experience.
• Strong proficiency in UNIX shell scripting.
• Experience with Oracle, Java, Python, Spark, Kafka, HDFS, Airflow, and Elastic Search.
• Hands-on experience with SparkSQL, Impala, and Big Data technologies.
• Working knowledge of AWS services including Lambda and EC2.
• Experience with real-time streaming data.
• Proficiency in relational database environments such as Oracle, Teradata, or SQL Server.
• Experience using Confluence and Jira for documentation and information structuring.
• Working proficiency in SDLC CI/CD execution using GitHub and Jenkins.

Preferred Skills

• Experience with Object-Oriented development and scripting languages.
• Familiarity with Data Science concepts for large-scale data analysis.

Key Skills

Education

Any Graduate