Description

You will build and maintain scalable data solutions within a big data ecosystem.

Responsibilities

• Design and implement data ingestion, profiling, and quality processes using PySpark, Hive, and Impala.
• Assist with Azure cloud migrations of existing data processes from classical infrastructure.
• Develop ETL jobs and workflows using SQOOP, PySpark, and Stream Sets to meet business requirements.
• Perform gap and impact analysis on ETL and IOP jobs for new requirements and enhancements.
• Conduct unit testing and create mockup data to validate results in lower environments.

Required Skills

• 5+ years of experience building data sets.
• 3+ years of experience implementing complex ETL logic.
• Strong SQL expertise with Oracle and Hadoop (Hive, Impala).
• Proficiency in Java, Scala, Python, and SQL.
• Experience with Hadoop, Spark, and Hive implementations.
• Knowledge of relational and NoSQL databases, data modeling, and performance tuning.
• 2+ years of experience with UNIX/LINUX shell scripting.
• Familiarity with Waterfall and Agile methodologies.
• Bachelor’s Degree in Computer Science, Computer Engineering, or a closely related field.

Key Skills

Education

Any Graduate