Description

You will design, build, and maintain efficient Data Lake solutions and high-profile data ingestion pipelines. You will take ownership of end-to-end data engineering projects, from architecture to production troubleshooting.

Responsibilities

• Design and implement end-to-end Data Lake solutions and modern data warehouse architectures.
• Build and maintain data ingestion pipelines from various sources using Spark and Hadoop.
• Troubleshoot and fix real-time production jobs in Spark and Hadoop environments.
• Perform independent data analysis and provide end-to-end problem resolution for stakeholders.
• Implement, test, and bug-fix functionality while communicating technical requirements.

Required Skills

• 7+ years of experience in a data engineering role.
• 3+ years of hands-on experience with Hadoop, Spark, and Snowflake.
• Extensive knowledge of Hive, Sqoop, and HBase.
• Proficiency in Scala, Python, and Shell scripting.
• Expertise in writing complex SQL statements using Hive, SnowSQL, and RDBMS standards.
• Experience with Cloudera and CDP Hadoop distributions.
• Bachelor's degree in Computer Science or a similar field.

Preferred Skills

• Knowledge of AWS cloud services including S3, IAM, and EMR.

Key Skills

Education

Any Graduate