Description

Design and build hybrid data solutions using on-premise and cloud services. You will own the release pipeline, manage production deployments, and provide post-production support for terabyte-scale data systems.

Responsibilities

• Build tools to ingest and process terabyte-scale data daily.
• Create detailed design documents, technical specifications, and test plans.
• Lead unit testing, SIT, and UAT processes.
• Troubleshoot performance bottlenecks and resolve technical issues in an agile environment.
• Manage development tasks and production deployment lifecycles.

Required Skills

• 8 to 13 years of experience in data architecture and engineering.
• Deep knowledge of the Hadoop ecosystem: HDFS, YARN, Spark, MapReduce, Pig, Hive, Flume, Sqoop, Oozie, Impala, ZooKeeper, and Kafka.
• Proficiency in Spark/PySpark and Hive.
• Experience with either Redshift or Snowflake.
• Strong SQL skills with experience in MySQL, Oracle DB, or SQL Server.
• Experience with NoSQL technologies such as Cassandra and MongoDB.
• Expertise in SQL writing, performance tuning, optimization, and data modeling.
• Proven track record with ETL, data warehousing, and large-scale data migration (history and incremental loads).
• Programming proficiency in Python, R, Java, or Scala.

Preferred Skills

• Knowledge of AWS services including EMR, Glue, Lambda, DMS, Kinesis, and DynamoDB.

Key Skills

Education

ANY GRADUATE