Description

You will design and manage complex big data systems and data modeling strategies.

Responsibilities

• Define data retention policies and monitor system performance to advise on infrastructure changes.
• Implement ETL/ELT processes and orchestrate complex data flows.
• Mentor junior engineers and collaborate with other architects on solution delivery.
• Recommend and drive the adoption of new tools and techniques from the big data ecosystem.
• Scale data pipelines using open-source components and AWS services.

Required Skills

• 12+ years of industry experience building and managing big data systems.
• Expertise in building, monitoring, and optimizing cost-efficient pipelines for SaaS.
• Experience with stream-processing systems such as Storm or Spark-Streaming.
• Proficiency in Cloudera, Hortonworks, Spark, HDFS, and NiFi.
• Deep knowledge of RDBMS and NoSQL databases including Vertica and Redshift.
• Strong skills in data modeling, physical design, and SQL performance optimization.
• Experience with messaging systems such as JMS, Active MQ, Rabbit MQ, and Kafka.
• Competency in Big Data querying tools like Pig, Hive, and Impala.
• Hands-on experience with Core Java, Spring/IOC, and design patterns.
• Understanding of distributed computing principles and web-based SOA architecture.
• Proficiency with reporting solutions including Pentaho, PowerBI, and Looker.

Preferred Skills

• Experience working with SaaS-based data management products.
• Knowledge of Web-based SOA architecture implementation and design patterns.

Key Skills

Education

Any Graduate