Description

Build data pipelines and interfaces for a new analytics platform within a global Data Lake environment.

Responsibilities

• Build and manage data pipelines to ingest various data sources into the Data Lake.
• Develop, test, and manage ETL processes and data warehousing using Big Data technologies.
• Collaborate with Quants to convert models into strategic software while maintaining research agility.
• Define and manage best practices for Data Lake configuration and management.
• Resolve maintenance issues, data bugs, and perform application production support.

Required Skills

• 5+ years of experience in big data environments.
• Strong technical knowledge of the Hadoop stack: HDFS, HBase, Phoenix, Hive, and Pig.
• Proficiency in SQL, PL/SQL, and Transact-SQL.
• Experience with Spring Framework, specifically Spring Core and Spring Batch.
• Hands-on experience with REST web services.
• Familiarity with web, FTP, and API-based ETL technologies.
• Experience managing data in formats such as flat files, XML, Parquet, and data frames.
• Proficiency with GIT, MAVEN, and command line/shell scripting.
• Knowledge of J2EE (JDBC, JNDI, JMS) and Tomcat.
• Experience with performance tuning, optimization, and NoSQL data stores.
• Operational knowledge of schedulers, query performance, and security/encryption.

Preferred Skills

• Experience with Oracle 11g or higher.
• Experience with Spring Integration.

Key Skills

Education

Any gradudate