Description

You will design and implement technical components for big data engineering solutions, translating client requirements into functional designs and building optimized data pipelines.

This role is on-site.

Responsibilities

• Build data ingestion functionality from heterogeneous sources using batch and real-time processing.
• Develop components for data analytics, search, and aggregation.
• Implement data integration, transformation, and storage frameworks.
• Drive design discussions to ensure the long-term health and scalability of data solutions.
• Optimize data pipeline performance and manage infrastructure within cloud environments.

Required Skills

• 5+ years of experience in data engineering or related fields.
• Hands-on experience with the Hadoop stack: HDFS, Sqoop, Kafka, Pulsar, NiFi, Spark, Spark Streaming, Flink, Storm, Hive, Oozie, and Airflow.
• Strong programming proficiency in Java, Scala, and Python, with a preference for Java.
• Working knowledge of NoSQL and MPP data platforms such as HBase, MongoDB, Cassandra, AWS Redshift, Azure SQLDW, or GCP BigQuery.
• Experience building end-to-end data pipelines.
• Knowledge of AWS data platform-related services.

Preferred Skills

• Experience with traditional ETL tools (Informatica, Talend) and RDBMS (Oracle, MySQL, SQL Server, Postgres).
• Knowledge of data governance processes and tools (Collibra, Alation).
• Familiarity with distributed messaging frameworks (ActiveMQ, RabbitMQ, Solace), search/indexing, and microservices architectures.

Key Skills

Education

Any Graduate