Description

You will design and manage data pipelines using the Elastic Stack to support real-time visualization and data parsing.

Responsibilities

• Build complex Logstash configurations to parse data and integrate diverse data sources.
• Optimize Logstash performance using advanced pipeline management, persistent queues, and worker tuning.
• Ensure high availability and fault tolerance across data workflows.
• Develop and maintain ETL pipelines to streamline data ingestion and processing.

Required Skills

• 5+ years of experience in data engineering or related roles.
• Expertise in Logstash configuration and data parsing.
• Proficiency in Python.
• Proficiency in Bash scripting.
• Strong SQL skills for data manipulation.
• Experience with AWS environments and ETL pipelines.
• Ability to optimize data workflows for real-time visualization.

Preferred Skills

• Experience with advanced pipeline management and worker tuning.

Key Skills

Education

Any Graduate