Description

You will build and scale a large-scale data lake to support enterprise-wide monitoring and analysis.

Responsibilities

• Build and implement a data lake designed to handle 100 petabytes of data per month.
• Deploy and manage ClickHouse databases across the organization to ensure scalability.
• Develop data pipelines to collect and structure logs from various sources.
• Optimize SQL queries for efficiency within a multi-tenant environment.
• Integrate event management layers to trigger actions in ServiceNow.

Required Skills

• 5+ years of experience in data engineering or related roles.
• Hands-on experience with ClickHouse and managing it within Kubernetes.
• Strong understanding of AWS infrastructure and how components connect at scale.
• Proficiency in SQL for complex querying and optimization.
• Experience working with Kubernetes for container orchestration.
• Ability to work with Databricks, Power BI, and Grafana.
• Familiarity with ServiceNow integration.
• Experience managing large-scale, multi-tenant data architectures.

Preferred Skills

• Experience with vectorized development for rate limiting and security.
• Knowledge of using Keep for event management and rules engines.

Key Skills

Education

Any Graduate