Description

Build and maintain threat detection pipelines for the security team using PySpark, SQL, and Databricks.

This role is on-site.

Responsibilities

• Migrate detection rules and content from legacy systems to Databricks native internal detection framework.
• Build PySpark log analytics pipelines for threat detection.
• Deploy and manage pipeline infrastructure using Terraform.
• Write unit tests and perform data validation checks.
• Deploy production-ready detection frameworks.

Required Skills

• 5+ years of professional experience in data engineering.
• Advanced proficiency in SQL.
• Strong experience with PySpark and Python.
• Hands-on experience with Databricks platform.
• Proficiency in Terraform for infrastructure deployment.
• Experience with GitHub for version control.

Preferred Skills

• Experience in security or threat detection domains.
• Familiarity with legacy platform migration projects.

Key Skills

Education

Any Gradute