Description

You will design and implement complex data architectures, including data lakes, warehouses, and lakehouse solutions for cybersecurity applications.

Responsibilities

• Design and implement secure, scalable, and highly available data structures supporting ML-driven insights and real-time analytics.
• Integrate data science workflows and deploy ML models for both real-time and batch processing.
• Build CI/CD pipelines to automate testing, security scans, and deployment processes for data processing workflows.
• Implement automated data quality checks, data lineage, and governance standards using Databricks DQM or similar tools.
• Develop analytics and visualizations in Power BI or Tableau to report on threat patterns and vulnerability trends.

Required Skills

• 8+ years of experience in data engineering.
• Expertise in Python, PySpark, and SQL.
• Extensive hands-on experience with Databricks and big data frameworks.
• Advanced proficiency in AWS, specifically EC2, S3, Lambda, and ELB.
• Experience with container orchestration using Docker and Kubernetes.
• Proven experience with secure DevOps practices, including SCA, SAST, and DAST/WAS within an SDLX framework.
• Demonstrated ability to manage large-scale data environments optimized for performance and compliance.
• Experience working in Agile/Scrum environments.
• Bachelor's degree in Computer Science, Data Science, Information Systems, or equivalent professional experience.

Preferred Skills

• Experience deploying ML models within a cybersecurity context.

Key Skills

Education

Bachelor's degree in Computer Science