Description

You will design, build, and maintain scalable big data architectures supporting enterprise analytics and operational needs.

This role is hybrid.

Responsibilities

• Develop, implement, and optimize data pipelines using Apache Airflow, Databricks, and related technologies.
• Manage and enhance data workflows for both batch and real-time processing to ensure efficiency and scalability.
• Collaborate with data scientists, analysts, and business stakeholders to translate requirements into data solutions.
• Implement data governance, security, and compliance practices to protect sensitive information.
• Develop monitoring dashboards and alert systems to ensure pipeline health and performance.

Required Skills

• 5+ years of experience in Big Data architecture design, including Hadoop, Spark, and Databricks.
• Strong expertise in performance tuning for large-scale data systems.
• Hands-on experience with Apache Airflow for workflow orchestration.
• Proficiency in SQL for managing and querying large databases.
• Extensive experience with Python for scripting, automation, and data processing workflows.
• Experience working with cloud platforms (Azure, AWS, or GCP).

Preferred Skills

• Knowledge of containerization (Docker, Kubernetes).

Key Skills

Education

Any Gradute