Description

You will design and implement end-to-end data pipelines for ingestion, transformation, and storage of structured, semi-structured, and time-series data.

This role is on-site.

Responsibilities

• Build real-time and batch processing frameworks using Databricks to support scalable analytics and AI workloads.
• Develop and maintain ETL/ELT workflows using Python and SQL, ensuring code is reusable and maintainable.
• Architect and optimize data lakes and lakehouses (Azure Synapse, Delta Lake, BigQuery, or Snowflake) for efficient querying and cost control.
• Design and manage NoSQL (MongoDB) and time-series databases for sensor and operational data.
• Orchestrate and monitor data pipelines using Azure Data Factory, Azure Functions, and Event Hub.

Required Skills

• 8+ years of experience in data engineering, analytics, or big data systems.
• Strong programming skills in Python and SQL for data transformation, orchestration, and automation.
• Expertise in Azure data services including Synapse, Data Factory, Event Hub, Azure Functions, and Databricks.
• Hands-on experience with MongoDB and time-series databases.
• Proven experience with streaming frameworks and workflow orchestrators.
• Proficiency in Docker and Kubernetes for containerization and scalable deployment.
• Familiarity with data lake/lakehouse architectures and cloud-agnostic patterns.
• Knowledge of CI/CD pipelines and infrastructure-as-code tools.

Key Skills

Education

Any Graduate