Description

Must-Have Skills
✅ 10+ years of overall IT experience
✅ Strong hands-on experience with Azure Databricks and production-grade data pipelines
✅ Expert-level PySpark, Spark SQL, Delta Lake, and Spark performance tuning
✅ Strong experience with Azure Cloud services for data engineering
✅ Real-time streaming experience using Azure Event Hubs and Azure Stream Analytics
✅ Experience building ETL/ELT pipelines, Data Services, APIs, and Microservices
✅ Expertise in CI/CD, DevOps practices, and production support
✅ Experience handling structured and unstructured data in cloud and on-prem environments
✅ Strong understanding of data quality, validation, governance, and privacy for regulated environments
🔹 Preferred Skills
⭐ Azure Event Hubs & Stream Analytics
⭐ IoT & Telemetry Data Processing
⭐ Medallion Architecture
⭐ ML Feature Pipelines and AI/ML Data Engineering
⭐ Azure Integration Services
⭐ Experience in Medical Device or Healthcare domains
🔹 Key Responsibilities
✔️ Design and develop scalable batch and real-time streaming data pipelines using Azure Databricks
✔️ Build ETL/ELT workflows for large-scale structured and unstructured datasets
✔️ Develop Data Services, APIs, and Microservices for downstream applications and ML platforms
✔️ Implement real-time telemetry ingestion using Azure Event Hubs and Stream Analytics
✔️ Optimize Databricks workloads with Spark tuning, partitioning, caching, and Delta Lake optimization
✔️ Troubleshoot production issues and perform root cause analysis
✔️ Collaborate with Data Analysts, Software Engineers, ML Engineers, and Business Stakeholders
✔️ Apply best practices in software engineering, testing, documentation, CI/CD, and data governance
🎯 Expected Outcomes
📌 Stabilize and optimize existing Azure Databricks batch and streaming pipelines
📌 Deliver production-ready real-time telemetry ingestion solutions for connected medical devices
📌 Build scalable data services supporting APIs, Analytics, and AI/ML initiatives
📌 Enable secure, reliable, and near-real-time data access across the enterprise
 

Key Skills

Education

Bachelor's degree