Description

Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark. Build and optimize ETL/ELT workflows for processing large-scale structured and unstructured datasets. Develop data ingestion frameworks to integrate data from multiple sources, including databases, APIs, applications, and cloud storage. Implement and manage Lakehouse architecture using Databricks and Delta Lake. Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency. Collaborate with analytics, BI, and business teams to understand data requirements and deliver reusable data assets. Implement data quality frameworks, monitoring solutions, and governance controls across data pipelines. Troubleshoot production issues and provide timely resolution to ensure business continuity. Participate in architecture reviews, code reviews, and technical discussions to promote best practices. Create and maintain technical documentation, reusable components, and deployment standards

Key Skills

Education

Bachelor's degree