Description

• Design, develop, and maintain scalable data pipelines using Azure Databricks and Apache Spark.
• Build and optimize ETL/ELT processes for large-scale structured and unstructured data.
• Develop data transformation and processing frameworks using Java and Spark.
• Write complex SQL queries, stored procedures, and performance-tuned data solutions.
• Integrate data from multiple sources into enterprise data lakes and data warehouses.
• Monitor, troubleshoot, and optimize data processing jobs for performance and reliability.
• Implement data quality, governance, and security standards.
• Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
• Support CI/CD deployment and automation of data engineering workflows.
________________________________________

Required Skills

Data Engineering
• Azure Databricks
• Apache Spark
• Data pipeline design, development, and maintenance • ETL/ELT processes • Large-scale structured and unstructured data processing Programming & Development • Java • Data transformation and processing frameworks using Java and Spark Database & SQL • SQL • Complex SQL queries • Stored procedures • Performance-tuned data solutions Data Integration • Integration of data from multiple sources • Enterprise data lakes • Enterprise data warehouses Operations & Optimization • Data processing job monitoring • Troubleshooting • Performance optimization • Reliability optimization Data Governance & Security • Data quality standards • Data governance standards • Data security standards DevOps • CI/CD deployment • Automation of data engineering workflows Collaboration • Collaboration with business stakeholders • Collaboration with data architects • Collaboration with analytics teams • Translation of business requirements into technical solutions

Key Skills

Education

Bachelor's degree