Description

Must-Have Skills:

• Python Development
• Apache Spark & PySpark
• Big Data Ecosystems – Hadoop, Hive, HDFS, Kafka
• SQL & Relational Databases
• Distributed Computing
• Spark Performance Tuning & Optimization
• ETL/ELT Pipeline Development
• Data Transformation, Cleansing & Validation
• Git / Version Control
• Linux / Unix
• Production Support & Troubleshooting
• GenAI Concepts & Applications
• AI-Driven Solution Development

Additional Skills:

• Data Products
• Metadata Management
• Knowledge Graphs
• Ontology
• Semantic Technologies

Key Responsibilities:

• Develop and maintain data processing applications using Python and PySpark
• Design, build and optimize ETL/ELT pipelines for large-scale datasets
• Process structured, semi-structured and unstructured data from multiple sources
• Implement data transformation, cleansing and validation frameworks
• Collaborate with Data Engineers, Data Scientists, Business Analysts and stakeholders
• Optimize Spark jobs for performance, scalability and reliability
• Develop reusable data processing components and frameworks
• Monitor and troubleshoot production Spark application issues
• Identify opportunities to leverage AI and implement AI-driven solutions

Education: Bachelor's Degree in Computer Science

Key Skills

Education

Bachelor's degree