Description

You will be responsible for delivering support data, analytics, and reporting functionalities within the GWD platform, working closely with technical leads and Product Owners.

Responsibilities

• Develop and implement hands-on applications across the entire Software Development Life Cycle (SDLC)—design, build, test, and deploy.
• Design and implement data pipelines using PySpark and related big data tools to ingest, process, and deliver large datasets.
• Optimize and troubleshoot Spark jobs for performance, focusing on partitioning strategies and analyzing execution plans.
• Ensure data integrity, security, and performance when modernizing database environments.
• Lead the migration of legacy database modules onto the big data stack.

Required Skills

• 10+ years of extensive experience in data architecture, database design, or data engineering roles, focused on big data.
• Significant experience with Oracle SQL programming and development.
• Deep expertise in PySpark development, including RDDs, DataFrames, transformations, and actions, writing optimized, scalable jobs.
• Strong grasp of Object-Oriented Programming (OOP) principles applied within PySpark for maintainable code.
• Hands-on knowledge of Hadoop ecosystem components (HDFS, YARN) and Hive for distributed storage.
• Proven experience working with large datasets (e.g., ~100 million records) and optimizing for scalability.
• Experience in deep data analysis to identify gaps and patterns supporting enhancements.
• Familiarity with the full SDLC process.

Key Skills

Education

Any Graduate