Description

Responsibilities:

●        Python & PySpark: Write, test, and maintain Python and PySpark code for data processing and analytics.

●        SQL Proficiency: Develop complex SQL queries and perform data analysis to support business requirements.

●        MongoDB Proficiency: Hands on experience of writing MongoDB query, aggregations, creating API from it.

●        Spark Architecture: Understand and leverage Spark architecture and optimization techniques to improve job performance and efficiency.

●        Collaboration: Work closely with product managers, UX/UI designers, and other developers to gather requirements and translate them into technical specifications.

●        Testing and Debugging: Write unit tests, perform debugging, and ensure code quality through thorough testing and code reviews.

●        Deployment and Maintenance: Oversee the deployment of applications and updates. Monitor performance and troubleshoot issues as they arise.

●        Documentation: Maintain comprehensive documentation of code, processes, and procedures.

Requirements:

Technical Skills:

●        Should have experience in Data and Analytics and overseen end-to-end implementation of data pipelines on cloud-based data platforms.

●        Strong programming skills in Python, Pyspark and some combination Java, Scala (good to have)

●        Deep understanding of modern data processing technology stacks: Spark, HBase, Hive and other Hadoop ecosystem technologies. Development using Scala.

●        Experience writing SQL, Structuring data, and data storage practices.

●        Experience in Pyspark for Data Processing and transformation.

●        Experience building stream-processing applications (Spark streaming, Apache-Flink, Kafka, etc.)

●        Maintaining and developing CI/CD pipelines based on Gitlab.

●        You have been involved in assembling large, complex structured and unstructured datasets that meet functional/non-functional business requirements.

●        Experience of working with cloud data platform and services.

●        Conduct code reviews, maintain code quality, and ensure best practices are followed.

●        Debug and upgrade existing systems.

●        Nice to have some knowledge in Devops.

●        Experience:

o    Proven experience as a Data Engineer or similar role for a product company.

o    Demonstrated ability to build and maintain both front-end and back-end systems.

●        Soft Skills:

o    Strong problem-solving skills and the ability to think critically.

o    Excellent communication and teamwork abilities.

o    Attention to detail and a commitment to producing high-quality work.

●        Education:

o    Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent experience.

Benefits:

●        Work from the office is mandatory. Remote work option only in case of emergency

●        Fun filled work environment and meritocracy driven work culture

●        Opportunities for professional growth and development.

●        A collaborative and innovative work environment.

Key Skills

Education

Bachelor’s degree in Computer Science, Engineering