Description

Design, build, and maintain scalable data pipelines and infrastructure to support big data initiatives.

On-site.

Responsibilities

• Design, develop, and implement end-to-end data pipelines using Hadoop, Spark, and Kafka.
• Build and maintain scalable data ingestion and transformation processes for data lake and data warehouse solutions.
• Optimize and tune data pipelines for performance, reliability, and scalability.
• Implement data governance, security, and privacy measures to protect sensitive information.
• Monitor and troubleshoot pipeline issues to provide timely resolutions.

Required Skills

• 8+ years of experience as a Big Data Engineer or in a similar role.
• Strong programming skills in Python, Scala, or Java.
• Hands-on experience with Hadoop, Spark, Kafka, and HBase.
• Proficiency in SQL and NoSQL databases.
• Experience with cloud platforms such as AWS, Azure, or Google Cloud Platform.
• Familiarity with data warehousing concepts and ETL processes.
• Understanding of data governance, security, and data quality best practices.
• Bachelor's degree in Computer Science.

Preferred Skills

• Experience with containerization and orchestration tools like Docker and Kubernetes.
• Knowledge of machine learning and data science concepts.
• Experience in the financial services industry.

Key Skills

Education

Bachelor's degree in Computer Science