Description

You will design and build scalable data pipelines and manage data services within an AWS environment.

Responsibilities

• Design and build scalable data pipelines using Hadoop, Hive, PySpark, and Python.
• Integrate and manage data services with Amazon S3 for secure object storage.
• Develop data models and database designs to support business requirements.
• Implement job scheduling using Autosys and maintain Unix/Shell scripts.
• Contribute to CI/CD pipelines and optimize data workflows and transformations.

Required Skills

• 5+ years of experience in Software Engineering or equivalent.
• Proficiency with Hadoop, Hive, PySpark, and Python.
• Experience managing data services using Amazon S3.
• Hands-on experience with Autosys for job scheduling.
• Ability to create visualizations and dashboards using Power BI and Dremio.
• Experience writing and maintaining Unix/Shell scripts.
• Knowledge of CI/CD pipelines.
• Solid understanding of database fundamentals such as MySQL.
• Any Graduate degree.

Preferred Skills

• Experience with GCP.
• Familiarity with financial services or domain-specific data.

Key Skills

Education

Any Graduate