Description

You will design, implement, and maintain batch and real-time data pipelines to produce reliable data for various consumption use cases.

Responsibilities

• Design and maintain data tools to ensure data quality, governance, and compliance.
• Manage the data warehouse and integrate various data sources including Salesforce and ERP.
• Build and own automation and monitoring frameworks for operational KPIs and pipeline performance.
• Develop and maintain data marts to support specific business needs.
• Execute proof of concept evaluations for new open source and proprietary technologies.

Required Skills

• 5+ years of experience in data engineering and building production-grade systems.
• Expertise in programming pipelines using Python or Scala.
• Advanced experience with AWS cloud technologies.
• Proficiency with AWS Glue for data integration.
• Experience with Redshift for data warehousing.
• Hands-on experience with Airflow for data orchestration and scheduling.
• Strong command of SQL and RDBMS concepts.
• Experience with ETL, data architecture, and distributed data processing patterns.
• Proficiency with version control (GitHub, BitBucket) and CI/CD tools.

Preferred Skills

• Understanding of metadata management, data lineage, and data glossaries.
• Experience working within the fintech industry.

Key Skills

Education

Any Graduate