Description

Build and scale big data applications on distributed platforms.

Responsibilities

• Architect data products using streaming, serverless, and microservices architectures.
• Configure Jenkins pipelines for CI/CD processes involving Managed Spark jobs and Docker images.
• Implement data warehousing, modelling, governance, and architecture standards.
• Partner with product and engineering teams to solve complex problems and communicate technical solutions to stakeholders.

Required Skills

• 5+ years of experience developing scalable Big Data applications on distributed platforms.
• 4+ years of experience with Spark, Python, and Scala.
• 3+ years of experience with AWS services including S3, Managed Airflow, EMR, EC2, and IAM.
• 3+ years of experience with Databricks (Data Engineering, Delta Lake, and Lakehouse Medallion architecture).
• Experience with Data warehousing tools such as SQL databases, Presto, and Snowflake.
• Experience with Jenkins for CI/CD and building Docker images.
• Working knowledge of reporting and analytical tools like Tableau and QuickSight.
• Experience working within Agile and Scrum development processes.

Preferred Skills

• Ability to quickly learn new programming languages, technologies, and frameworks.

Key Skills

Education

Any Gradute