Description

You will build and orchestrate data pipelines and distributed computing applications within an AWS ecosystem.

Responsibilities

  • Build and orchestrate data pipelines and ETL processes.
  • Develop distributed computing applications using PySpark.
  • Design and implement data models using normalization, denormalization, and schema design.
  • Write, maintain, and execute automated unit tests following Test-Driven Development (TDD) practices.
  • Build APIs and manage serverless architectures.

Required Skills

  • 5+ years of experience in big data environments.
  • Proficiency in Python programming.
  • Strong expertise in SQL, Presto, Hive, and Spark.
  • Experience with PySpark and libraries including Pandas, Polars, and NumPy.
  • Extensive experience with AWS services: EMR, Lambda, Glue ETL, Step Functions, S3, ECS, Kinesis, IAM, RDS PostgreSQL, DynamoDB, CloudWatch Events/EventBridge, Athena, SNS, SQS, and VPC.
  • Experience with relational and NoSQL databases, including Amazon Redshift.
  • Knowledge of trading and investment data.
  • Experience with OneTick or KDB.
  • Understanding of CI/CD, source control, and data warehousing concepts.

Preferred Skills

  • Proficiency in data visualization tools, specifically Tableau.

Key Skills
Education

Any Graduate

Back To Jobs