Description

You will design and implement platforms and frameworks for extracting, transforming, and loading data from diverse sources.

Responsibilities

• Integrate data from multiple internal and external sources using batch and real-time methods.
• Coordinate and enhance data quality and availability across the data estate.
• Automate and optimize analytic workloads by identifying patterns in data and technology usage.
• Design and deploy data pipelines and frameworks for optimal ETL processes.
• Contribute to the technical skill building of the team through shared engineering practices.

Required Skills

• Solid Python development skills.
• Highly proficient in SQL.
• 3+ years of experience building and performance tuning Spark code, preferably using Scala.
• 3+ years of experience using relational databases for ETL processes and large data transformations.
• 2+ years of experience designing and deploying data analytics pipelines in the cloud using Azure Data Services.
• Expertise with distributed computing and big data architectures.
• In-depth knowledge of AWS, specifically IAM, Neptune, and EC2.
• Hands-on experience with Hadoop and Kafka.
• Experience building self-contained applications using Docker, Vagrant, or Chef.
• Bachelor’s Degree in Computer Science, Computer Engineering, or a closely related field.

Preferred Skills

• Machine Learning or Data Science exposure.
• Experience with Big Data migrations or transformations in Data Warehousing or Business Intelligence.
• AWS experience with ECS/EKS, Lambda, SQS, Kinesis, or Graph ML tools.

Key Skills

Education

Any Graduate