Description

You will design, build, and optimize large-scale data architectures and processing pipelines. You own the end-to-end lifecycle of data transformation, metadata management, and workload distribution.

Responsibilities

• Build and optimize big data pipelines, architectures, and datasets for scalability.
• Implement processes for data transformation, metadata management, and dependency tracking.
• Perform root cause analysis on internal and external data to resolve issues and improve processes.
• Extract value from large, disconnected, and unstructured datasets.
• Collaborate with cross-functional teams to support data-driven business requirements.

Required Skills

• 6-9 years of experience in a Data Engineering role.
• Graduate degree in Computer Science, Statistics, Informatics, Information Systems, or a quantitative field.
• Advanced SQL knowledge and experience with relational databases and query authoring.
• Proficiency with big data tools including Hadoop, Spark, and Kafka.
• Experience with AWS cloud services: EC2, EMR, RDS, and Redshift.
• Hands-on experience with stream-processing systems such as Storm or Spark-Streaming.
• Experience with NoSQL databases and data pipeline/workflow management tools.
• Strong programming skills in Python, Java, C++, or Scala.
• Working knowledge of message queuing and highly scalable big data stores.

Preferred Skills

• Strong analytical skills specifically applied to unstructured datasets.
• Proven project management and organizational abilities.

Key Skills

Education

Bachelor's degree in Computer Science