Description

Lead the expansion and optimization of data pipeline architecture and data flow for cross-functional teams.

Responsibilities

• Design and build cloud-optimized data solutions and architectures from the ground up.
• Orchestrate data management tools within cloud environments and optimize workload management.
• Perform root cause analysis on internal and external data processes to identify improvement opportunities.
• Support data management engineers, consultants, analysts, and product managers to ensure optimal architecture delivery.

Required Skills

• 20+ years of relevant experience in data engineering or architecture.
• Master's degree in Computer Science or a related field.
• Advanced SQL knowledge and experience with relational and NoSQL databases, including Postgres and Cassandra.
• Extensive experience with big data tools including Hadoop, Spark, Kafka, and Kubernetes.
• Proficiency in scripting languages such as Python, Java, Scala, or C++.
• Hands-on experience with AWS cloud services including EC2, EMR, Snowflake, and Elastic-Search.
• Experience with stream-processing systems like Storm, Spark-Streaming, or Kafka.
• Expertise in data pipeline and workflow management tools such as Airflow, Azkabhan, or Luigi.
• Proven ability to manipulate and extract value from large, disconnected, and unstructured datasets.

Preferred Skills

• Knowledge of Agile methodologies.

Key Skills

Education

Master's degree