Description

Lead the design and optimization of scalable data pipeline architectures to support cross-functional teams and next-generation products.

Responsibilities

• Design and implement cloud-optimized data solutions from the ground up.
• Optimize data flow, collection, and architecture for multiple data management squads and analytics platforms.
• Build processes for data transformation, metadata management, dependency tracking, and workload management.
• Perform root cause analysis on internal and external data processes to identify and execute improvements.
• Support data management engineers, consultants, analysts, and product managers in delivering optimal architecture.

Required Skills

• 12+ years of relevant professional experience in data engineering.
• Advanced SQL knowledge and experience with relational and NoSQL databases, including Postgres and Cassandra.
• Expertise in building and optimizing big data pipelines and architectures.
• Proficiency in Python, Java, C++, or Scala.
• Hands-on experience with AWS services, including EC2 and EMR.
• Experience with Snowflake and Elastic-search.
• Experience with big data tools such as Hadoop, Spark, Kafka, and Kubernetes.
• Working knowledge of stream processing systems like Storm, Spark-Streaming, or Kafka.
• Experience with workflow management tools such as Airflow, Azkabun, or Luigi.

Preferred Skills

• Knowledge of agile methods.

Key Skills

Education

Any Graduate