Description

Lead data engineering initiatives and own end-to-end data solutions for on-site delivery in Austin.

This role is on-site.

Responsibilities

• Design and build data pipelines and streaming applications using Python and Kafka.
• Develop and tune complex SQL queries, including joins, views, and recursive logic, optimizing performance.
• Own data pipeline automation, scheduling processes on DataLab, and connecting to DataLake/S3.
• Collaborate with business stakeholders to gather requirements, perform impact analysis, and deliver data projects.
• Lead project delivery, coordinating with offshore teams while independently developing solutions.

Required Skills

• 10+ years of professional experience in data engineering.
• Expert proficiency in Python scripting, including Pandas, NumPy, and handling APIs.
• Deep expertise in SQL, including complex joins, partitioning, and performance tuning.
• Hands-on experience with ETL, Dimensional Data Modelling, and Slowly Changing Dimensions (Type 1, 2, 3).
• Experience with Apache Kafka, producing and consuming messages using Python client libraries.
• Familiarity with Snowflake architecture, including warehouse concepts, scaling, and SnowSQL/Snowpipe.
• Experience with data modeling concepts (Fact/Dimension tables).
• Proficiency in using tools like Tableau server modules.
• Experience setting up and scheduling Python processes on platforms like DataLab.

Key Skills

Education

Bachelor's degree