Description

You will build and maintain data pipelines to move data between MS SQL, Snowflake, and email archives. You will ensure data quality and integrity across systems while optimizing processes for datasets containing billions of rows.

Responsibilities

• Develop and maintain data pipelines between MS SQL, Snowflake, and email archives.
• Write Python scripts to automate data processing, cleaning, and visualization.
• Optimize SQL queries and data processes to handle large-scale datasets.
• Implement LLMs via APIs for text classification and information extraction from email conversations.
• Communicate complex data insights and recommendations to non-technical stakeholders.

Required Skills

• 2+ years of experience as a Data Engineer or Data Analyst.
• Strong SQL skills for querying and manipulating large datasets.
• Proficiency in Python for data analysis and automation.
• Experience with Airflow or similar open-source workflow orchestrators.
• Experience with Airbyte or similar open-source integration tools.
• Knowledge of machine learning algorithms and production implementation.
• Associate degree or higher in Computer Science, Mathematics, Statistics, Physics, or a related field.

Preferred Skills

• Experience using Snowflake.
• Experience with retail datasets including PO, Invoice, and Sales at supplier and item levels.
• Experience with Docker and Kubernetes.

Key Skills

Education

Any Graduate