Description

Build and optimize large-scale data pipelines and architectures to extract value from disconnected datasets.

Responsibilities

• Build processes for data transformation, metadata management, dependency tracking, and workload management.
• Perform root cause analysis on internal and external data to identify process improvements.
• Execute data acquisition through web scraping and research domain-specific data sources.
• Design meaningful visualizations and transform complex concepts into production applications.
• Mentor less experienced team members by providing technical expertise and guidance.

Required Skills

• 5+ years of experience in data engineering and big data pipeline optimization.
• Advanced SQL knowledge for query authoring and working with relational databases.
• Proficiency in Python and R.
• Experience with data wrangling, normalization, and homogenization.
• Hands-on experience with Qlik or Tableau for data visualization.
• Ability to handle large quantities of unstructured datasets.
• Knowledge of ETL processes and QA techniques to identify and remove duplicates.
• Understanding of message queuing, stream processing, and scalable big data stores.
• Bachelor's degree in Computer Science.

Preferred Skills

• Experience with Deep Learning, Neural Networks, Computer Vision, or Text Analytics.
• Exposure to Azure AI or AWS Machine Learning modules.
• Knowledge of Predictive Analytics and traffic analysis using AI.

Key Skills

Education

Bachelor's degree in Computer Science