Description

Lead data engineering efforts and architect data-driven solutions.

Responsibilities

• Design and implement scalable data pipelines using ETL fundamentals.
• Develop software using Python and Java within complex ecosystems.
• Build and maintain data models and SQL database designs.
• Apply data mining and segmentation techniques to large datasets.
• Collaborate with teams to integrate machine learning and data science concepts.

Required Skills

• 10+ years of experience in data engineering or a similar role.
• Expertise in Python and Java programming.
• Proficiency with Python frameworks including Django, Flask, or Pyramid.
• Deep knowledge of Python stacks: Numpy, SciPy, Pandas, Dask, spaCy, NLTK, scikit-learn, and PyTorch.
• Hands-on experience with Spark, PySpark, HDFS, Hive, MapReduce, and Oozie.
• Strong SQL database design and PL/SQL skills.
• Familiarity with NoSQL databases.
• Understanding of cloud platforms such as AWS, Google Cloud, or Azure.
• Bachelor's Degree in a relevant field.

Preferred Skills

• Experience with Databricks, Flink, or NoSQL databases.
• Data engineering certification.
• Contributions to open-source Python projects.

Key Skills

Education

Bachelor's Degree