Description

You will build and manage scalable data solutions and end-to-end pipelines.

Responsibilities

• Design and implement data pipelines and ETL processes.
• Profile datasets and develop scripts to improve data quality and integrity.
• Monitor and optimize pipelines for performance and reliability.
• Collaborate with cross-functional teams to solve complex data problems.
• Work with large datasets to design scalable architectures.

Required Skills

• 5+ years of experience as a Data Engineer.
• Strong SQL skills.
• Strong Python skills with experience using Python data libraries like Pandas.
• Bachelor's degree in Computer Science.
• Experience working with large datasets and scalable data solutions.
• Proven ability to optimize data pipelines for performance.

Preferred Skills

• Experience with PySpark, Spark, Hadoop, or Parquet.
• Experience with Azure Synapse Analytics or Azure Data Factory.
• Experience with Apache NiFi.

Key Skills

Education

Bachelor's degree in Computer Science