Description

Build and maintain scalable data pipelines for ingestion, transformation, and integration. Optimize data storage and processing on cloud and on-premise infrastructure. Design data models and architectures that support business needs and ensure high data quality through validation and monitoring. Collaborate with data science and analytics teams to deliver accessible datasets. Document data processes and standards to maintain consistency.

Responsibilities

• Design and develop scalable, reliable data pipelines for multi-source integration.
• Optimize data storage, retrieval, and processing efficiency across cloud and on-premise environments.
• Implement data validation, cleansing, and monitoring practices to ensure accuracy.
• Collaborate with data scientists and analysts to translate business requirements into technical solutions.
• Maintain comprehensive documentation for data processes and dictionaries.

Required Skills

• 7+ years of data engineering experience building large-scale data pipelines.
• Proficiency in SQL, Python, and/or Scala.
• Experience with data processing frameworks like Spark or Hadoop.
• Knowledge of cloud platforms (AWS, GCP, or Azure).
• Understanding of data modeling, warehousing concepts, and data architecture best practices.
• Strong problem-solving skills with attention to detail regarding data quality and consistency.

Preferred Skills

• Experience working in complex, fast-paced environments.
• Ability to adapt to new tools and methodologies.

Key Skills

Education

Any Graduate