Description

You will build and manage systems and pipelines for efficient data ingestion, processing, and storage.

Responsibilities

• Build and manage data pipelines for reliable ingestion, processing, and storage from various sources.
• Implement and maintain ETL processes to ensure data accuracy, completeness, and consistency.
• Design and manage data storage systems, including relational databases, NoSQL databases, and data warehouses.
• Optimize infrastructure, workflows, and pipelines for performance and scalability.
• Monitor systems for errors and anomalies, implementing solutions to address performance issues.

Required Skills

• 5+ years of experience as a Backend Data Engineer or in a similar role.
• Proficiency in Python, Java, Scala, or SQL.
• Hands-on experience with Python libraries including pandas, NumPy, and Spark.
• Experience using Apache Airflow for managing data pipelines and workflows.
• Strong understanding of ETL processes and data warehousing concepts.
• Experience with big data technologies such as Hadoop, Spark, Kafka, and Flink.
• Knowledge of relational databases (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., MongoDB, Cassandra).
• Familiarity with engineering best practices including source control, automated testing, CI/CD, and peer review.
• Bachelor's degree in Computer Science, Information Technology, or a related field.

Preferred Skills

• Experience with cloud computing platforms.
• Familiarity with agile development methodologies and software design patterns.

Key Skills

Education

Any Graduate