Description

You will develop and manage large-scale data processing solutions and distributed systems.

Responsibilities

• Develop data ingestion pipelines from APIs, files, and messaging systems.
• Process and analyze large datasets using distributed systems frameworks.
• Write complex queries against various data stores.
• Optimize data workflows for performance within Hadoop environments.

Required Skills

• 5+ years of experience in Big Data development.
• Strong proficiency in SQL, including MySQL, Hive, and Impala.
• Hands-on experience with Apache Spark and related technologies.
• Familiarity with distributed systems and Hadoop architecture.
• Experience integrating with Kafka and Sqoop.
• Proficiency in Python for scripting and development tasks.
• Knowledge of NoSQL databases, Elasticsearch, and Kibana.
• Working knowledge of Linux environments.

Preferred Skills

• Bachelor's degree in Computer Science or related field.
• Experience with complex query optimization.

Key Skills

Education

Bachelor's degree