Description

You will own the end-to-end lifecycle of traditional databases and big data ETL processes.

Responsibilities

• Build data applications with high accuracy and performance across traditional and distributed computing platforms.
• Develop database processes to gather and process raw data at scale via web scraping, APIs, and SQL queries.
• Administer data processing workflows using tools such as MySQL, Oozie, ZooKeeper, Sqoop, Hive, and Impala.
• Manage deployments of big data clusters across private and public cloud platforms.
• Perform routine monitoring, backups, and check-ups of the MySQL and Hadoop ecosystem.

Required Skills

• 4+ years of experience with SQL and MySQL.
• 2+ years of hands-on experience with Cloudera Hadoop Distribution and Apache Spark.
• Proficiency in Java, Python, or Scala.
• Working expertise with Apache Spark, Spark Streaming, and Jupyter Notebook.
• Experience with Hive, Impala, Kafka, HBase, Pig, and Map-Reduce.
• Strong understanding of Hadoop Architecture and HDFS.
• Ability to write SQL queries, DDL/DML statements, stored procedures, triggers, and functions.
• Experience processing large volumes of structured and unstructured data from relational databases or distributed file systems.
• Knowledge of database administration, performance tuning, replication, and data restoration.
• Proficiency with BI tools such as Looker Studio or Power BI.
• Bachelor's degree in Computer Science or a related field.

Preferred Skills

• Experience with Linux, GitHub, Jira, and CI/CD pipelines.
• Familiarity with caching, load balancers, containerization, and distributed logging.
• Knowledge of REST, TDD, and JUnit.

Key Skills

Education

Bachelor's degree