Description

You will design and develop large-scale data batch and near-real-time processing solutions.

Responsibilities

• Design and develop data solutions for large-scale batch and near-real-time processing.
• Develop and debug data logic using SQL, Python, and Scala.
• Solve technical challenges involving high throughput, low latency, and high availability using big data technologies.
• Maintain the stability of big data components under high concurrency and extreme throughput.
• Conduct technical feasibility tests to support early-stage data projects.

Required Skills

• Minimum 5 years of experience in data engineering or related roles.
• Proficiency in Spark SQL, HiveQL, and Python data processing.
• Hands-on experience with Spark, Hive, Flink, HBase, and Elasticsearch.
• Experience with mainstream databases including Oracle and DB2.
• Strong skills in data modeling, database storage development, and SQL tuning.
• Knowledge of big data development and job scheduling processes.
• Ability to perform data query performance analysis.
• Bachelor’s degree or higher in Computer Science or a related major.

Preferred Skills

• Experience with terabyte-level data processing.
• Experience designing large-scale data platforms or data warehouses.
• Experience with Microsoft Azure data services including ADF, Azure Databricks, and Azure Synapse Analytics.

Key Skills

Education

Any Graduate