← Back to jobs

United IT Solutions Logo
Data Engineer

United IT Solutions

 

Irving, TX, USA

Posted On: Just posted
Experience: 5+ years
Availability: Onsite
Openings: 2
Category: Data Engineer
Tenure: No Preference/Any
Related Jobs

No related jobs found

Description

You will design, build, and optimize scalable data pipelines for massive datasets.

This role is on-site.

Responsibilities

  • Design and maintain ETL/ELT pipelines using PySpark, Spark SQL, and Hive.
  • Manage data layout, partitioning, and indexing in Apache Hive and data lakes.
  • Tune Spark job performance by resolving bottlenecks, data skew, and memory issues.
  • Ingest high-volume structured and unstructured data from various sources.
  • Orchestrate automated workflows using Apache Airflow or platform schedulers.

Required Skills

  • 5+ years experience with Apache Spark architecture, drivers, executors, and DAGs.
  • Strong Python coding skills with extensive PySpark API experience.
  • Proficiency in HiveQL, ANSI SQL, and data partitioning techniques.
  • Experience with optimized storage formats: Parquet, ORC, and Avro.
  • Knowledge of dimensional data modeling, including Star and Snowflake schemas.
  • Hands-on experience with Apache Airflow for workflow orchestration.
  • Familiarity with CI/CD tools like Git and Jenkins.

Preferred Skills

  • Experience with cloud-native big data utilities (AWS EMR, Databricks).
  • Exposure to NoSQL databases such as HBase, Cassandra, or MongoDB.

Education

Any Graduate

Related Jobs

No related jobs found

← Back to jobs