Description

Senior Data Engineer based in Virginia, USA.

Responsibilities

• Design and implement large-scale data architectures and distributed computing solutions.
• Implement and tune Hadoop and Spark workloads to optimize big data processing.
• Develop data pipelines using Python and PySpark within AWS environments.
• Build and maintain data storage solutions including data lakes and data warehouses.
• Write automated unit, integration, regression, performance, and acceptance tests.

Required Skills

• 10+ years of professional experience in data engineering or related fields.
• 5+ years of development experience using Python and SQL.
• 3+ years of hands-on experience implementing and tuning Hadoop and Spark.
• Extensive experience with Amazon EMR and AWS distributed computing services.
• Proficiency in SQL with familiarity in Postgres or Snowflake.
• Strong understanding of object-oriented programming and API integration.
• Experience with Boto3 for AWS service interaction.
• Bachelor’s degree in Computer Science, Data Science, or a related field.

Preferred Skills

• Experience with data lake patterns and enterprise-scale data warehousing.

Key Skills

Education

Any Graduate