Description

You will build and maintain large-scale data pipelines and integration layers.

Responsibilities

• Design and implement data pipelines using Databricks to process large volumes of data.
• Develop and optimize T-SQL queries, stored procedures, and functions for ETL processes.
• Automate data tasks and workflows using Python or Java.
• Build and maintain RESTful APIs for secure and scalable data access.
• Monitor pipelines to resolve issues, ensuring data integrity and implementing governance.

Required Skills

• 8+ years of experience in data engineering.
• Strong T-SQL (SQL Server) development expertise.
• Proficiency in Databricks, including Spark SQL and Data Frame API.
• Experience developing RESTful APIs using Flask or Spring Boot.
• Proficiency in Python or Java for automation and scripting.
• Solid understanding of data warehousing, dimensional modeling, and ETL practices.
• Experience with cloud platforms including AWS, Azure, or GCP (S3, Redshift, or BigQuery).

Preferred Skills

• Master's degree in a technical field.

Key Skills

Education

Any graduate