Description

You will manage and optimize batch processes and data pipelines in Azure Databricks and Azure Data Factory, integrating Bloomberg market data feeds into trading and analytics platforms. You will develop, automate, and troubleshoot ETL pipelines using Python, Spark, and SQL to ensure data quality and timeliness for downstream systems. Your work includes managing FTP/SFTP transfers, resolving data incidents, and collaborating with operations teams to enhance data workflows.

This role is on-site.

Responsibilities

• Develop and optimize ETL pipelines using Python, Spark, and SQL within Azure Databricks and Azure Data Factory.
• Integrate and process Bloomberg market data feeds (BPIPE, TSIP) and files into trading and analytics platforms.
• Monitor, troubleshoot, and resolve data and system issues related to trading applications and market data ingestion.
• Manage FTP/SFTP file transfers between internal systems and external vendors.
• Collaborate with operations, application support, and infrastructure teams to resolve incidents and enhance data workflows.

Required Skills

• 10+ years of experience in data engineering or production support within financial services or trading environments.
• Hands-on experience with Azure Databricks, Azure Data Factory, and Azure Storage.
• Strong programming skills in Python and SQL.
• Experience with Azure Logic Apps and Fabric.
• Experience with Spark for distributed data processing.
• Proven track record of managing batch processes and data pipelines at scale.

Preferred Skills

• Demonstrated experience with Bloomberg data feeds (BPIPE, TSIP, SFTP).

Key Skills

Education

Any Graduate