Description

1. Azure Data Factory (ADF) – Data Engineering

• Design, develop, deploy, and maintain data pipelines using Azure Data Factory (ADF) to extract, transform, and load data from multiple sources.

• Create and manage ADF pipelines, linked services, datasets, triggers, integration runtimes, parameters, and variables.

• Integrate data from SQL Server, Azure SQL Database, Azure Data Lake Storage (ADLS), PostgreSQL, and other cloud or on-premises systems.

• Perform data extraction, transformation, cleansing, validation, and loading to ensure data accuracy and consistency.

• Implement error-handling, logging, monitoring, and recovery mechanisms for reliable pipeline execution.

• Monitor and troubleshoot pipeline failures, data-loading errors, and database connectivity issues.

• Perform source-to-target data reconciliation and validation to ensure data integrity.

• Optimize SQL queries and database operations to improve performance and reduce execution time.

• Manage code versioning and deployments across development, testing, and production environments using Azure DevOps.

• Collaborate with business analysts, application developers, and infrastructure teams to understand requirements and deliver reliable data solutions.

2. PostgreSQL – Database Administration and Performance Optimization

• Create, configure, maintain, and manage PostgreSQL databases and database objects, including schemas, tables, views, indexes, sequences, functions, and triggers.

• Manage database schemas, including creation, modification, ownership, permissions, and object organization based on application requirements.

• Create and manage database users and roles, configure access controls, and grant or revoke privileges in accordance with security requirements.

• Monitor database health and performance, analyze execution plans using EXPLAIN and EXPLAIN ANALYZE, and optimize SQL queries.

• Identify and troubleshoot slow-running queries, blocking sessions, locking issues, and other database performance bottlenecks.

• Create, modify, and maintain indexes, constraints, and table structures to improve query performance and data integrity.

• Monitor resource utilization, including CPU, memory, disk space, database connections, transaction activity, and storage growth.

• Manage database backup and recovery activities using PostgreSQL utilities such as pg_dump, pg_restore, and pg_basebackup, as applicable.

• Support database migrations, schema changes, release activities, and deployments across development, testing, and production environments.

• Troubleshoot database connectivity, data access, and application-related database issues in coordination with application and infrastructure teams.

Technical Skills

• Azure Data Factory (ADF): Pipeline development, orchestration, linked services, datasets, triggers, integration runtimes, monitoring, and troubleshooting.

• PostgreSQL: Database administration, schema management, SQL, database objects, indexing, query optimization, performance monitoring, user and role management, and backup and recovery.

• SQL: Strong SQL development skills, query tuning, execution plan analysis, joins, subqueries, and performance optimization.

• Azure DevOps: Version control, source code management, and deployment management.

• Data Integration: Experience integrating data across relational databases, cloud storage, and on-premises systems.

*Preferred Qualifications*

• Hands-on experience working with Azure Data Factory and PostgreSQL in development or production environments.

• Understanding of database security, access management, performance tuning, backup and recovery, and deployment best practices.

• Experience troubleshooting data pipeline failures and database performance issues.

• Strong analytical, problem-solving, communication, and collaboration skills

Key Skills

Education

Bachelor's degree