Description

Key Responsibilities

Data Engineering & Integration

• Design, develop, maintain, and optimize ETL/ELT pipelines supporting forecasting, telemetry, weather, market, asset, and operational data.
• Develop and maintain automated data ingestion and transformation processes across APIs, databases, flat files, SFTP, cloud storage, and external data providers.
• Build scalable and reusable Python and SQL-based data processing frameworks.
• Support PostgreSQL and other relational database environments used by forecasting applications and analytical workflows.
• Improve data pipeline monitoring, logging, restart ability, validation, and failure recovery.
• Implement data-quality checks, reconciliation processes, and source-to-target validation.
• Support onboarding and integration of new wind, solar, storage, and other generation assets into forecasting data pipelines.
• Partner with Data Scientists to ensure reliable, timely, and well-structured data is available for model development, training, production forecasting, and performance analysis.
• Support modernization and standardization of legacy data workflows and reduce manual or hard-coded processing wherever practical.

Forecasting Platform & Operational Support

• Provide secondary technical support for production load, wind, and solar forecasting applications and associated data-delivery processes.
• Troubleshoot production issues across application, data, database, cloud infrastructure, integration, and delivery layers.
• Support AWS-hosted forecasting workloads and services such as S3, Lambda, ECS, EC2, CloudWatch, and related cloud resources.
• Assist with application deployments, release validation, configuration changes, operational readiness, and post-deployment monitoring.
• Support scheduled jobs, automated forecast processes, reporting services, and customer-facing forecast delivery workflows.
• Assist with SFTP services, secure file transfers, SSH keys, directory configurations, and customer data-exchange processes.
• Participate in incident response, root-cause analysis, change management, service-ticket resolution, and production support.
• Maintain sufficient cross-functional knowledge to provide backup coverage for key forecasting platform responsibilities when the primary platform resource is unavailable.

Reliability, Automation & Documentation

• Identify recurring manual processes and develop automation to improve reliability and reduce operational effort.
• Establish effective monitoring and alerting for pipelines, scheduled processes, data feeds, and production forecasting applications.
• Document system dependencies, troubleshooting procedures, support processes, recovery steps, and operational handoffs.
• Participate in cross-training and knowledge-sharing activities to ensure critical systems have appropriate primary and backup technical coverage.
• Support continuous improvement of the team's operational support model and technical resiliency.

Required Technical Experience

• Strong experience designing, developing, and supporting enterprise ETL/ELT or data-integration pipelines.
• Advanced SQL skills and experience with relational databases such as PostgreSQL, Oracle, SQL Server, or similar platforms.
• Proficiency with Python for data processing, automation, integration, or application-support activities.
• Experience integrating data from APIs, databases, flat files, cloud storage, SFTP, or other enterprise data sources.
• Experience supporting production data pipelines, scheduled jobs, and troubleshooting failed processing workflows.
• Experience working with AWS or comparable cloud environments.
• Strong understanding of logging, monitoring, troubleshooting, root-cause analysis, and production-support practices.
• Experience with Git-based source control and modern software-development practices.

Preferred Qualifications

• Experience with AWS services including S3, Lambda, ECS, EC2, CloudWatch, Step Functions, or related services.
• Experience with PostgreSQL/Aurora, Redshift, Databricks, Spark/PySpark, or similar data platforms.
• Experience supporting CI/CD pipelines, application deployments, or cloud-hosted production applications.
• Experience with SFTP administration, secure data exchange, SSH key management, or automated file-transfer processes.
• Experience building data-quality, monitoring, reconciliation, and automated alerting frameworks.
• Familiarity with energy markets, utility operations, renewable generation, SCADA/telemetry data, weather data, or load/wind/solar forecasting.
• Experience supporting machine-learning or forecasting data pipelines is strongly preferred.
• Experience working in Agile environments using tools such as Jira and Confluence

Key Skills

Education

Any Gradute