Description
Key Skills: Python, SQL, REST API, Databricks, Apache Spark, DBT, FastAPI, Cloud Platforms, Data Pipelines, Problem Solving
Good to Have Skills: Delta Lake, LLM frameworks, RAG architecture, LangChain, LangGraph, OpenAI APIs, Claude Code, Cursor, US energy markets knowledge, ISO/RTO operations, Indian power sector understanding, IEX, DISCOM data structures, CI/CD pipelines, GitOps workflows, DevOps practices, observability tools like CloudWatch, Grafana, Elastic, Datadog.
Roles & Responsibilities:
- Design and build data pipelines ingesting feeds from US energy markets including ISO/RTO data, utility analytics, and regulatory intelligence sources.
- Develop analytics and reporting infrastructure for US consulting clients, turning complex market data into clear actionable outputs.
- Integrate AI and LLM capabilities to automate report generation, surface market signals, and accelerate analyst workflows.
- Design and build data ingestion pipelines processing energy consumption and exchange data from DISCOMs, IEX, and client sources.
- Develop and maintain DBT models for data transformation, testing, and layered analytics adhering to best practices.
- Architect and implement backend components for Exchange Bid Automation system including REST APIs and scheduling logic.
- Own the full lifecycle of assigned solutions from design through production support including monitoring and alerting.
- Proactively identify and resolve production issues, conducting root cause analysis and implementing preventive measures.
- Leverage observability tools and cloud-native monitoring to track pipeline health, data freshness, and system uptime.
- Work closely with product, analysts, and domain experts to translate energy market requirements into robust technical solutions.
Experience Required: 2-5 years of professional software or data engineering experience with strong proficiency in Python and SQL for building scalable applications and querying large datasets. Experience building or maintaining data pipelines with willingness to ramp on Databricks and DBT quickly.
Education: Bachelor's degree in Computer Science, Software Engineering, or related field (or equivalent professional experience)