Repository for Open Source EMF implementation proof of concept
Incoming IGM-s need to be validated by EMF tool:
- To ensure that the IGM-s can be imported to the tool
- It is possible to solve the LoadFlow
- To provide feedback for TSO-s on QAS portal
- To check and store the AC NP and HVDC flows data
IGM validation process should run continuously:
- To give TSO-s continuous and timely feedback on their IGM, witch enables them to update or fix IGM before the merging process starts
- To achieve better performance during the merging process, as all IGM-s would be already validated by that time
If IGM is deemed unusable:
- TSO can either send a new IGM before the gate closure or
- RCC can use replacement logic to use an older IGM from different timestamp or process
IGM replacement logic is defined in EMF requirements document.
TODO - That logic should be put to a configuration file.
Purpose is to create one electrical grid model from all available models for a given Scenario Time
- Retrieve all valid IGM-s (including replaced IGM-s) for given merge type and scenario time
- Retrieve the latest valid BDS (Boundary Set)
- Retrieve all reference schedules values for given scenario time
- Import all files to Load Flow tool
- Switch off or set to 0 matched cim:EquivalentInjection-s on matched borders
- Proceed to Scaling
-
Scale the Loads to match each Area AC NP to the reference program (the current state is recorded during IGM Validation)
-
Set the HVDC flows to match reference program
-
Run fist LoadFlow
-
Continue Scaling Loads until desired mismatch achieved
-
TODO: copy from EMF dock the exact process
-
TODO: describe when to use relaxed PF settings
- EMF Requirements
- EMF Assessment
- Quality of Datasets
TODO: add links
- Must be able to import CIM XML files (EQ, SSH, TP, SV)
- Able to use CIM LoadFlow settings
- Export CIM Merged model (N x SSH, SV)
- Store and Edit Process configurations
- Store and Edit Time Frame configurations
- TODO: add description tables
Timeframe Configuration Example
Timeframe Configuration Schema
- Store Process configurations
- Responsible for triggering automatic tasks
- Check if there are any tasks to be run based on valid Process Configurations
- Check if there are any event that should trigger a task to be run
- Enable manual creation of Tasks
- Publish Tasks to Service Bus
- TODO: Add LF settings
- TODO: Add Reference Schedule
- TODO: Should replacement logic be linked here as well?
- TODO: Add description of the format and schema
- TODO: Add Validation task example
- Manage and monitor the task trough out the task life cycle
This system consists of two separate but integrates systems
- Metadata Storage:
- Store all extracted metadata about Models
- Store reference to original data stored in Object Storage
- Store reference schedules as searchable documents per each timestamp in timeseries TODO - Add example
- ELK stack is planned to be used
- Object Storage:
- Store all original data (Reference Schedules and Models)
- MinIO is planned to be used (any S3 compatible storage is suitable)
Asynchronous message exchange platform that allows publish and subscribe functionality
- Tasks are published
- Each worker will create a que and subscribe to tasks
- All data received via EDX and OPDM is published
- Object Storage and Metadata Storage will subscribe and store the data
- All generated data will be published
- Routed to EDX or OPDM
- Stored by Object Storage and Metadata Storage
Integration between message bus and EDX. This integration is needed to
- Receive Reference Schedules
- Receive IGM-s and Boundary Set published by OPDM Service Provider
- Publish IGM Validation Reports
- Publish CGM Validation Reports
Integration between message bus and OPDM. This integration is needed to:
- Subscribe for IGM-s and Boundary Set
- Publish CGM-s
- Language: Python 3.11
- LoadFlow Engine: PyPowSyBl
- Object Storage: MinIO
- Metadata Storage: ELK
- Logging: ELK
- Service Bus: RabbitMQ
- Process Configurator: GIT
- Process Scheduler: Jenkins and custom scripts
- Task Manager: ELK and custom scripts
- TODO