diff --git a/docs/source/using_gridappsd/Developing_Apps.rst b/docs/source/using_gridappsd/Developing_Apps.rst index 1721120..cd6fca4 100644 --- a/docs/source/using_gridappsd/Developing_Apps.rst +++ b/docs/source/using_gridappsd/Developing_Apps.rst @@ -7,7 +7,7 @@ Hosting Application or Service ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Developers can create application and services using GridAPPS-D API and use following instruction to host it with the platform. -For example of application and service working with GridAPPS-D, please see: https://github.com/GRIDAPPSD/gridappsd-sample-app and +For example of application and service working with GridAPPS-D, please see: https://github.com/GRIDAPPSD/gridappsd-sample-app and https://github.com/GRIDAPPSD/gridappsd-state-estimator 1. Create proper folder structure for the application or service. @@ -16,19 +16,19 @@ Following is the recommended structure for applications or services working with For application: :: - + . ├── README.md └── my_app ├── app - │ ├── [application exe or pythod code] + │ ├── [application exe or python code] ├── requirements.txt ├── my_app.config └── setup.py - + For service: :: - + . ├── README.md └── my_service @@ -37,8 +37,8 @@ For service: ├── requirements.txt ├── my_service.config └── setup.py - -Where config file is used by GridAPPS-D to launch the application or service from inside the gridappsd container. + +Where config file is used by GridAPPS-D to launch the application or service from inside the gridappsd container. Example config for application: @@ -81,26 +81,26 @@ Example config for service: 2. Clone the repository https://github.com/GRIDAPPSD/gridappsd-docker (refered to as gridappsd-docker repository) next to this repository (they should both have the same parent folder) :: - + . ├── gridappsd-docker └── gridappsd-sample-app - + 3. Add application or service to platform -In order to add your application/service to the container you will need to modify the docker-compose.yml file included in the gridappsd-docker repository. -Under the gridappsd service there is an example volumes leaf that is commented out. Uncomment and modify these lines to add the path for your application and config file. +In order to add your application/service to the container you will need to modify the docker-compose.yml file included in the gridappsd-docker repository. +Under the gridappsd service there is an example volumes leaf that is commented out. Uncomment and modify these lines to add the path for your application and config file. Adding these lines will mount the application/service on the container's filesystem when the container is started. For application: :: - + # volumes: # - ~/git/gridappsd-sample-app/sample_app:/gridappsd/applications/sample_app # - ~/git/gridappsd-sample-app/sample_app/sample_app.config:/gridappsd/applications/sample_app.config - + volumes: - ~/git/[my_app_directory]/[my_app]:/gridappsd/applications/[my_app] - ~/git/[my_app_directory]/[my_app]/[my_app.config]:/gridappsd/applications/[my_app.config] @@ -108,11 +108,11 @@ For application: For service: :: - + # volumes: # - ~/git/gridappsd-sample-app/sample_app:/gridappsd/applications/sample_app # - ~/git/gridappsd-sample-app/sample_app/sample_app.config:/gridappsd/applications/sample_app.config - + volumes: - ~/git/[my_service_directory]/[my_service]:/gridappsd/services/[my_service] - ~/git/[my_service_directory]/[my_service]/[my_service.config]:/gridappsd/services/[my_service.config] diff --git a/docs/source/using_gridappsd/EventClassDiagram.png b/docs/source/using_gridappsd/EventClassDiagram.png new file mode 100644 index 0000000..be16f10 Binary files /dev/null and b/docs/source/using_gridappsd/EventClassDiagram.png differ diff --git a/docs/source/using_gridappsd/index.rst b/docs/source/using_gridappsd/index.rst index 614f62c..0b273d8 100644 --- a/docs/source/using_gridappsd/index.rst +++ b/docs/source/using_gridappsd/index.rst @@ -23,11 +23,11 @@ Applications and services can use either publish/subscribe mechanism or Python A Publish/Subscribe mechanism can be implemented using any of the language bindings for ActiveMQ messaging framework. -Python API wraps the publish/subscribe messaging and makes the interaction easier for Python apps/services. -For more information on Python API and how to use it, look at https://github.com/GRIDAPPSD/gridappsd-python and -https://github.com/GRIDAPPSD/gridappsd-sample-app. +Python API wraps the publish/subscribe messaging and makes the interaction easier for Python apps/services. +For more information on Python API and how to use it, look at https://github.com/GRIDAPPSD/gridappsd-python and +https://github.com/GRIDAPPSD/gridappsd-sample-app. -Following sections describe the messaging APIs and the corresponding Python API function to interact with platform. +Following sections describe the messaging APIs and the corresponding Python API function to interact with platform. Where no Python API function is mentioned, following generic functions can be used. :: @@ -66,3 +66,13 @@ Hosting Application or Service ------------------------------ .. include:: Developing_Apps.rst + +Test Manager Events +------------------- + +.. include:: test_manager_events.rst + +Test Manager +------------ + +.. include:: test_manager.rst diff --git a/docs/source/using_gridappsd/test_manager.rst b/docs/source/using_gridappsd/test_manager.rst new file mode 100644 index 0000000..92de80f --- /dev/null +++ b/docs/source/using_gridappsd/test_manager.rst @@ -0,0 +1,145 @@ +The Test Manager is responsible for testing an application or service against different events or scenarios. +The Test Manager Configuration is sent as part of the RequestSimulation. The expected results and events are sent to the Test Manager for processing. + +1. Test Configuration + +The test configuration contains the application ID, rules settings, expected results, and events. +Other information need by the Test Manager like the simulation ID is obtained by the Simulation Context. + +Example: + +:: + + { + "appId":String + "rules":{} + "expectedResults":{ + }, + "events":{ + } + } + + +2. Expected results series: + +Time series json structure with expected results. + +.. code-block:: JSON + :caption: Expected results + + { + "output": { + "1248130802": { + "simulation_id": "559402036", + "message": { + "timestamp": 1535574871, + "measurements": [ + { + "angle": -122.66883087158203, + "magnitude": 2438.561767578125, + "measurement_mrid": "_84541f26-084d-4ea7-a254-ea43678d51f9" + }, + { + "angle": 21.723935891052907, + "magnitude": 45368.78524042436, + "measurement_mrid": "_c48d8d88-12be-4b15-8b44-eedc752250c6" + }, + { + "measurement_mrid": "_4a316ed2-4e5f-4b8c-9b25-605f5c9e249c", + "value": 0 + } + ] + } + }, + "1248130805": { + "simulation_id": "559402036", + "message": { + "timestamp": 1535574872, + "measurements": [ + { + "angle": -38.381605233862224, + "magnitude": 52769.16136465681, + "measurement_mrid": "_84541f26-084d-4ea7-a254-ea43678d51f9" + }, + { + "angle": 21.723935891052907, + "magnitude": 45368.78524042436, + "measurement_mrid": "_c48d8d88-12be-4b15-8b44-eedc752250c6" + }, + { + "measurement_mrid": "_4a316ed2-4e5f-4b8c-9b25-605f5c9e249c", + "value": 1 + } + ] + } + } + } + + + + +3. Rules + +The rules application is started by the test manager and messages sent to +simulation.input.[simulationId] and simulation.output.[simulationId] will be +forwarded to http://localhost:5000/input/events + +Snippet to listen for changes to ShuntCompensators, i.e. CIM capacitors. + +.. code-block:: python + + shunt_dict = defaultdict(lambda: {'count':0}) + shunt_threshold = 4 + + # A Reverse and a Forward difference is a state change. + @when_all((m.message.reverse_differences.allItems(item.attribute == 'ShuntCompensator.sections')) & ( + m.message.forward_differences.allItems(item.attribute == 'ShuntCompensator.sections'))) + def shunt_change(c): + # consequent + for i,f in enumerate(c.m.message.reverse_differences): + c.post({'shunt_object': f['object'], + 'action': f['attribute'], + 'timestamp': c.m.message.timestamp}) + + @when_all(+m.shunt_object) + def count_shunt_object(c): + shunt_dict[c.m.shunt_object]['count']+=1 + if shunt_dict[c.m.shunt_object]['count'] == shunt_threshold: + print ('Shunt change threshold '+str(shunt_threshold)+' exceeded for shunt object ' + c.m.shunt_object) + send_log_msg('Shunt change threshold '+str(shunt_threshold)+' exceeded for shunt object ' + c.m.shunt_object) + + +5. Request Test message API + +There is a request_test.py python script provided for the sample app in gridappsd-sample-app/sample_app/tests/request_test.py +The request_test script will work outside the docker container and submits a request to run a simulation. +It will wait to capture the returned simulation ID. The simulation ID is set in the +test configuration message and that message is sent to the "goss.gridappsd.test" topic. +This will cause put the test manager into test mode. The test manager will now forward simulation +input and output to the specified port for the rules application. + +The test message contains the following: + +* testConfigPath - Full path to the test config. +* testScriptPath - Full path to the test config. +* rulePort - Port to use for the rules app, the default is 5000. +* topic - topic to use for the rule app, the default is input. +* expectedResult - Full path to the expected result test series data. + +.. code-block:: python + + loc ='/gridappsd/applications/sample_app/tests' + testCfg = {"testConfigPath":loc+"/SampleTestConfig.json", + "testScriptPath":loc+"/SampleTestScript.json", + "simulationID": 1234, + "rulePort": 5000, + "topic":"input", + "expectedResult":loc + "/expected_result_series_filtered_8500.json" + } + + +The script works from outside of the docker container from either an IDE like PyCharm or from the command line. + +.. code-block:: bash + + user@usermachine>python sample_app/tests/request_test.py diff --git a/docs/source/using_gridappsd/test_manager_events.rst b/docs/source/using_gridappsd/test_manager_events.rst new file mode 100644 index 0000000..e86971e --- /dev/null +++ b/docs/source/using_gridappsd/test_manager_events.rst @@ -0,0 +1,342 @@ +There are 3 types events supported by the TestManager and the platform: + + 1. CIM defined fault events, used when a line is down or for taking a piece of equipment out of service. + 2. Communication outage events which simulates measurements or control message outages. + 3. Scheduled command at specific time which sends commands to a piece of equipment. + +There are 2 commands to the TestManager: + + 1. Update + 2. QueryStatus + + +|event_classes_image0| + +Fault Events +^^^^^^^^^^^^ + +Fault Events are defined in a Test Script and define the CIM Fault events that will be intialized and cleared at scheduled times. + +The phases string is all the combinations of the 3-phases plus neutral and secondary phases. +Some examples are : + +1. "A" +2. "AN" would be line to line. +3. "AB" would be line to line. +4. "S12N" both hot wires to ground +5. "S12" both hot wires together. +  +PhaseConnectedFaultKind is an enumeration with the following values: + +1. lineToGround +2. lineToLine +3. lineToLineToGround +4. lineOpen + +.. code-block:: none + :caption: Fault Events in Test Script JSON schema + + { + "PhaseConnectedFaultKind": string, + "FaultImpedance": { + "rGround": float, + "xGround": float, + "rLineToLine":float, + "xLineToLine":float + }, + "ObjectMRID": [string], + "phases": string, + "event_type": string, + "occuredDateTime": long, + "stopDateTime": long + } +.. + + +.. code-block:: JSON + :caption: Fault Event Example in test script + + { + "command": "new_events", + "events" : [{ + "PhaseConnectedFaultKind": "lineToGround", + "FaultImpedance": { + "rGround": 0.001, + "xGround": 0.001 + }, + "ObjectMRID": ["_9EF94B67-7279-21F4-5CEE-B2724E3C3FE6"], + "phases": "ABC", + "event_type": "Fault", + "occuredDateTime": 1248130809, + "stopDateTime": 1248130816 + } + ] + } +.. + + +Fault Commands sent from the Test Manager to the simulation + +.. code-block:: JSON + :caption: Initialize a Fault Command example + + { + "command": "update", + "input": { + "timestamp": 1553201000414, + "reverse_differences": [], + "difference_mrid": "_ee4e4055-222f-4ccf-bed1-93063bd4392c", + "forward_differences": [ + { + "ObjectMRID": "12344", + "FaultImpedance": { + "xLineToLine": 0.0, + "rGround": 0.001, + "rLineToLine": 0.0, + "xGround": 0.001 + }, + "FaultMRID": "1233", + "PhaseCode": "AN", + "PhaseConnectedFaultKind": "lineToGround" + } + ] + } + } +.. + +.. code-block:: JSON + :caption: Clear a Fault Command example + + { + "command": "update", + "input": { + "timestamp": 1553201003561, + "reverse_differences": [ + { + "ObjectMRID": "12344", + "FaultImpedance": { + "xLineToLine": 0.0, + "rGround": 0.001, + "rLineToLine": 0.0, + "xGround": 0.001 + }, + "FaultMRID": "1233", + "PhaseCode": "AN", + "PhaseConnectedFaultKind": "lineToGround" + } + ], + "difference_mrid": "_00b4668d-8454-4f1c-aed9-42d1424af149", + "forward_differences": [] + } + } +.. + +Communication Event +^^^^^^^^^^^^^^^^^^^ + +Communication Outage events are separate from the CIM events but have occuredDateTime and stopDateTime. + +1. The inputOutageList is the list of objectMRID and attribute pair. The objectMRID is anything that can be controllable and specific control attribute i.e. "RegulatingControl.mode". +2. The outputOutageList is the list of MRIDs for the measurement device that is associated to the . +3. If allInputOutage is True the inputOutageList is not needed as all inputs to the simulator are blocked. +4. If allOutputOutage is True the outputOutageList is not needed as all outputs from the simulator are blocked. + +.. code-block:: none + :caption: JSON Communication Outage schema command for the TestManager + + { + "allOutputOutage": boolean, + "allInputOutage": boolean, + "inputOutageList": [{"objectMRID":string, "attribute":string}], + "outputOutageList": [string], + "event_type": string, + "occuredDateTime": long, + "stopDateTime": long + } +.. + +.. code-block:: JSON + :caption: JSON Communication Outage command for the TestManager + + {"command": "new_events", + "events": [ + { + "allOutputOutage": false, + "allInputOutage": false, + "inputOutageList": [{"objectMRID":"_EF2FF8C1-A6A6-4771-ADDD-A371AD929D5B", "attribute":"ShuntCompensator.sections"}, {"objectMRID":"_C0F73227-012B-B70B-0142-55C7C991A343", "attribute":"ShuntCompensator.sections"}], + "outputOutageList": ["_5405BE1A-BC86-5452-CBF2-BD1BA8984093"], + "event_type": "CommOutage", + "occuredDateTime": 1248130819, + "stopDateTime": 1248130824 + } + ] + } +.. + +For reference this is the complete JSON schema of the internal Communication Event for the platform and goes between the Test Manager and the fncs_goss_bridge.py. + +.. code-block:: JSON + :caption: Communication Event to the Simulation Bridge + + { + "command": "CommOutage", + "input": { + "timestamp": 1248130819, + "forward_differences": [ + { + "allOutputOutage": false, + "allInputOutage": false, + "inputOutageList": [ + { + "objectMRID": "_EF2FF8C1-A6A6-4771-ADDD-A371AD929D5B", + "attribute": "ShuntCompensator.sections" + }, + { + "objectMRID": "_C0F73227-012B-B70B-0142-55C7C991A343", + "attribute": "ShuntCompensator.sections" + } + ], + "outputOutageList": [ + "_5405BE1A-BC86-5452-CBF2-BD1BA8984093" + ], + "faultMRID": "_ce5ee4c9-9c41-4f5e-8c5c-f19990f9cfba", + "event_type": "CommOutage", + "occuredDateTime": 1248130819, + "stopDateTime": 1248130824 + } + ], + "reverse_differences": [] + } + } +.. + +The object will be the EventID generated by TestManager + +Updating Events +^^^^^^^^^^^^^^^ + +Events can be updated with the "update_events" command and the list of events to be updated. +This is used to update the occuredDateTime and stopDateTime. + + + +.. code-block:: none + :caption: JSON update command to Test Manager + :emphasize-lines: 6 + + { + "command": "update_events", + "events":[{}] + } + } +.. + +Query +^^^^^ +The Test Manager can be queried the for list of faults and statuses. +The return is the fault MRID and status which can be SCHEDULED, INITIATED, CLEARED, or CANCELLED. + +.. code-block:: none + :caption: Query the for list of faults and status + + {"command":"query_events", "simulationID":int} +.. + +.. code-block:: none + :caption: Result JSON Schema + + { + "data": [ + { + , + "status": # SCHEDULED, INITIATED, CLEARED, CANCELLED + } + ] + } +.. + + +.. code-block:: JSON + :caption: Result CIM Fault Events example + + { + "data": [ + { + "allOutputOutage": false, + "allInputOutage": false, + "inputOutageList": [ + { + "objectMRID": "_EF2FF8C1-A6A6-4771-ADDD-A371AD929D5B", + "attribute": "ShuntCompensator.sections" + }, + { + "objectMRID": "_C0F73227-012B-B70B-0142-55C7C991A343", + "attribute": "ShuntCompensator.sections" + } + ], + "outputOutageList": [ + "_5405BE1A-BC86-5452-CBF2-BD1BA8984093" + ], + "faultMRID": "_ce5ee4c9-9c41-4f5e-8c5c-f19990f9cfba", + "event_type": "CommOutage", + "occuredDateTime": 1248130819, + "stopDateTime": 1248130824, + "status": "CLEARED" + } + ] + } +.. + +Scheduled Commands +^^^^^^^^^^^^^^^^^^ + +Commands that can be scheduled for a specific point in time of the simulation. This can be used to trigger a fault like behavior like change the taps of a regular or mimicking behavior of protective devices like a recloser with a with a switch. + +.. code-block:: none + :caption: JSON scheduled command schema + + { + "command": "new_events", + "events":[{ + "message":{ + "forward_differences":[], + "reverse_differences":[] + }, + "occuredDateTime":long, + "stopDateTime":long, + }] + } +.. + + +.. code-block:: JSON + :caption: Scheduled command example + + { + "command": "new_events", + "events":[{ + "message": { + "forward_differences": [ + { + "object": "_8D0EAC3F-AD56-C5A6-ED03-863DBB4A8C5F", + "attribute": "ShuntCompensator.sections", + "value": "0" + } + ], + "reverse_differences": [ + { + "object": "_8D0EAC3F-AD56-C5A6-ED03-863DBB4A8C5F", + "attribute": "ShuntCompensator.sections", + "value": "1" + } + ] + }, + "event_type": "ScheduledCommandEvent", + "occuredDateTime": 1248130812, + "stopDateTime": 1248130842 + }] + } +.. + + +.. |event_classes_image0| image:: EventClassDiagram.png