Added test manager documentation.

docs/source/using_gridappsd/Developing_Apps.rst

@@ -7,7 +7,7 @@ Hosting Application or Service
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Developers can create application and services using GridAPPS-D API and use following instruction to host it with the platform.
-For example of application and service working with GridAPPS-D, please see: https://github.com/GRIDAPPSD/gridappsd-sample-app and 
+For example of application and service working with GridAPPS-D, please see: https://github.com/GRIDAPPSD/gridappsd-sample-app and
 https://github.com/GRIDAPPSD/gridappsd-state-estimator
 
 1. Create proper folder structure for the application or service.
@@ -16,19 +16,19 @@ Following is the recommended structure for applications or services working with
 
 For application:
 ::
-		
+
 	.
 	├── README.md
 	└── my_app
 	    ├── app
-	    │   ├── [application exe or pythod code]
+	    │   ├── [application exe or python code]
 	    ├── requirements.txt
 	    ├── my_app.config
 	    └── setup.py
-	    
+
 For service:
 ::
-		
+
 	.
 	├── README.md
 	└── my_service
@@ -37,8 +37,8 @@ For service:
 	    ├── requirements.txt
 	    ├── my_service.config
 	    └── setup.py
-	    
-Where config file is used by GridAPPS-D to launch the application or service from inside the gridappsd container. 
+
+Where config file is used by GridAPPS-D to launch the application or service from inside the gridappsd container.
 
 Example config for application:
 
@@ -81,38 +81,38 @@ Example config for service:
 2. Clone the repository https://github.com/GRIDAPPSD/gridappsd-docker (refered to as gridappsd-docker repository) next to this repository (they should both have the same parent folder)
 
 ::
-	
+
 	.
 	├── gridappsd-docker
 	└── gridappsd-sample-app
-	
+
 
 3. Add application or service to platform
 
-In order to add your application/service to the container you will need to modify the docker-compose.yml file included in the gridappsd-docker repository. 
-Under the gridappsd service there is an example volumes leaf that is commented out. Uncomment and modify these lines to add the path for your application and config file. 
+In order to add your application/service to the container you will need to modify the docker-compose.yml file included in the gridappsd-docker repository.
+Under the gridappsd service there is an example volumes leaf that is commented out. Uncomment and modify these lines to add the path for your application and config file.
 Adding these lines will mount the application/service on the container's filesystem when the container is started.
 
 For application:
 
 ::
-	
+
 	#    volumes:
 	#      - ~/git/gridappsd-sample-app/sample_app:/gridappsd/applications/sample_app
 	#      - ~/git/gridappsd-sample-app/sample_app/sample_app.config:/gridappsd/applications/sample_app.config
-	
+
 	    volumes:
 	      - ~/git/[my_app_directory]/[my_app]:/gridappsd/applications/[my_app]
 	      - ~/git/[my_app_directory]/[my_app]/[my_app.config]:/gridappsd/applications/[my_app.config]
 
 For service:
 
 ::
-	
+
 	#    volumes:
 	#      - ~/git/gridappsd-sample-app/sample_app:/gridappsd/applications/sample_app
 	#      - ~/git/gridappsd-sample-app/sample_app/sample_app.config:/gridappsd/applications/sample_app.config
-	
+
 	    volumes:
 	      - ~/git/[my_service_directory]/[my_service]:/gridappsd/services/[my_service]
 	      - ~/git/[my_service_directory]/[my_service]/[my_service.config]:/gridappsd/services/[my_service.config]	 

docs/source/using_gridappsd/index.rst

@@ -23,11 +23,11 @@ Applications and services can use either publish/subscribe mechanism or Python A
 
 Publish/Subscribe mechanism can be implemented using any of the language bindings for ActiveMQ messaging framework.
 
-Python API wraps the publish/subscribe messaging and makes the interaction easier for Python apps/services. 
-For more information on Python API and how to use it, look at  https://github.com/GRIDAPPSD/gridappsd-python and 
-https://github.com/GRIDAPPSD/gridappsd-sample-app. 
+Python API wraps the publish/subscribe messaging and makes the interaction easier for Python apps/services.
+For more information on Python API and how to use it, look at  https://github.com/GRIDAPPSD/gridappsd-python and
+https://github.com/GRIDAPPSD/gridappsd-sample-app.
 
-Following sections describe the messaging APIs and the corresponding Python API function to interact with platform. 
+Following sections describe the messaging APIs and the corresponding Python API function to interact with platform.
 Where no Python API function is mentioned, following generic functions can be used.
 
 ::
@@ -66,3 +66,13 @@ Hosting Application or Service
 ------------------------------
 
 .. include:: Developing_Apps.rst
+
+Test Manager Events
+-------------------
+
+.. include:: test_manager_events.rst
+
+Test Manager
+------------
+
+.. include:: test_manager.rst

docs/source/using_gridappsd/test_manager.rst

@@ -0,0 +1,145 @@
+The Test Manager is responsible for testing an application or service against different events or scenarios.
+The Test Manager Configuration is sent as part of the RequestSimulation. The expected results and events are sent to the Test Manager for processing.
+
+1. Test Configuration
+
+The test configuration contains the application ID, rules settings, expected results, and events.
+Other information need by the Test Manager like the simulation ID is obtained by the Simulation Context.
+
+Example:
+
+::
+
+  {
+    "appId":String
+    "rules":{}
+    "expectedResults":{
+    },
+    "events":{
+    }
+  }
+
+
+2. Expected results series:
+
+Time series json structure with expected results.
+
+.. code-block:: JSON
+   :caption: Expected results
+
+    {
+    "output": {
+        "1248130802": {
+        "simulation_id": "559402036",
+        "message": {
+            "timestamp": 1535574871,
+            "measurements": [
+            {
+                "angle": -122.66883087158203,
+                "magnitude": 2438.561767578125,
+                "measurement_mrid": "_84541f26-084d-4ea7-a254-ea43678d51f9"
+            },
+            {
+                "angle": 21.723935891052907,
+                "magnitude": 45368.78524042436,
+                "measurement_mrid": "_c48d8d88-12be-4b15-8b44-eedc752250c6"
+            },
+            {
+                "measurement_mrid": "_4a316ed2-4e5f-4b8c-9b25-605f5c9e249c",
+                "value": 0
+            }
+            ]
+        }
+        },
+        "1248130805": {
+        "simulation_id": "559402036",
+        "message": {
+            "timestamp": 1535574872,
+            "measurements": [
+            {
+                "angle": -38.381605233862224,
+                "magnitude": 52769.16136465681,
+                "measurement_mrid": "_84541f26-084d-4ea7-a254-ea43678d51f9"
+            },
+            {
+                "angle": 21.723935891052907,
+                "magnitude": 45368.78524042436,
+                "measurement_mrid": "_c48d8d88-12be-4b15-8b44-eedc752250c6"
+            },
+            {
+                "measurement_mrid": "_4a316ed2-4e5f-4b8c-9b25-605f5c9e249c",
+                "value": 1
+            }
+            ]
+        }
+        }
+    }
+
+
+
+
+3. Rules
+
+The rules application is started by the test manager and messages sent to
+simulation.input.[simulationId] and simulation.output.[simulationId] will be
+forwarded to http://localhost:5000/input/events
+
+Snippet to listen for changes to ShuntCompensators, i.e. CIM capacitors.
+
+.. code-block:: python
+
+  shunt_dict = defaultdict(lambda: {'count':0})
+  shunt_threshold = 4
+
+  # A Reverse and a Forward difference is a state change.
+  @when_all((m.message.reverse_differences.allItems(item.attribute == 'ShuntCompensator.sections')) & (
+  m.message.forward_differences.allItems(item.attribute == 'ShuntCompensator.sections')))
+  def shunt_change(c):
+      # consequent
+      for i,f in enumerate(c.m.message.reverse_differences):
+          c.post({'shunt_object': f['object'],
+                  'action': f['attribute'],
+                  'timestamp': c.m.message.timestamp})
+
+  @when_all(+m.shunt_object)
+  def count_shunt_object(c):
+      shunt_dict[c.m.shunt_object]['count']+=1
+      if shunt_dict[c.m.shunt_object]['count'] == shunt_threshold:
+          print ('Shunt change threshold '+str(shunt_threshold)+' exceeded for shunt object ' + c.m.shunt_object)
+          send_log_msg('Shunt change threshold '+str(shunt_threshold)+' exceeded for shunt object ' + c.m.shunt_object)
+
+
+5. Request Test message API
+
+There is a request_test.py python script provided for the sample app in gridappsd-sample-app/sample_app/tests/request_test.py
+The request_test script will work outside the docker container and submits a request to run a simulation.
+It will wait to capture the returned simulation ID. The simulation ID is set in the
+test configuration message and that message is sent to the "goss.gridappsd.test" topic.
+This will cause put the test manager into test mode. The test manager will now forward simulation
+input and output to the specified port for the rules application.
+
+The test message contains the following:
+
+* testConfigPath - Full path to the test config.
+* testScriptPath - Full path to the test config.
+* rulePort - Port to use for the rules app, the default is 5000.
+* topic - topic to use for the rule app, the default is input.
+* expectedResult - Full path to the expected result test series data.
+
+.. code-block:: python
+
+  loc ='/gridappsd/applications/sample_app/tests'
+  testCfg = {"testConfigPath":loc+"/SampleTestConfig.json",
+          "testScriptPath":loc+"/SampleTestScript.json",
+          "simulationID": 1234,
+          "rulePort": 5000,
+          "topic":"input",
+          "expectedResult":loc + "/expected_result_series_filtered_8500.json"
+          }
+
+
+The script works from outside of the docker container from either an IDE like PyCharm or from the command line.
+
+.. code-block:: bash
+
+  user@usermachine>python sample_app/tests/request_test.py