PI-MCTS#342
Conversation
vadithiyan
commented
Oct 27, 2025
vadithiyan
commented
- Added PISingleTreeNode
- Modified SingleTreeNode to handle PI-MCTS
- Modified MCTSEnums to include Perfect Inforamtion as an Information Policy and added A new enum for Perfect Information Aggregation Policy
- Added NumDeterminizations and PerfectInformationPolicy as Tunable parameters to MCTSParams
- Modified MCTSPlayer to use PISingleTreeNode when Perfect Information is selected as information policy
- Added PIMCTSMetrics class
Implemented PI-MCTS and Four aggregation Policies
hopshackle
left a comment
hopshackle
left a comment
There was a problem hiding this comment.
Good - I;m being a bit fussy with some suggested changes.
The main one is that we should not have Perfect_Information as a new enum on Information, but just add a new parameter (num_determinizations) that extends the current Open_Loop / Closed_Loop information enums.
A secondary point is some refactoring of PISingleTreeNode to change its name, and streamline bits of the code.
|
|
||
| public enum Information { | ||
| Closed_Loop, Open_Loop, Information_Set | ||
| Closed_Loop, Open_Loop, Information_Set, Perfect_Information |
There was a problem hiding this comment.
Do we need a separate enum here. Isn;t this the same as Open_Loop (with a default of 1 determinisation)
| root = SingleTreeNode.createRootNode(this, gameState, rnd, getFactory()); | ||
| else { | ||
| if(getParameters().information == MCTSEnums.Information.Perfect_Information) | ||
| { |
There was a problem hiding this comment.
This then is if num_determinisations > 1
| import static players.PlayerConstants.*; | ||
|
|
||
| public class PISingleTreeNode extends SingleTreeNode { | ||
| SingleTreeNode[] roots; |
There was a problem hiding this comment.
A better name for this would be ForestNode (or something that indicates it constructs a number of different trees, and then aggregates results across them)
| int numDeterminizations = 1; | ||
| MCTSPlayer mctsPlayer; | ||
| double epsilon = 1e-6; | ||
| //Action Stats variable to hold aggregated values |
There was a problem hiding this comment.
Use the value from MCTSParams
| roots[i].mctsSearch(initialisationTime); | ||
| } | ||
| } | ||
| //Returns the best action after search based on the chosen AggrergationPolicy |
There was a problem hiding this comment.
I reckon we can rely on the mctsSearch in the SingleTreeNode for this (and reduce code repetition a little)
| else | ||
| root = SingleTreeNode.createRootNode(this, gameState, rnd, getFactory()); | ||
| else { | ||
| if(getParameters().information == MCTSEnums.Information.Perfect_Information) |
There was a problem hiding this comment.
Also, this doesn't take account of reuseTree (not the end of the world, but I wonder if we could implement this)
| SingleTreeNode[] roots; | ||
| AbstractGameState state; | ||
| int numDeterminizations = 1; | ||
| MCTSPlayer mctsPlayer; |
There was a problem hiding this comment.
We shoudl just read this directly from Params
| { | ||
| root = new PISingleTreeNode(this, gameState, rnd); | ||
| } | ||
| else { |
There was a problem hiding this comment.
A better form might be:
root = ForestNode.createRootNode(...) to keep the same pattern as for SingleTree node a couple of lines below [and then I can go an refactor MultiTreeNode to the same pattern]
| } | ||
|
|
||
| public enum PerfectInformationPolicy{ | ||
| SingleVote, TotalValue, AverageValue, TotalVisits |
There was a problem hiding this comment.
Also - remove the TotalValue code, as we know it is a bit theoretically suspect
Done : - Renamed from PISingleTreeNode to ForestNode - Perfect_Information Enum removed, ForestNode is created when numbDeterminizations>1 - Refactored to use noise epsilon from MCTSParams instead of creating a speerate epsilon definition - Refactored to read numDeterminization from MCTSParams instead of creating a local copy - TotalValue aggregation policy purged TODO : - 'reuseTree' Implementation - Change from using constructor to create ForestNode to using ForestNode.createRootNode(...)
Fixed issues with MAST Statistic accumulation when using Determinized Trees
2 participants
