@@ -30,10 +30,10 @@ Gives a global observation of the entire rail environment.
The observation is composed of the following elements:
- transition map array with dimensions (``env.height``, ``env.width``, ``16``), assuming **16 bits encoding of transitions**.
- Two 2D arrays (``map_height``, ``map_width``, ``2``) containing respectively the position of the given agent target and the positions of the other agents targets.
- A 3D array (``map_height``, ``map_width``, ``8``) with the **4 first channels** containing the **one hot encoding** of the direction of the given agent and the 4 second channels containing the positions of the other agents at their position coordinates.
- Two 2D arrays (``map_height``, ``map_width``, ``2``) containing respectively the position of the given agent target and the positions of the other agents' targets.
- A 3D array (``map_height``, ``map_width``, ``8``) with the **first 4 channels** containing the **one hot encoding** of the direction of the given agent and the second 4 channels containing the positions of the other agents at their position coordinates.
Feel free to enhance this observation with any layer you think might help solve the problem.
We encourage you to enhance this observation with any layer you think might help solve the problem.
It would also be possible to construct a global observation for a super agent that controls all agents at once.
Local Grid Observation
...
...
@@ -51,20 +51,20 @@ We encourage you to come up with creative ways to overcome this problem. In the
Tree Observation
----------------
The tree observations is build by exploiting the graph structure of the railway network. The observation is generated by spanning a **4 branched tree** from the current position of the agent. Each branch follows the allowed transitions (backward branch only allowed at dead-ends) untill a cell with multiple allowed transitions is reached. Here the information gathered along the branch is stored as a node in the tree.
Figure bellow illustrates how such a tree observation is build:
The tree observation is built by exploiting the graph structure of the railway network. The observation is generated by spanning a **4 branched tree** from the current position of the agent. Each branch follows the allowed transitions (backward branch only allowed at dead-ends) until a cell with multiple allowed transitions is reached. Here the information gathered along the branch is stored as a node in the tree.
The figure below illustrates how the tree observation is built:
1. From Agent location probe all 4 directions (``L:Blue``, ``F:Green``, ``R:Purple``, ``B:Red``) starting with left and start branches when transition is allowed.
1. For each branch walk along the allowed transition till you reach a dead-end, switch or the target destination.
2. Create a node an fill in node information as stated below.
3. If max depth of tree is not reached and there are possible transistions start new branches and repeat above steps.
2. Fill up all non existing branches with -infinity such that tree size is invariant to number of possible transitions at branching points.
1. For each branch walk along the allowed transition until you reach a dead-end, switch or the target destination.
2. Create a node and fill in the node information as stated below.
3. If max depth of tree is not reached and there are possible transitions, start new branches and repeat the steps above.
2. Fill up all non existing branches with -infinity such that tree size is invariant to the number of possible transitions at branching points.
Note that we always start with the left branch according to the agent orientation. Thus the tree observation is independent of the orientation of cells and only consideres relative orientation of transition object to the agent.
Note that we always start with the left branch according to the agent orientation. Thus the tree observation is independent of the NESW orientation of cells, and only considers the transitions relative to the agent's orientation.
The colors in the figure bellow illustrate what branch the cell belongs two. If there are multiple colors in a cell, this cell is visited by different branches of the tree observation.
The right side of the figure shows the resulting tree of the railway network on the left. Cross means no branch was build. If a node has no children it either was a terminal node (dead-end, max depth reached or no transition possible). Circle indicate nodes that are filled with the corresponding infromation as stated below in Node Information.
The colors in the figure bellow illustrate what branch the cell belongs to. If there are multiple colors in a cell, this cell is visited by different branches of the tree observation.
The right side of the figure shows the resulting tree of the railway network on the left. Cross means no branch was built. If a node has no children it was either a terminal node (dead-end, max depth reached or no transition possible). A circle indicates a node filled with the corresponding information stated below in Node Information.
.. image:: https://i.imgur.com/sGBBhzJ.png
...
...
@@ -77,20 +77,20 @@ Node Information
Each node is filled with information gathered along the path to the node. Currently each node contains 9 features:
- 1: if own target lies on the explored branch the current distance from the agent in number of cells is stored.
- 2: if another agents target is detected the distance in number of cells from current agent position is stored.
- 3: if another agent is detected the distance in number of cells from current agent position is stored.
- 2: if another agent's target is detected, the distance in number of cells from the current agent position is stored.
- 3: if another agent is detected, the distance in number of cells from the current agent position is stored.
- 4: possible conflict detected (This only works when we use a predictor and will not be important in this tutorial)
- 5: if an not usable switch (for agent) is detected we store the distance. An unusable switch is a switch where the agent does not have any choice of path, but other agents coming from different directions might.
- 5: if an unusable switch (for the agent) is detected we store the distance. An unusable switch is a switch where the agent does not have any choice of path, but other agents coming from different directions might.
- 6: This feature stores the distance (in number of cells) to the next node (e.g. switch or target or dead-end)
- 7: minimum remaining travel distance from node to the agent's target given the direction of the agent if this path is chosen
- 7: minimum remaining travel distance from this node to the agent's target given the direction of the agent if this path is chosen
- 8: agent in the same direction found on path to node
- ``n`` = number of agents present same direction (possible future use: number of other agents in the same direction in this branch)
- ``0`` = no agent present same direction
- ``n`` = number of agents present in the same direction (possible future use: number of other agents in the same direction in this branch)
- ``0`` = no agent present in the same direction
- 9: agent in the opposite direction on path to node
- ``n`` = number of agents present other direction than myself
- ``0`` = no agent present other direction than myself
- ``n`` = number of agents present in the opposite direction to the observing agent
- ``0`` = no agent present in other direction to the observing agent
