diff --git a/.gitattributes b/.gitattributes
new file mode 100644
index 0000000000000000000000000000000000000000..333aa3985a8bf8a54f045c3563bcb2c25214c369
--- /dev/null
+++ b/.gitattributes
@@ -0,0 +1,12 @@
+# auto detect text files and perform LF normalization
+* text=auto
+
+# shell scripts will always have LF only (even in Windows environment)
+*.sh text eol=lf
+
+# perl scripts will always have LF only (even in Windows environment)
+*.pl text eol=lf
+
+# denote all files that are truly binary and should not be modified
+*.png binary
+*.jpg binary
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 5dd5539ca8218b0277c496802529134daac34e4d..d0d9efa380574eb36d13450043e7d81f35cf0bdb 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -25,6 +25,9 @@ tests:
script:
- apt update
- apt install -y libgl1-mesa-glx xvfb graphviz xdg-utils libcairo2-dev libjpeg-dev libgif-dev
+ - wget -nv https://repo.anaconda.com/archive/Anaconda3-5.3.1-Linux-x86_64.sh -O /tmp/Anaconda3-5.3.1-Linux-x86_64.sh
+ - bash /tmp/Anaconda3-5.3.1-Linux-x86_64.sh -b -p /tmp/anaconda3
+ - export PATH=/tmp/anaconda3/bin:$PATH; bash getting_started/getting_started.sh
- pip install tox
- xvfb-run tox -v --recreate
diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
index 814bc50af509a91d7165fd471ef369d77bfceca0..dd9500ff8435f1e88e0d4debf07ffb0883811743 100644
--- a/CONTRIBUTING.rst
+++ b/CONTRIBUTING.rst
@@ -73,6 +73,12 @@ Ready to contribute? Here's how to set up `flatland` for local development.
$ cd flatland
$ pip install -e .
+ These steps are performed if you run
+
+ $ getting_started/getting_started.bat/.sh
+
+ from Anaconda prompt. Only prerequisites: git and Anaconda.
+
4. Create a branch for local development::
diff --git a/README.rst b/README.rst
index 6811219991cfb849f7d1a2288ab413a4c889c0d7..23d96aa7083814f85f62f9868da284d01912e879 100644
--- a/README.rst
+++ b/README.rst
@@ -16,7 +16,7 @@ Flatland
:align: center
Flatland is a opensource toolkit for developing and comparing Multi Agent Reinforcement Learning algorithms in little (or ridiculously large !) gridworlds.
-The base environment is a two-dimensional grid in which many agents can be placed, and each agent must solve one or more navigational tasks in the grid world. More details about the environment and the problem statement can be found in the official docs `here <http://flatland-rl-docs.s3-website.eu-central-1.amazonaws.com/>`_.
+The base environment is a two-dimensional grid in which many agents can be placed, and each agent must solve one or more navigational tasks in the grid world. More details about the environment and the problem statement can be found in the `official docs <http://flatland-rl-docs.s3-website.eu-central-1.amazonaws.com/>`_.
This library was developed by `SBB <https://www.sbb.ch/en/>`_ , `AIcrowd <https://www.aicrowd.com/>`_ and numerous contributors and AIcrowd research fellows from the AIcrowd community.
diff --git a/docs/observation_actions.rst b/docs/observation_actions.rst
new file mode 100644
index 0000000000000000000000000000000000000000..397b4f9bb9bce0cab6d7061dc2417c1dbdc5ee1f
--- /dev/null
+++ b/docs/observation_actions.rst
@@ -0,0 +1,96 @@
+=============================
+Observation and Action Spaces
+=============================
+This is an introduction to the three standard observations and the action space of **Flatland**.
+
+Action Space
+============
+Flatland is a railway simulation. Thus the actions of an agent are strongly limited to the railway network. This means that in many cases not all actions are valid.
+The possible actions of an agent are
+
+- ``0`` **Do Nothing**: If the agent is moving it continues moving, if it is stopped it stays stopped
+- ``1`` **Deviate Left**: If the agent is at a switch with a transition to its left, the agent will chose th eleft path. Otherwise the action has no effect. If the agent is stopped, this action will start agent movement again if allowed by the transitions.
+- ``2`` **Go Forward**: This action will start the agent when stopped. This will move the agent forward and chose the go straight direction at switches.
+- ``3`` **Deviate Right**: Exactly the same as deviate left but for right turns.
+- ``4`` **Stop**: This action causes the agent to stop.
+
+Observation Spaces
+==================
+In the **Flatland** environment we have included three basic observations to get started. The figure below illustrates the observation range of the different basic observation: ``Global``, ``Local Grid`` and ``Local Tree``.
+
+.. image:: https://i.imgur.com/oo8EIYv.png
+ :height: 100
+ :width: 200
+
+
+Global Observation
+------------------
+Gives a global observation of the entire rail environment.
+
+The observation is composed of the following elements:
+
+- transition map array with dimensions (``env.height``, ``env.width``, ``16``), assuming **16 bits encoding of transitions**.
+- Two 2D arrays (``map_height``, ``map_width``, ``2``) containing respectively the position of the given agent target and the positions of the other agents targets.
+- A 3D array (``map_height``, ``map_width``, ``8``) with the **4 first channels** containing the **one hot encoding** of the direction of the given agent and the 4 second channels containing the positions of the other agents at their position coordinates.
+
+Feel free to enhance this observation with any layer you think might help solve the problem.
+It would also be possible to construct a global observation for a super agent that controls all agents at once.
+
+Local Grid Observation
+----------------------
+Gives a local observation of the rail environment around the agent.
+The observation is composed of the following elements:
+
+- transition map array of the local environment around the given agent, with dimensions (``2*view_radius + 1``, ``2*view_radius + 1``, ``16``), assuming **16 bits encoding of transitions**.
+- Two 2D arrays (``2*view_radius + 1``, ``2*view_radius + 1``, ``2``) containing respectively, if they are in the agent's vision range, its target position, the positions of the other targets.
+- A 3D array (``2*view_radius + 1``, ``2*view_radius + 1``, ``4``) containing the one hot encoding of directions of the other agents at their position coordinates, if they are in the agent's vision range.
+- A 4 elements array with one hot encoding of the direction.
+
+Be aware that this observation **does not** contain any clues about target location if target is out of range. Thus navigation on maps where the radius of the observation does not guarantee a visible target at all times will become very difficult.
+We encourage you to come up with creative ways to overcome this problem. In the tree observation below we introduce the concept of distance maps.
+
+Tree Observation
+----------------
+The tree observations is build by exploiting the graph structure of the railway network. The observation is generated by spanning a **4 branched tree** from the current position of the agent. Each branch follows the allowed transitions (backward branch only allowed at dead-ends) untill a cell with multiple allowed transitions is reached. Here the information gathered along the branch is stored as a node in the tree.
+Figure bellow illustrates how such a tree observation is build:
+
+1. From Agent location probe all 4 directions (``L:Blue``, ``F:Green``, ``R:Purple``, ``B:Red``) starting with left and start branches when transition is allowed.
+
+ 1. For each branch walk along the allowed transition till you reach a dead-end, switch or the target destination.
+ 2. Create a node an fill in node information as stated below.
+ 3. If max depth of tree is not reached and there are possible transistions start new branches and repeat above steps.
+2. Fill up all non existing branches with -infinity such that tree size is invariant to number of possible transitions at branching points.
+
+Note that we always start with the left branch according to the agent orientation. Thus the tree observation is independent of the orientation of cells and only consideres relative orientation of transition object to the agent.
+
+The colors in the figure bellow illustrate what branch the cell belongs two. If there are multiple colors in a cell, this cell is visited by different branches of the tree observation.
+The right side of the figure shows the resulting tree of the railway network on the left. Cross means no branch was build. If a node has no children it either was a terminal node (dead-end, max depth reached or no transition possible). Circle indicate nodes that are filled with the corresponding infromation as stated below in Node Information.
+
+
+.. image:: https://i.imgur.com/sGBBhzJ.png
+ :height: 100
+ :width: 200
+
+
+Node Information
+----------------
+Each node is filled with information gathered along the path to the node. Currently each node contains 9 features:
+
+- 1: if own target lies on the explored branch the current distance from the agent in number of cells is stored.
+- 2: if another agents target is detected the distance in number of cells from current agent position is stored.
+- 3: if another agent is detected the distance in number of cells from current agent position is stored.
+- 4: possible conflict detected (This only works when we use a predictor and will not be important in this tutorial)
+- 5: if an not usable switch (for agent) is detected we store the distance. An unusable switch is a switch where the agent does not have any choice of path, but other agents coming from different directions might.
+- 6: This feature stores the distance (in number of cells) to the next node (e.g. switch or target or dead-end)
+- 7: minimum remaining travel distance from node to the agent's target given the direction of the agent if this path is chosen
+- 8: agent in the same direction found on path to node
+
+ - ``n`` = number of agents present same direction (possible future use: number of other agents in the same direction in this branch)
+ - ``0`` = no agent present same direction
+- 9: agent in the opposite direction on path to node
+
+ - ``n`` = number of agents present other direction than myself
+ - ``0`` = no agent present other direction than myself
+
+
+
diff --git a/examples/custom_railmap_example.py b/examples/custom_railmap_example.py
index 9ccef3fdf58aab56f1819a928477118fa8a467a5..6dc157f33ab90d5efa06dc45c914a8de79def7e8 100644
--- a/examples/custom_railmap_example.py
+++ b/examples/custom_railmap_example.py
@@ -17,11 +17,13 @@ def custom_rail_generator():
grid_map = GridTransitionMap(width=width, height=height, transitions=rail_trans)
rail_array = grid_map.grid
rail_array.fill(0)
-
+ new_tran = rail_trans.set_transition(1,1,1,1)
+ print(new_tran)
agents_positions = []
agents_direction = []
agents_target = []
-
+ rail_array[0, 0] = new_tran
+ rail_array[0, 1] = new_tran
return grid_map, agents_positions, agents_direction, agents_target
return generator
diff --git a/flatland/__init__.py b/flatland/__init__.py
index cccf18e6fc8ff614ce5f8b6eec433a883d474d81..feccf8656b98057e0c7486483d2a6c6281077a37 100644
--- a/flatland/__init__.py
+++ b/flatland/__init__.py
@@ -4,4 +4,4 @@
__author__ = """S.P. Mohanty"""
__email__ = 'mohanty@aicrowd.com'
-__version__ = '0.1.2'
+__version__ = '0.2.0'
diff --git a/flatland/core/grid/grid4.py b/flatland/core/grid/grid4.py
index 714123ed5c5d2dd704c6f1262df5cc18fba8ef10..b4b5b17c9b1e6f5c7abd3c82a1fbf1e5ee96fcfe 100644
--- a/flatland/core/grid/grid4.py
+++ b/flatland/core/grid/grid4.py
@@ -1,4 +1,5 @@
from enum import IntEnum
+from typing import Type
import numpy as np
@@ -218,7 +219,7 @@ class Grid4Transitions(Transitions):
cell_transition = value
return cell_transition
- def get_direction_enum(self) -> IntEnum:
+ def get_direction_enum(self) -> Type[Grid4TransitionsEnum]:
return Grid4TransitionsEnum
def has_deadend(self, cell_transition):
diff --git a/flatland/core/transition_map.py b/flatland/core/transition_map.py
index cb09a628136c5907c93024970757df7e5980842f..5e0f6cd72e8ca22c80e3576798e5214f8b036558 100644
--- a/flatland/core/transition_map.py
+++ b/flatland/core/transition_map.py
@@ -7,6 +7,7 @@ from importlib_resources import path
from numpy import array
from flatland.core.grid.grid4 import Grid4Transitions
+from flatland.core.transitions import Transitions
class TransitionMap:
@@ -110,7 +111,7 @@ class GridTransitionMap(TransitionMap):
GridTransitionMap implements utility functions.
"""
- def __init__(self, width, height, transitions=Grid4Transitions([])):
+ def __init__(self, width, height, transitions: Transitions = Grid4Transitions([])):
"""
Builder for GridTransitionMap object.
@@ -132,7 +133,25 @@ class GridTransitionMap(TransitionMap):
self.grid = np.zeros((height, width), dtype=self.transitions.get_type())
- def get_transitions(self, cell_id):
+ def get_full_transitions(self, row, column):
+ """
+ Returns the full transitions for the cell at (row, column) in the format transition_map's transitions.
+
+ Parameters
+ ----------
+ row: int
+ column: int
+ (row,column) specifies the cell in this transition map.
+
+ Returns
+ -------
+ self.transitions.get_type()
+ The cell content int the format of this map's Transitions.
+
+ """
+ return self.grid[row][column]
+
+ def get_transitions(self, row, column, orientation):
"""
Return a tuple of transitions available in a cell specified by
`cell_id' (e.g., a tuple of size of the maximum number of transitions,
@@ -150,15 +169,10 @@ class GridTransitionMap(TransitionMap):
Returns
-------
tuple
- List of the validity of transitions in the cell.
+ List of the validity of transitions in the cell as given by the maps transitions.
"""
- assert len(cell_id) in (2, 3), \
- 'GridTransitionMap.get_transitions() ERROR: cell_id tuple must have length 2 or 3.'
- if len(cell_id) == 3:
- return self.transitions.get_transitions(self.grid[cell_id[0]][cell_id[1]], cell_id[2])
- elif len(cell_id) == 2:
- return self.grid[cell_id[0]][cell_id[1]]
+ return self.transitions.get_transitions(self.grid[row][column], orientation)
def set_transitions(self, cell_id, new_transitions):
"""
@@ -308,7 +322,7 @@ class GridTransitionMap(TransitionMap):
grcPos = array(rcPos)
grcMax = self.grid.shape
- binTrans = self.get_transitions(rcPos) # 16bit integer - all trans in/out
+ binTrans = self.get_full_transitions(*rcPos) # 16bit integer - all trans in/out
lnBinTrans = array([binTrans >> 8, binTrans & 0xff], dtype=np.uint8) # 2 x uint8
g2binTrans = np.unpackbits(lnBinTrans).reshape(4, 4) # 4x4 x uint8 binary(0,1)
gDirOut = g2binTrans.any(axis=0) # outbound directions as boolean array (4)
@@ -328,7 +342,7 @@ class GridTransitionMap(TransitionMap):
# Get the transitions out of gPos2, using iDirOut as the inbound direction
# if there are no available transitions, ie (0,0,0,0), then rcPos is invalid
- t4Trans2 = self.get_transitions((*gPos2, iDirOut))
+ t4Trans2 = self.get_transitions(*gPos2, iDirOut)
if any(t4Trans2):
continue
else:
diff --git a/flatland/envs/grid4_generators_utils.py b/flatland/envs/grid4_generators_utils.py
index 4b2ab8cbc8b956698383cd3961d77df6fb1ef195..dedd76b6bfd04c13ad59092adfefbde6ae98fc18 100644
--- a/flatland/envs/grid4_generators_utils.py
+++ b/flatland/envs/grid4_generators_utils.py
@@ -75,7 +75,7 @@ def get_rnd_agents_pos_tgt_dir_on_rail(rail, num_agents):
return 1
if node not in visited:
visited.add(node)
- moves = rail.get_transitions((node[0][0], node[0][1], node[1]))
+ moves = rail.get_transitions(node[0][0], node[0][1], node[1])
for move_index in range(4):
if moves[move_index]:
stack.append((get_new_position(node[0], move_index),
@@ -84,7 +84,7 @@ def get_rnd_agents_pos_tgt_dir_on_rail(rail, num_agents):
# If cell is a dead-end, append previous node with reversed
# orientation!
nbits = 0
- tmp = rail.get_transitions((node[0][0], node[0][1]))
+ tmp = rail.get_full_transitions(node[0][0], node[0][1])
while tmp > 0:
nbits += (tmp & 1)
tmp = tmp >> 1
@@ -96,7 +96,7 @@ def get_rnd_agents_pos_tgt_dir_on_rail(rail, num_agents):
valid_positions = []
for r in range(rail.height):
for c in range(rail.width):
- if rail.get_transitions((r, c)) > 0:
+ if rail.get_full_transitions(r, c) > 0:
valid_positions.append((r, c))
re_generate = True
@@ -116,7 +116,7 @@ def get_rnd_agents_pos_tgt_dir_on_rail(rail, num_agents):
valid_movements = []
for direction in range(4):
position = agents_position[i]
- moves = rail.get_transitions((position[0], position[1], direction))
+ moves = rail.get_transitions(position[0], position[1], direction)
for move_index in range(4):
if moves[move_index]:
valid_movements.append((direction, move_index))
diff --git a/flatland/envs/observations.py b/flatland/envs/observations.py
index 18af8a0c88067ee00f9d9b2fb6f8bbbecc486909..b1f46ec638d8a865a2ce36a9715d89b452c782f2 100644
--- a/flatland/envs/observations.py
+++ b/flatland/envs/observations.py
@@ -16,8 +16,10 @@ class TreeObsForRailEnv(ObservationBuilder):
TreeObsForRailEnv object.
This object returns observation vectors for agents in the RailEnv environment.
- The information is local to each agent and exploits the tree structure of the rail
+ The information is local to each agent and exploits the graph structure of the rail
network to simplify the representation of the state of the environment for each agent.
+
+ For details about the features in the tree observation see the get() function.
"""
observation_dim = 9
@@ -204,7 +206,7 @@ class TreeObsForRailEnv(ObservationBuilder):
[... from 'right] +
[... from 'back']
- Finally, each node information is composed of 8 floating point values:
+ Each node information is composed of 9 features:
#1: if own target lies on the explored branch the current distance from the agent in number of cells is stored.
@@ -252,7 +254,7 @@ class TreeObsForRailEnv(ObservationBuilder):
if handle > len(self.env.agents):
print("ERROR: obs _get - handle ", handle, " len(agents)", len(self.env.agents))
agent = self.env.agents[handle] # TODO: handle being treated as index
- possible_transitions = self.env.rail.get_transitions((*agent.position, agent.direction))
+ possible_transitions = self.env.rail.get_transitions(*agent.position, agent.direction)
num_transitions = np.count_nonzero(possible_transitions)
# Root node - current position
@@ -382,8 +384,8 @@ class TreeObsForRailEnv(ObservationBuilder):
last_is_target = True
break
- cell_transitions = self.env.rail.get_transitions((*position, direction))
- total_transitions = bin(self.env.rail.get_transitions(position)).count("1")
+ cell_transitions = self.env.rail.get_transitions(*position, direction)
+ total_transitions = bin(self.env.rail.get_full_transitions(*position)).count("1")
num_transitions = np.count_nonzero(cell_transitions)
exploring = False
# Detect Switches that can only be used by other agents.
@@ -393,7 +395,7 @@ class TreeObsForRailEnv(ObservationBuilder):
if num_transitions == 1:
# Check if dead-end, or if we can go forward along direction
nbits = 0
- tmp = self.env.rail.get_transitions(tuple(position))
+ tmp = self.env.rail.get_full_transitions(*position)
while tmp > 0:
nbits += (tmp & 1)
tmp = tmp >> 1
@@ -468,7 +470,7 @@ class TreeObsForRailEnv(ObservationBuilder):
# Start from the current orientation, and see which transitions are available;
# organize them as [left, forward, right, back], relative to the current orientation
# Get the possible transitions
- possible_transitions = self.env.rail.get_transitions((*position, direction))
+ possible_transitions = self.env.rail.get_transitions(*position, direction)
for branch_direction in [(direction + 4 + i) % 4 for i in range(-1, 3)]:
if last_is_dead_end and self.env.rail.get_transition((*position, direction),
(branch_direction + 2) % 4):
@@ -571,7 +573,7 @@ class GlobalObsForRailEnv(ObservationBuilder):
self.rail_obs = np.zeros((self.env.height, self.env.width, 16))
for i in range(self.rail_obs.shape[0]):
for j in range(self.rail_obs.shape[1]):
- bitlist = [int(digit) for digit in bin(self.env.rail.get_transitions((i, j)))[2:]]
+ bitlist = [int(digit) for digit in bin(self.env.rail.get_full_transitions(i, j))[2:]]
bitlist = [0] * (16 - len(bitlist)) + bitlist
self.rail_obs[i, j] = np.array(bitlist)
@@ -629,7 +631,7 @@ class GlobalObsForRailEnvDirectionDependent(ObservationBuilder):
self.rail_obs = np.zeros((self.env.height, self.env.width, 16))
for i in range(self.rail_obs.shape[0]):
for j in range(self.rail_obs.shape[1]):
- bitlist = [int(digit) for digit in bin(self.env.rail.get_transitions((i, j)))[2:]]
+ bitlist = [int(digit) for digit in bin(self.env.rail.get_full_transitions(i, j))[2:]]
bitlist = [0] * (16 - len(bitlist)) + bitlist
self.rail_obs[i, j] = np.array(bitlist)
@@ -676,10 +678,10 @@ class LocalObsForRailEnv(ObservationBuilder):
with dimensions (2*view_radius + 1, 2*view_radius + 1, 16),
assuming 16 bits encoding of transitions.
- - Two 2D arrays containing respectively, if they are in the agent's vision range,
- its target position, the positions of the other targets.
+ - Two 2D arrays (2*view_radius + 1, 2*view_radius + 1, 2) containing respectively,
+ if they are in the agent's vision range, its target position, the positions of the other targets.
- - A 3D array (map_height, map_width, 4) containing the one hot encoding of directions
+ - A 3D array (2*view_radius + 1, 2*view_radius + 1, 4) containing the one hot encoding of directions
of the other agents at their position coordinates, if they are in the agent's vision range.
- A 4 elements array with one hot encoding of the direction.
@@ -700,7 +702,7 @@ class LocalObsForRailEnv(ObservationBuilder):
self.env.width + 2 * self.view_radius, 16))
for i in range(self.env.height):
for j in range(self.env.width):
- bitlist = [int(digit) for digit in bin(self.env.rail.get_transitions((i, j)))[2:]]
+ bitlist = [int(digit) for digit in bin(self.env.rail.get_full_transitions(i, j))[2:]]
bitlist = [0] * (16 - len(bitlist)) + bitlist
self.rail_obs[i + self.view_radius, j + self.view_radius] = np.array(bitlist)
diff --git a/flatland/envs/predictions.py b/flatland/envs/predictions.py
index 3dbd163ca2d67ffa698a5fe2c931ce98c2dd8bbc..ca675ce209bc7e4aeb96305eb945751bb903b8ed 100644
--- a/flatland/envs/predictions.py
+++ b/flatland/envs/predictions.py
@@ -131,7 +131,7 @@ class ShortestPathPredictorForRailEnv(PredictionBuilder):
prediction[index] = [index, *agent.position, agent.direction, RailEnvActions.STOP_MOVING]
continue
# Take shortest possible path
- cell_transitions = self.env.rail.get_transitions((*agent.position, agent.direction))
+ cell_transitions = self.env.rail.get_transitions(*agent.position, agent.direction)
new_position = None
new_direction = None
diff --git a/flatland/envs/rail_env.py b/flatland/envs/rail_env.py
index 8abfd1b3b295bbe347ff5a25a5cd685314e1621a..149e32bd66bb0c0e1ac96bbfe213705b4de63253 100644
--- a/flatland/envs/rail_env.py
+++ b/flatland/envs/rail_env.py
@@ -1,8 +1,5 @@
"""
-Definition of the RailEnv environment and related level-generation functions.
-
-Generator functions are functions that take width, height and num_resets as arguments and return
-a GridTransitionMap object.
+Definition of the RailEnv environment.
"""
# TODO: _ this is a global method --> utils or remove later
@@ -46,20 +43,35 @@ class RailEnv(Environment):
to avoid bottlenecks.
The valid actions in the environment are:
- 0: do nothing
- 1: turn left and move to the next cell; if the agent was not moving, movement is started
- 2: move to the next cell in front of the agent; if the agent was not moving, movement is started
- 3: turn right and move to the next cell; if the agent was not moving, movement is started
- 4: stop moving
+
+ - 0: do nothing (continue moving or stay still)
+ - 1: turn left at switch and move to the next cell; if the agent was not moving, movement is started
+ - 2: move to the next cell in front of the agent; if the agent was not moving, movement is started
+ - 3: turn right at switch and move to the next cell; if the agent was not moving, movement is started
+ - 4: stop moving
Moving forward in a dead-end cell makes the agent turn 180 degrees and step
to the cell it came from.
+
The actions of the agents are executed in order of their handle to prevent
deadlocks and to allow them to learn relative priorities.
- TODO: WRITE ABOUT THE REWARD FUNCTION, and possibly allow for alpha and
- beta to be passed as parameters to __init__().
+ Reward Function:
+
+ It costs each agent a step_penalty for every time-step taken in the environment. Independent of the movement
+ of the agent. Currently all other penalties such as penalty for stopping, starting and invalid actions are set to 0.
+
+ alpha = 1
+ beta = 1
+ Reward function parameters:
+
+ - invalid_action_penalty = 0
+ - step_penalty = -alpha
+ - global_reward = beta
+ - stop_penalty = 0 # penalty for stopping a moving agent
+ - start_penalty = 0 # penalty for starting a stopped agent
+
"""
def __init__(self,
@@ -326,7 +338,7 @@ class RailEnv(Environment):
new_position,
np.clip(new_position, [0, 0], [self.height - 1, self.width - 1]))
and # check the new position has some transitions (ie is not an empty cell)
- self.rail.get_transitions(new_position) > 0)
+ self.rail.get_full_transitions(*new_position) > 0)
# If transition validity hasn't been checked yet.
if transition_valid is None:
@@ -342,7 +354,7 @@ class RailEnv(Environment):
def check_action(self, agent, action):
transition_valid = None
- possible_transitions = self.rail.get_transitions((*agent.position, agent.direction))
+ possible_transitions = self.rail.get_transitions(*agent.position, agent.direction)
num_transitions = np.count_nonzero(possible_transitions)
new_direction = agent.direction
diff --git a/flatland/utils/editor.py b/flatland/utils/editor.py
index dd0c4d4b96d8feb5e331802639445c9919646413..ea4056faf7f01bf93b03710b84f0a87830691f2d 100644
--- a/flatland/utils/editor.py
+++ b/flatland/utils/editor.py
@@ -494,7 +494,7 @@ class EditorModel(object):
if len(lrcStroke) >= 2:
# If the first cell in a stroke is empty, add a deadend to cell 0
- if self.env.rail.get_transitions(lrcStroke[0]) == 0:
+ if self.env.rail.get_full_transitions(*lrcStroke[0]) == 0:
self.mod_rail_2cells(lrcStroke, bAddRemove, iCellToMod=0)
# Add transitions for groups of 3 cells
@@ -504,7 +504,7 @@ class EditorModel(object):
# If final cell empty, insert deadend:
if len(lrcStroke) == 2:
- if self.env.rail.get_transitions(lrcStroke[1]) == 0:
+ if self.env.rail.get_full_transitions(*lrcStroke[1]) == 0:
self.mod_rail_2cells(lrcStroke, bAddRemove, iCellToMod=1)
# now empty out the final two cells from the queue
@@ -752,7 +752,7 @@ class EditorModel(object):
self.log(*args, **kwargs)
def debug_cell(self, rcCell):
- binTrans = self.env.rail.get_transitions(rcCell)
+ binTrans = self.env.rail.get_full_transitions(*rcCell)
sbinTrans = format(binTrans, "#018b")[2:]
self.debug("cell ",
rcCell,
diff --git a/flatland/utils/rendertools.py b/flatland/utils/rendertools.py
index a148a5aed4e52e8b8f6a42e4edd85cbd36365e0a..6fa68f02a409659e6cb9444ab1469c41bcdd48bc 100644
--- a/flatland/utils/rendertools.py
+++ b/flatland/utils/rendertools.py
@@ -86,7 +86,7 @@ class RenderTool(object):
for visit in lVisits:
# transition for next cell
- tbTrans = self.env.rail.get_transitions((*visit.rc, visit.iDir))
+ tbTrans = self.env.rail.get_transitions(*visit.rc, visit.iDir)
giTrans = np.where(tbTrans)[0] # RC list of transitions
gTransRCAg = rt.gTransRC[giTrans]
self.plotTrans(visit.rc, gTransRCAg, depth=str(visit.iDepth), color=color)
@@ -125,7 +125,7 @@ class RenderTool(object):
)
"""
- tbTrans = self.env.rail.get_transitions((*rcPos, iDir))
+ tbTrans = self.env.rail.get_transitions(*rcPos, iDir)
giTrans = np.where(tbTrans)[0] # RC list of transitions
# HACK: workaround dead-end transitions
@@ -459,7 +459,7 @@ class RenderTool(object):
xyCentre = array([x0, y1]) + cell_size / 2
# cell transition values
- oCell = env.rail.get_transitions((r, c))
+ oCell = env.rail.get_full_transitions(r, c)
bCellValid = env.rail.cell_neighbours_valid((r, c), check_this_cell=True)
@@ -482,7 +482,7 @@ class RenderTool(object):
from_ori = (orientation + 2) % 4 # 0123=NESW -> 2301=SWNE
from_xy = coords[from_ori]
- tMoves = env.rail.get_transitions((r, c, orientation))
+ tMoves = env.rail.get_transitions(r, c, orientation)
for to_ori in range(4):
to_xy = coords[to_ori]
diff --git a/getting_started/getting_started.bat b/getting_started/getting_started.bat
new file mode 100644
index 0000000000000000000000000000000000000000..48833ab83ebc62a3bbb7edfc8312251c48bb17dc
--- /dev/null
+++ b/getting_started/getting_started.bat
@@ -0,0 +1,55 @@
+@echo on
+
+set FLATLAND_BASEDIR=%~dp0\..
+
+@echo off
+echo "************ TESTING PREREQUISITES PYTHON3 + GIT + GIT *************************"
+@echo on
+
+git --version || goto :error
+python --version || goto :error
+rem deactivate in case we're in virtualenv
+call conda deactivate || call deactivate
+
+
+@echo off
+echo "************ SETUP VIRTUAL ENVIRONMENT FLATLAND *************************"
+@echo on
+(conda info --envs | findstr flatland-rl) || conda create python=3.6 -y --name flatland-rl || goto :error
+(call conda activate flatland-rl || call activate flatland-rl) || goto :error
+
+
+@echo off
+echo "************ INSTALL FLATLAND AND DEPENDENCIES IN THE VIRTUALENV *************************"
+@echo on
+rem TODO we should get rid of having to install these packages outside of setup.py with conda!
+call conda install -y -c conda-forge cairosvg pycairo || goto :error
+call conda install -y -c anaconda tk || goto :error
+call python -m pip install --upgrade pip || goto :error
+python setup.py install || goto :error
+
+# ensure jupyter is installed in the virtualenv
+python -m pip install --upgrade -r %FLATLAND_BASEDIR%/requirements_dev.txt -r %FLATLAND_BASEDIR%/requirements_continuous_integration.txt || goto :error
+
+
+@echo off
+echo "************ INSTALL JUPYTER EXTENSION *************************"
+@echo on
+jupyter nbextension install --py --sys-prefix widgetsnbextension || goto :error
+jupyter nbextension enable --py --sys-prefix widgetsnbextension || goto :error
+jupyter nbextension install --py --sys-prefix jpy_canvas || goto :error
+jupyter nbextension enable --py --sys-prefix jpy_canvas || goto :error
+
+
+@echo off
+echo "************ RUN JUPYTER NOTEBOOKS *************************"
+@echo on
+jupyter notebook || goto :error
+
+
+goto :EOF
+
+
+:error
+echo Failed with error #%errorlevel%.
+pause
diff --git a/getting_started/run_notebooks.sh b/getting_started/getting_started.sh
old mode 100755
new mode 100644
similarity index 57%
rename from getting_started/run_notebooks.sh
rename to getting_started/getting_started.sh
index 70654ae50fa2e51c7b19df1e71e07408d3c4e7ce..0a245787f32a42092c548605b77ed27426ec4903
--- a/getting_started/run_notebooks.sh
+++ b/getting_started/getting_started.sh
@@ -1,4 +1,4 @@
-#!/usr/bin/env bash
+#!/bin/bash
set -e # stop on error
set -x # echo commands
@@ -12,27 +12,34 @@ set +x
echo "************ TESTING PREREQUISITES PYTHON3 + GIT *************************"
set -x
+git --version
+python --version
+conda --version
+echo $PATH
+
set +x
echo "************ SETUP VIRTUAL ENVIRONMENT FLATLAND *************************"
set -x
+source deactivate
+(conda info --envs | fgrep flatland-rl) || conda create python=3.6 -y --name flatland-rl
+source activate flatland-rl
-export WORKON_HOME=${FLATLAND_BASEDIR}/getting_started/envs
-echo WORKON_HOME=$WORKON_HOME
-echo PWD=$PWD
-mkdir -p ${WORKON_HOME}
-# cannot work with virtualenvwrapper in script
-cd ${WORKON_HOME}
-python3 -m venv flatland
-source flatland/bin/activate
set +x
echo "************ INSTALL FLATLAND IN THE VIRTUALENV *************************"
set -x
-cd ${FLATLAND_BASEDIR}
-python setup.py install
+
+# TODO we should get rid of having to install these packages outside of setup.py with conda!
+conda install -y -c conda-forge cairosvg pycairo
+conda install -y -c anaconda tk
+python -m pip install --upgrade pip
+
+python ${FLATLAND_BASEDIR}/setup.py install
+
# ensure jupyter is installed in the virtualenv
-pip install -r ${FLATLAND_BASEDIR}/requirements_dev.txt -r requirements_continuous_integration.txt
+python -m pip install --upgrade -r ${FLATLAND_BASEDIR}/requirements_dev.txt -r requirements_continuous_integration.txt
+
set +x
echo "************ INSTALL JUPYTER EXTENSION *************************"
@@ -41,4 +48,9 @@ jupyter nbextension install --py --sys-prefix widgetsnbextension
jupyter nbextension enable --py --sys-prefix widgetsnbextension
jupyter nbextension install --py --sys-prefix jpy_canvas
jupyter nbextension enable --py --sys-prefix jpy_canvas
-jupyter notebook
+
+
+set +x
+echo "************ RUN JUPYTER NOTEBOOKS *************************"
+set -x
+jupyter notebook &
diff --git a/getting_started/run_notebooks.bat b/getting_started/run_notebooks.bat
deleted file mode 100644
index afe41e08040c981deddcb1718e3876c2bb097708..0000000000000000000000000000000000000000
--- a/getting_started/run_notebooks.bat
+++ /dev/null
@@ -1,53 +0,0 @@
-@echo on
-
-set PWD_BEFORE=%cd%
-
-@echo off
-echo "************ TESTING PREREQUISITES PYTHON3 + GIT *************************"
-@echo on
-
-git --version || goto :error
-python3 --version || goto :error
-
-
-@echo off
-echo "************ SETUP VIRTUAL ENVIRONMENT FLATLAND *************************"
-@echo on
-
-set FLATLAND_BASEDIR=%~dp0\..
-set WORKON_HOME=%FLATLAND_BASEDIR%\getting_started\envs_win
-if not exist "%WORKON_HOME%" md "%WORKON_HOME%" || goto :error
-cd "%WORKON_HOME%"
-rem use venv instead of virtualenv/virtualenv-wrapper because of https://github.com/pypa/virtualenv/issues/355
-python3 -m venv flatland || goto :error
-rem ignore activation error: https://stackoverflow.com/questions/51358202/python-3-7-activate-venv-error-parameter-format-not-correct-65001-windows
-call "%WORKON_HOME%\flatland\Scripts\activate.bat" || true
-
-
-@echo off
-echo "************ INSTALL FLATLAND IN THE VIRTUALENV *************************"
-@echo on
-python -m pip install --upgrade pip || goto :error
-cd %FLATLAND_BASEDIR% || goto :error
-python setup.py install || goto :error
-REM ensure jupyter is installed in the virtualenv
-pip install -r "%FLATLAND_BASEDIR%/requirements_dev.txt" -r "%FLATLAND_BASEDIR%\requirements_continuous_integration.txt" || goto :error
-
-@echo off
-echo "************ INSTALL JUPYTER EXTENSION *************************"
-@echo on
-jupyter nbextension install --py --sys-prefix widgetsnbextension || goto :error
-jupyter nbextension enable --py --sys-prefix widgetsnbextension || goto :error
-jupyter nbextension install --py --sys-prefix jpy_canvas || goto :error
-jupyter nbextension enable --py --sys-prefix jpy_canvas || goto :error
-jupyter notebook || goto :error
-
-
-goto :EOF
-
-
-:error
-echo Failed with error #%errorlevel%.
-cd "%PWD_BEFORE%" || true
-deactivate || true
-pause
diff --git a/notebooks/Scene_Editor.ipynb b/notebooks/Scene_Editor.ipynb
index 64bbad77d3a9b63caea062795cd0ab8f0cce72f3..87b2d38c314140a3f51afa92058b0d5c3c3cc6af 100644
--- a/notebooks/Scene_Editor.ipynb
+++ b/notebooks/Scene_Editor.ipynb
@@ -70,7 +70,7 @@
{
"data": {
"application/vnd.jupyter.widget-view+json": {
- "model_id": "0b92e7084e37450cbfb3855dc3a58543",
+ "model_id": "86a323f55bb54ff78169a9c7ca63730f",
"version_major": 2,
"version_minor": 0
},
@@ -111,7 +111,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.6.5"
+ "version": "3.6.8"
},
"latex_envs": {
"LaTeX_envs_menu_present": true,
diff --git a/requirements_continuous_integration.txt b/requirements_continuous_integration.txt
index fa2b57bc94cd6f2c2a41b0299f9a77d0291f3fe0..a165e673a153db966a05ded2e18b7aafc1ff0042 100644
--- a/requirements_continuous_integration.txt
+++ b/requirements_continuous_integration.txt
@@ -1,16 +1,17 @@
# contains all additional requirements to run the ```tox .``` in addition to requirements_dev.txt for ```tox -e py36,py37```
-bumpversion==0.5.3
-wheel==0.32.1
-watchdog==0.9.0
-benchmarker==4.0.1
-coverage==4.5.1
-Sphinx==1.8.1
-sphinx-rtd-theme==0.4.3
-flake8==3.7.7
-flake8-eradicate==0.2.0
-twine==1.12.1
-pydeps==1.7.2
-jupyter==1.0.0
-pytest-xvfb==1.2.0
+bumpversion>=0.5.3
+wheel>=0.32.1
+watchdog>=0.9.0
+benchmarker>=4.0.1
+coverage>=4.5.1
+Sphinx>=1.8.1
+sphinx-rtd-theme>=0.4.3
+flake8>=3.7.7
+flake8-eradicate>=0.2.0
+twine>=1.12.1
+pydeps>=1.7.2
+jupyter>=1.0.0
+jupyter-core>=4.5.0
+notebook>=5.7.8
+pytest-xvfb>=1.2.0
git+https://github.com/who8mylunch/Jupyter_Canvas_Widget.git@bd151ae1509c50b5809944dd3294f58b7b069c86
-
diff --git a/requirements_dev.txt b/requirements_dev.txt
index 002be2a1736601bfec55406eb47fd33ec90bb358..ea46eb245842881f1c51ededa5284300836a36ba 100644
--- a/requirements_dev.txt
+++ b/requirements_dev.txt
@@ -1,18 +1,17 @@
# contains all requirements to run the tests by running ```tox -e py36,py37```
-tox==3.5.2
-twine==1.12.1
-pytest==3.8.2
-pytest-runner==4.2
-numpy==1.16.4
-recordtype==1.3
-xarray==0.11.3
-matplotlib==3.0.2
-Pillow==5.4.1
-# pycairo==1.18.1 is installed via setup.py since it needs special treatment under Windows
-CairoSVG==2.3.1
-msgpack==0.6.1
-svgutils==0.3.1
-screeninfo==0.3.1
-pyarrow==0.13.0
-importlib-metadata==0.17
-importlib_resources==1.0.2
+tox>=3.5.2
+twine>=1.12.1
+pytest>=3.8.2
+pytest-runner>=4.2
+numpy>=1.16.4
+recordtype>=1.3
+xarray>=0.11.3
+matplotlib>=3.0.2
+Pillow>=5.4.1
+CairoSVG>=2.3.1
+msgpack>=0.6.1
+svgutils>=0.3.1
+screeninfo>=0.3.1
+pyarrow>=0.13.0
+importlib-metadata>=0.17
+importlib_resources>=1.0.2
diff --git a/setup.cfg b/setup.cfg
index dd16bd77df8727f52ed2f3102b1042799271a611..d83f09f0c50a153fce8c40c22cf7d496eb02304b 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -1,5 +1,5 @@
[bumpversion]
-current_version = 0.1.2
+current_version = 0.2.0
commit = True
tag = True
diff --git a/setup.py b/setup.py
index 3cc28b34f7acf0d05958606c79beef439632035f..0afa7bf1ba7ce9fbe170178caf808dd7ce267937 100644
--- a/setup.py
+++ b/setup.py
@@ -3,7 +3,6 @@
"""The setup script."""
import os
-import platform
import sys
from setuptools import setup, find_packages
@@ -12,51 +11,6 @@ assert sys.version_info >= (3, 6)
with open('README.rst') as readme_file:
readme = readme_file.read()
-# install pycairo on Windows
-if os.name == 'nt':
- p = platform.architecture()
- is64bit = p[0] == '64bit'
- if sys.version[0:3] == '3.5':
- if is64bit:
- url = 'https://download.lfd.uci.edu/pythonlibs/t4jqbe6o/pycairo-1.18.1-cp35-cp35m-win_amd64.whl'
- else:
- url = 'https://download.lfd.uci.edu/pythonlibs/t4jqbe6o/pycairo-1.18.1-cp35-cp35m-win32.whl'
-
- if sys.version[0:3] == '3.6':
- if is64bit:
- url = 'https://download.lfd.uci.edu/pythonlibs/t4jqbe6o/pycairo-1.18.1-cp36-cp36m-win_amd64.whl'
- else:
- url = 'https://download.lfd.uci.edu/pythonlibs/t4jqbe6o/pycairo-1.18.1-cp36-cp36m-win32.whl'
-
- if sys.version[0:3] == '3.7':
- if is64bit:
- url = 'https://download.lfd.uci.edu/pythonlibs/t4jqbe6o/pycairo-1.18.1-cp37-cp37m-win_amd64.whl'
- else:
- url = 'https://download.lfd.uci.edu/pythonlibs/t4jqbe6o/pycairo-1.18.1-cp37-cp37m-win32.whl'
-
- try:
- import pycairo
- except:
- call_cmd = "pip install " + url
- os.system(call_cmd)
-
- import site
- import ctypes.util
-
- default_os_path = os.environ['PATH']
- os.environ['PATH'] = ''
- for s in site.getsitepackages():
- os.environ['PATH'] = os.environ['PATH'] + ';' + s + '\\cairo'
- os.environ['PATH'] = os.environ['PATH'] + ';' + default_os_path
- print(os.environ['PATH'])
- if ctypes.util.find_library('cairo') is not None:
- print("cairo installed: OK")
-else:
- try:
- import pycairo
- except:
- os.system("pip install pycairo==1.18.1")
-
def get_all_svg_files(directory='./svg/'):
ret = []
@@ -114,6 +68,6 @@ setup(
test_suite='tests',
tests_require=test_requirements,
url='https://gitlab.aicrowd.com/flatland/flatland',
- version='0.1.2',
+ version='0.2.0',
zip_safe=False,
)
diff --git a/tests/test_flatland_core_transition_map.py b/tests/test_flatland_core_transition_map.py
index 8013e912d93ea1125f4f7bdc622f2f4e3d4b2333..a414231619cfa924c2d33776f9f140cf88280517 100644
--- a/tests/test_flatland_core_transition_map.py
+++ b/tests/test_flatland_core_transition_map.py
@@ -5,19 +5,42 @@ from flatland.core.transition_map import GridTransitionMap
def test_grid4_get_transitions():
grid4_map = GridTransitionMap(2, 2, Grid4Transitions([]))
- assert grid4_map.get_transitions((0, 0, Grid4TransitionsEnum.NORTH)) == (0, 0, 0, 0)
+ assert grid4_map.get_transitions(0, 0, Grid4TransitionsEnum.NORTH) == (0, 0, 0, 0)
+ assert grid4_map.get_transitions(0, 0, Grid4TransitionsEnum.EAST) == (0, 0, 0, 0)
+ assert grid4_map.get_transitions(0, 0, Grid4TransitionsEnum.SOUTH) == (0, 0, 0, 0)
+ assert grid4_map.get_transitions(0, 0, Grid4TransitionsEnum.WEST) == (0, 0, 0, 0)
+ assert grid4_map.get_full_transitions(0, 0) == 0
+
grid4_map.set_transition((0, 0, Grid4TransitionsEnum.NORTH), Grid4TransitionsEnum.NORTH, 1)
- assert grid4_map.get_transitions((0, 0, Grid4TransitionsEnum.NORTH)) == (1, 0, 0, 0)
+ assert grid4_map.get_transitions(0, 0, Grid4TransitionsEnum.NORTH) == (1, 0, 0, 0)
+ assert grid4_map.get_transitions(0, 0, Grid4TransitionsEnum.EAST) == (0, 0, 0, 0)
+ assert grid4_map.get_transitions(0, 0, Grid4TransitionsEnum.SOUTH) == (0, 0, 0, 0)
+ assert grid4_map.get_transitions(0, 0, Grid4TransitionsEnum.WEST) == (0, 0, 0, 0)
+ assert grid4_map.get_full_transitions(0, 0) == pow(2, 15) # the most significant bit is on
+
+ grid4_map.set_transition((0, 0, Grid4TransitionsEnum.NORTH), Grid4TransitionsEnum.WEST, 1)
+ assert grid4_map.get_transitions(0, 0, Grid4TransitionsEnum.NORTH) == (1, 0, 0, 1)
+ assert grid4_map.get_transitions(0, 0, Grid4TransitionsEnum.EAST) == (0, 0, 0, 0)
+ assert grid4_map.get_transitions(0, 0, Grid4TransitionsEnum.SOUTH) == (0, 0, 0, 0)
+ assert grid4_map.get_transitions(0, 0, Grid4TransitionsEnum.WEST) == (0, 0, 0, 0)
+ # the most significant and the fourth most significant bits are on
+ assert grid4_map.get_full_transitions(0, 0) == pow(2, 15) + pow(2, 12)
+
grid4_map.set_transition((0, 0, Grid4TransitionsEnum.NORTH), Grid4TransitionsEnum.NORTH, 0)
- assert grid4_map.get_transitions((0, 0, Grid4TransitionsEnum.NORTH)) == (0, 0, 0, 0)
+ assert grid4_map.get_transitions(0, 0, Grid4TransitionsEnum.NORTH) == (0, 0, 0, 1)
+ assert grid4_map.get_transitions(0, 0, Grid4TransitionsEnum.EAST) == (0, 0, 0, 0)
+ assert grid4_map.get_transitions(0, 0, Grid4TransitionsEnum.SOUTH) == (0, 0, 0, 0)
+ assert grid4_map.get_transitions(0, 0, Grid4TransitionsEnum.WEST) == (0, 0, 0, 0)
+ # the fourth most significant bits are on
+ assert grid4_map.get_full_transitions(0, 0) == pow(2, 12)
def test_grid8_set_transitions():
grid8_map = GridTransitionMap(2, 2, Grid8Transitions([]))
- assert grid8_map.get_transitions((0, 0, Grid8TransitionsEnum.NORTH)) == (0, 0, 0, 0, 0, 0, 0, 0)
+ assert grid8_map.get_transitions(0, 0, Grid8TransitionsEnum.NORTH) == (0, 0, 0, 0, 0, 0, 0, 0)
grid8_map.set_transition((0, 0, Grid8TransitionsEnum.NORTH), Grid8TransitionsEnum.NORTH, 1)
- assert grid8_map.get_transitions((0, 0, Grid8TransitionsEnum.NORTH)) == (1, 0, 0, 0, 0, 0, 0, 0)
+ assert grid8_map.get_transitions(0, 0, Grid8TransitionsEnum.NORTH) == (1, 0, 0, 0, 0, 0, 0, 0)
grid8_map.set_transition((0, 0, Grid8TransitionsEnum.NORTH), Grid8TransitionsEnum.NORTH, 0)
- assert grid8_map.get_transitions((0, 0, Grid8TransitionsEnum.NORTH)) == (0, 0, 0, 0, 0, 0, 0, 0)
+ assert grid8_map.get_transitions(0, 0, Grid8TransitionsEnum.NORTH) == (0, 0, 0, 0, 0, 0, 0, 0)
# TODO GridTransitionMap