From 292549ec252554cbe84ac18f826f649ee9173bbd Mon Sep 17 00:00:00 2001
From: u214892 <u214892@sbb.ch>
Date: Thu, 26 Sep 2019 15:08:13 +0200
Subject: [PATCH] cleanup tutorials
---
README.md | 16 +-
docs/03_tutorials.rst | 7 +-
docs/04_specifications.rst | 1 +
docs/index.rst | 1 -
.../intro_observation_actions.rst | 16 +-
.../01_gettingstarted.rst} | 88 +-----
...nbuilder.rst => 02_observationbuilder.rst} | 5 +-
.../03_rail_and_schedule_generator.md | 72 +++++
docs/tutorials/04_stochasticity.md | 74 +++++
docs/tutorials/05_multispeed.md | 128 +++++++++
flatland_2.0.md | 272 +-----------------
11 files changed, 316 insertions(+), 364 deletions(-)
rename docs/{tutorials => specifications}/intro_observation_actions.rst (97%)
rename docs/{02_gettingstarted.rst => tutorials/01_gettingstarted.rst} (72%)
rename docs/tutorials/{intro_observationbuilder.rst => 02_observationbuilder.rst} (99%)
create mode 100644 docs/tutorials/03_rail_and_schedule_generator.md
create mode 100644 docs/tutorials/04_stochasticity.md
create mode 100644 docs/tutorials/05_multispeed.md
diff --git a/README.md b/README.md
index 2c00d901..14102def 100644
--- a/README.md
+++ b/README.md
@@ -37,7 +37,8 @@ $ conda install -c conda-forge cairosvg pycairo
$ conda install -c anaconda tk
```
-### Stable release
+### Install Flatland
+#### Stable Release
To install flatland, run this command in your terminal:
@@ -54,9 +55,9 @@ you through the process.
.. _Python installation guide: http://docs.python-guide.org/en/latest/starting/installation/
-### From sources
+#### From sources
-The sources for flatland can be downloaded from the `Gitlab repo`_.
+The sources for flatland can be downloaded from [gitlab](https://gitlab.aicrowd.com/flatland/flatland)
You can clone the public repository:
```console
@@ -69,7 +70,14 @@ Once you have a copy of the source, you can install it with:
$ python setup.py install
```
-.. _Gitlab repo: https://gitlab.aicrowd.com/flatland/flatland
+### Test installation
+
+Test that the installation works
+
+```console
+$ flatland-demo
+```
+
### Jupyter Canvas Widget
diff --git a/docs/03_tutorials.rst b/docs/03_tutorials.rst
index 249b0bd3..e862221d 100644
--- a/docs/03_tutorials.rst
+++ b/docs/03_tutorials.rst
@@ -1,2 +1,5 @@
-.. include:: tutorials/intro_observation_actions.rst
-.. include:: tutorials/intro_observationbuilder.rst
+.. include:: tutorials/01_gettingstarted.rst
+.. include:: tutorials/02_observationbuilder.rst
+.. include:: tutorials/03_rail_and_schedule_generator.rst
+.. include:: tutorials/04_stochasticity.rst
+.. include:: tutorials/05_multispeed.rst
diff --git a/docs/04_specifications.rst b/docs/04_specifications.rst
index 61e4dc68..4a7ffee6 100644
--- a/docs/04_specifications.rst
+++ b/docs/04_specifications.rst
@@ -1,6 +1,7 @@
.. include:: specifications/intro.rst
.. include:: specifications/core.rst
.. include:: specifications/railway.rst
+.. include:: specifications/intro_observation_actions.rst
.. include:: specifications/rendering.rst
.. include:: specifications/visualization.rst
.. include:: specifications/FAQ.rst
diff --git a/docs/index.rst b/docs/index.rst
index 1c10fd60..94efbc91 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -8,7 +8,6 @@ Welcome to flatland's documentation!
:caption: Contents:
01_readme
- 02_gettingstarted
03_tutorials_toc
04_specifications_toc
05_apidoc
diff --git a/docs/tutorials/intro_observation_actions.rst b/docs/specifications/intro_observation_actions.rst
similarity index 97%
rename from docs/tutorials/intro_observation_actions.rst
rename to docs/specifications/intro_observation_actions.rst
index 70723e30..1d9fd9c1 100644
--- a/docs/tutorials/intro_observation_actions.rst
+++ b/docs/specifications/intro_observation_actions.rst
@@ -1,10 +1,10 @@
-=============================
+
Observation and Action Spaces
-=============================
+----------------------------
This is an introduction to the three standard observations and the action space of **Flatland**.
Action Space
-============
+^^^^^^^^^^^^
Flatland is a railway simulation. Thus the actions of an agent are strongly limited to the railway network. This means that in many cases not all actions are valid.
The possible actions of an agent are
@@ -15,7 +15,7 @@ The possible actions of an agent are
- ``4`` **Stop**: This action causes the agent to stop.
Observation Spaces
-==================
+^^^^^^^^^^^^^^^^^^
In the **Flatland** environment we have included three basic observations to get started. The figure below illustrates the observation range of the different basic observation: ``Global``, ``Local Grid`` and ``Local Tree``.
.. image:: https://i.imgur.com/oo8EIYv.png
@@ -24,7 +24,7 @@ In the **Flatland** environment we have included three basic observations to get
Global Observation
-------------------
+~~~~~~~~~~~~~~~~~~
Gives a global observation of the entire rail environment.
The observation is composed of the following elements:
@@ -37,7 +37,7 @@ We encourage you to enhance this observation with any layer you think might help
It would also be possible to construct a global observation for a super agent that controls all agents at once.
Local Grid Observation
-----------------------
+~~~~~~~~~~~~~~~~~~~~~~
Gives a local observation of the rail environment around the agent.
The observation is composed of the following elements:
@@ -50,7 +50,7 @@ Be aware that this observation **does not** contain any clues about target locat
We encourage you to come up with creative ways to overcome this problem. In the tree observation below we introduce the concept of distance maps.
Tree Observation
-----------------
+~~~~~~~~~~~~~~~~
The tree observation is built by exploiting the graph structure of the railway network. The observation is generated by spanning a **4 branched tree** from the current position of the agent. Each branch follows the allowed transitions (backward branch only allowed at dead-ends) until a cell with multiple allowed transitions is reached. Here the information gathered along the branch is stored as a node in the tree.
The figure below illustrates how the tree observation is built:
@@ -73,7 +73,7 @@ The right side of the figure shows the resulting tree of the railway network on
Node Information
-----------------
+~~~~~~~~~~~~~~~~
Each node is filled with information gathered along the path to the node. Currently each node contains 9 features:
- 1: if own target lies on the explored branch the current distance from the agent in number of cells is stored.
diff --git a/docs/02_gettingstarted.rst b/docs/tutorials/01_gettingstarted.rst
similarity index 72%
rename from docs/02_gettingstarted.rst
rename to docs/tutorials/01_gettingstarted.rst
index 8bde9adf..9ca370a0 100644
--- a/docs/02_gettingstarted.rst
+++ b/docs/tutorials/01_gettingstarted.rst
@@ -1,6 +1,5 @@
-===============
-Getting Started
-===============
+Getting Started Tutorial
+========================
Overview
--------
@@ -16,9 +15,8 @@ To use flatland in a project:
import flatland
-Part 1 : Basic Usage
---------------------
-
+Simple Example 1 : Basic Usage
+------------------------------
The basic usage of RailEnv environments consists in creating a RailEnv object
endowed with a rail generator, that generates new rail networks on each reset,
and an observation generator object, that is supplied with environment-specific
@@ -120,7 +118,8 @@ The complete code for this part of the Getting Started guide can be found in
Part 2 : Training a Simple an Agent on Flatland
------------------------------------------------
+---------------------------------------------------------
+
This is a brief tutorial on how to train an agent on Flatland.
Here we use a simple random agent to illustrate the process on how to interact with the environment.
The corresponding code can be found in examples/training_example.py and in the baselines repository
@@ -187,77 +186,4 @@ This dictionary is then passed to the environment which checks the validity of a
The environment returns an array of new observations, reward dictionary for all agents as well as a flag for which agents are done.
This information can be used to update the policy of your agent and if done['__all__'] == True the episode terminates.
-Part 3 : Customizing Observations and Level Generators
-------------------------------------------------------
-
-Example code for generating custom observations given a RailEnv and to generate
-random rail maps are available in examples/custom_observation_example.py and
-examples/custom_railmap_example.py .
-
-Custom observations can be produced by deriving a new object from the
-core.env_observation_builder.ObservationBuilder base class, for example as follows:
-
-.. code-block:: python
-
- class CustomObs(ObservationBuilder):
- def __init__(self):
- self.observation_space = [5]
-
- def reset(self):
- return
-
- def get(self, handle):
- observation = handle*np.ones((5,))
- return observation
-
-It is important that an observation_space is defined with a list of dimensions
-of the returned observation tensors. get() returns the observation for each agent,
-of handle 'handle'.
-
-A RailEnv environment can then be created as usual:
-
-.. code-block:: python
-
- env = RailEnv(width=7,
- height=7,
- rail_generator=random_rail_generator(),
- number_of_agents=3,
- obs_builder_object=CustomObs())
-
-As for generating custom rail maps, the RailEnv class accepts a rail_generator
-argument that must be a function with arguments `width`, `height`, `num_agents`,
-and `num_resets=0`, and that has to return a GridTransitionMap object (the rail map),
-and three lists of tuples containing the (row,column) coordinates of each of
-num_agent agents, their initial orientation **(0=North, 1=East, 2=South, 3=West)**,
-and the position of their targets.
-
-For example, the following custom rail map generator returns an empty map of
-size (height, width), with no agents (regardless of num_agents):
-
-.. code-block:: python
-
- def custom_rail_generator():
- def generator(width, height, num_agents=0, num_resets=0):
- rail_trans = RailEnvTransitions()
- grid_map = GridTransitionMap(width=width, height=height, transitions=rail_trans)
- rail_array = grid_map.grid
- rail_array.fill(0)
-
- agents_positions = []
- agents_direction = []
- agents_target = []
-
- return grid_map, agents_positions, agents_direction, agents_target
- return generator
-
-It is worth to note that helpful utilities to manage RailEnv environments and their
-related data structures are available in 'envs.env_utils'. In particular,
-envs.env_utils.get_rnd_agents_pos_tgt_dir_on_rail is fairly handy to fill in
-random (but consistent) agents along with their targets and initial directions,
-given a rail map (GridTransitionMap object) and the desired number of agents:
-
-.. code-block:: python
-
- agents_position, agents_direction, agents_target = get_rnd_agents_pos_tgt_dir_on_rail(
- rail_map,
- num_agents)
+The full source code of this example can be found in `examples/training_example.py <https://gitlab.aicrowd.com/flatland/flatland/blob/master/examples/training_example.py>`_.
diff --git a/docs/tutorials/intro_observationbuilder.rst b/docs/tutorials/02_observationbuilder.rst
similarity index 99%
rename from docs/tutorials/intro_observationbuilder.rst
rename to docs/tutorials/02_observationbuilder.rst
index 50d65fd4..fd5decae 100644
--- a/docs/tutorials/intro_observationbuilder.rst
+++ b/docs/tutorials/02_observationbuilder.rst
@@ -1,6 +1,5 @@
-==============================================================
-Getting Started with custom observations and custom predictors
-==============================================================
+Custom observations and custom predictors Tutorial
+==================================================
Overview
--------
diff --git a/docs/tutorials/03_rail_and_schedule_generator.md b/docs/tutorials/03_rail_and_schedule_generator.md
new file mode 100644
index 00000000..5a236a6d
--- /dev/null
+++ b/docs/tutorials/03_rail_and_schedule_generator.md
@@ -0,0 +1,72 @@
+# Level Generation Tutorial
+
+We are currently working on different new level generators and you can expect that the levels in the submission testing will not all come from just one but rather different level generators to be sure that the controllers can handle any railway specific challenge.
+
+Let's have a look at the `sparse_rail_generator`.
+
+## Sparse Rail Generator
+
+
+The idea behind the sparse rail generator is to mimic classic railway structures where dense nodes (cities) are sparsely connected to each other and where you have to manage traffic flow between the nodes efficiently.
+The cities in this level generator are much simplified in comparison to real city networks but it mimics parts of the problems faced in daily operations of any railway company.
+
+There are a few parameters you can tune to build your own map and test different complexity levels of the levels.
+**Warning** some combinations of parameters do not go well together and will lead to infeasible level generation.
+In the worst case, the level generator currently issues a warning when it cannot build the environment according to the parameters provided.
+This will lead to a crash of the whole env.
+We are currently working on improvements here and are **happy for any suggestions from your side**.
+
+To build an environment you instantiate a `RailEnv` as follows:
+
+```python
+ Initialize the generator
+rail_generator=sparse_rail_generator(
+ num_cities=10, # Number of cities in map
+ num_intersections=10, # Number of interesections in map
+ num_trainstations=50, # Number of possible start/targets on map
+ min_node_dist=6, # Minimal distance of nodes
+ node_radius=3, # Proximity of stations to city center
+ num_neighb=3, # Number of connections to other cities
+ seed=5, # Random seed
+ grid_mode=False # Ordered distribution of nodes
+)
+
+ Build the environment
+env = RailEnv(
+ width=50,
+ height=50,
+ rail_generator=rail_generator
+ schedule_generator=sparse_schedule_generator(),
+ number_of_agents=10,
+ obs_builder_object=TreeObsForRailEnv(max_depth=3,predictor=shortest_path_predictor)
+)
+```
+
+You can see that you now need both a `rail_generator` and a `schedule_generator` to generate a level. These need to work nicely together. The `rail_generator` will only generate the railway infrastructure and provide hints to the `schedule_generator` about where to place agents. The `schedule_generator` will then generate a schedule, meaning it places agents at different train stations and gives them tasks by providing individual targets.
+
+You can tune the following parameters in the `sparse_rail_generator`:
+
+- `num_cities` is the number of cities on a map. Cities are the only nodes that can host start and end points for agent tasks (Train stations). Here you have to be carefull that the number is not too high as all the cities have to fit on the map. When `grid_mode=False` you have to be carefull when chosing `min_node_dist` because leves will fails if not all cities (and intersections) can be placed with at least `min_node_dist` between them.
+- `num_intersections` is the number of nodes that don't hold any trainstations. They are also the first priority that a city connects to. We use these to allow for sparse connections between cities.
+- `num_trainstations` defines the *Total* number of trainstations in the network. This also sets the max number of allowed agents in the environment. This is also a delicate parameter as there is only a limitid amount of space available around nodes and thus if the number is too high the level generation will fail. *Important*: Only the number of agents provided to the environment will actually produce active train stations. The others will just be present as dead-ends (See figures below).
+- `min_node_dist` is only used if `grid_mode=False` and represents the minimal distance between two nodes.
+- `node_radius` defines the extent of a city. Each trainstation is placed at a distance to the closes city node that is smaller or equal to this number.
+- `num_neighb`defines the number of neighbouring nodes that connect to each other. Thus this changes the connectivity and thus the amount of alternative routes in the network.
+- `grid_mode` True -> Nodes evenly distriubted in env, False-> Random distribution of nodes
+- `enhance_intersection`: True -> Extra rail elements added at intersections
+- `seed` is used to initialize the random generator
+
+
+If you run into any bugs with sets of parameters please let us know.
+
+Here is a network with `grid_mode=False` and the parameters from above.
+
+
+
+and here with `grid_mode=True`
+
+
+
+## Example code
+
+To see all the changes in action you can just run the `flatland_example_2_0.py` file in the examples folder. The file can be found [here](https://gitlab.aicrowd.com/flatland/flatland/blob/master/examples/flatland_2_0_example.py).
diff --git a/docs/tutorials/04_stochasticity.md b/docs/tutorials/04_stochasticity.md
new file mode 100644
index 00000000..201e359b
--- /dev/null
+++ b/docs/tutorials/04_stochasticity.md
@@ -0,0 +1,74 @@
+# Stochasticity Tutorial
+
+Another area where we improved **Flat**land 2.0 are stochastic events added during the episodes.
+This is very common for railway networks where the initial plan usually needs to be rescheduled during operations as minor events such as delayed departure from trainstations, malfunctions on trains or infrastructure or just the weather lead to delayed trains.
+
+We implemted a poisson process to simulate delays by stopping agents at random times for random durations. The parameters necessary for the stochastic events can be provided when creating the environment.
+
+```python
+# Use a the malfunction generator to break agents from time to time
+
+stochastic_data = {
+ 'prop_malfunction': 0.5, # Percentage of defective agents
+ 'malfunction_rate': 30, # Rate of malfunction occurence
+ 'min_duration': 3, # Minimal duration of malfunction
+ 'max_duration': 10 # Max duration of malfunction
+}
+```
+
+The parameters are as follows:
+
+- `prop_malfunction` is the proportion of agents that can malfunction. `1.0` means that each agent can break.
+- `malfunction_rate` is the mean rate of the poisson process in number of environment steps.
+- `min_duration` and `max_duration` set the range of malfunction durations. They are sampled uniformly
+
+You can introduce stochasticity by simply creating the env as follows:
+
+```python
+env = RailEnv(
+ ...
+ stochastic_data=stochastic_data, # Malfunction data generator
+ ...
+)
+```
+In your controller, you can check whether an agent is malfunctioning:
+```python
+obs, rew, done, info = env.step(actions)
+...
+action_dict = dict()
+for a in range(env.get_num_agents()):
+ if info['malfunction'][a] == 0:
+ action_dict.update({a: ...})
+
+# Custom observation builder
+tree_observation = TreeObsForRailEnv(max_depth=2, predictor=ShortestPathPredictorForRailEnv())
+
+# Different agent types (trains) with different speeds.
+speed_ration_map = {1.: 0.25, # Fast passenger train
+ 1. / 2.: 0.25, # Fast freight train
+ 1. / 3.: 0.25, # Slow commuter train
+ 1. / 4.: 0.25} # Slow freight train
+
+env = RailEnv(width=50,
+ height=50,
+ rail_generator=sparse_rail_generator(num_cities=20, # Number of cities in map (where train stations are)
+ num_intersections=5, # Number of intersections (no start / target)
+ num_trainstations=15, # Number of possible start/targets on map
+ min_node_dist=3, # Minimal distance of nodes
+ node_radius=2, # Proximity of stations to city center
+ num_neighb=4, # Number of connections to other cities/intersections
+ seed=15, # Random seed
+ grid_mode=True,
+ enhance_intersection=True
+ ),
+ schedule_generator=sparse_schedule_generator(speed_ration_map),
+ number_of_agents=10,
+ stochastic_data=stochastic_data, # Malfunction data generator
+ obs_builder_object=tree_observation)
+```
+
+You will quickly realize that this will lead to unforeseen difficulties which means that **your controller** needs to observe the environment at all times to be able to react to the stochastic events.
+
+## Example code
+
+To see all the changes in action you can just run the `flatland_example_2_0.py` file in the examples folder. The file can be found [here](https://gitlab.aicrowd.com/flatland/flatland/blob/master/examples/flatland_2_0_example.py).
diff --git a/docs/tutorials/05_multispeed.md b/docs/tutorials/05_multispeed.md
new file mode 100644
index 00000000..99db7ee6
--- /dev/null
+++ b/docs/tutorials/05_multispeed.md
@@ -0,0 +1,128 @@
+# Different speed profiles Tutorial
+
+One of the main contributions to the complexity of railway network operations stems from the fact that all trains travel at different speeds while sharing a very limited railway network.
+In **Flat**land 2.0 this feature will be enabled as well and will lead to much more complex configurations. Here we count on your support if you find bugs or improvements :).
+
+The different speed profiles can be generated using the `schedule_generator`, where you can actually chose as many different speeds as you like.
+Keep in mind that the *fastest speed* is 1 and all slower speeds must be between 1 and 0.
+For the submission scoring you can assume that there will be no more than 5 speed profiles.
+
+
+
+Later versions of **Flat**land might have varying speeds during episodes. Therefore, we return the agent speeds.
+Notice that we do not guarantee that the speed will be computed at each step, but if not costly we will return it at each step.
+In your controller, you can get the agents' speed from the `info` returned by `step`:
+```python
+obs, rew, done, info = env.step(actions)
+...
+for a in range(env.get_num_agents()):
+ speed = info['speed'][a]
+```
+
+## Actions and observation with different speed levels
+
+Because the different speeds are implemented as fractions the agents ability to perform actions has been updated.
+We **do not allow actions to change within the cell **.
+This means that each agent can only chose an action to be taken when entering a cell.
+This action is then executed when a step to the next cell is valid. For example
+
+- Agent enters switch and choses to deviate left. Agent fractional speed is 1/4 and thus the agent will take 4 time steps to complete its journey through the cell. On the 4th time step the agent will leave the cell deviating left as chosen at the entry of the cell.
+ - All actions chosen by the agent during its travels within a cell are ignored
+ - Agents can make observations at any time step. Make sure to discard observations without any information. See this [example](https://gitlab.aicrowd.com/flatland/baselines/blob/master/torch_training/training_navigation.py) for a simple implementation.
+- The environment checks if agent is allowed to move to next cell only at the time of the switch to the next cell
+
+In your controller, you can check whether an agent requires an action by checking `info`:
+```python
+obs, rew, done, info = env.step(actions)
+...
+action_dict = dict()
+for a in range(env.get_num_agents()):
+ if info['action_required'][a] and info['malfunction'][a] == 0:
+ action_dict.update({a: ...})
+
+```
+Notice that `info['action_required'][a]` does not mean that the action will have an effect:
+if the next cell is blocked or the agent breaks down, the action cannot be performed and an action will be required again in the next step.
+
+## Rail Generators and Schedule Generators
+The separation between rail generator and schedule generator reflects the organisational separation in the railway domain
+- Infrastructure Manager (IM): is responsible for the layout and maintenance of tracks
+- Railway Undertaking (RU): operates trains on the infrastructure
+Usually, there is a third organisation, which ensures discrimination-free access to the infrastructure for concurrent requests for the infrastructure in a **schedule planning phase**.
+However, in the **Flat**land challenge, we focus on the re-scheduling problem during live operations.
+
+Technically,
+```python
+RailGeneratorProduct = Tuple[GridTransitionMap, Optional[Any]]
+RailGenerator = Callable[[int, int, int, int], RailGeneratorProduct]
+
+AgentPosition = Tuple[int, int]
+ScheduleGeneratorProduct = Tuple[List[AgentPosition], List[AgentPosition], List[AgentPosition], List[float]]
+ScheduleGenerator = Callable[[GridTransitionMap, int, Optional[Any]], ScheduleGeneratorProduct]
+```
+
+We can then produce `RailGenerator`s by currying:
+```python
+def sparse_rail_generator(num_cities=5, num_intersections=4, num_trainstations=2, min_node_dist=20, node_radius=2,
+ num_neighb=3, grid_mode=False, enhance_intersection=False, seed=0):
+
+ def generator(width, height, num_agents, num_resets=0):
+
+ # generate the grid and (optionally) some hints for the schedule_generator
+ ...
+
+ return grid_map, {'agents_hints': {
+ 'num_agents': num_agents,
+ 'agent_start_targets_nodes': agent_start_targets_nodes,
+ 'train_stations': train_stations
+ }}
+
+ return generator
+```
+And, similarly, `ScheduleGenerator`s:
+```python
+def sparse_schedule_generator(speed_ratio_map: Mapping[float, float] = None) -> ScheduleGenerator:
+ def generator(rail: GridTransitionMap, num_agents: int, hints: Any = None):
+ # place agents:
+ # - initial position
+ # - initial direction
+ # - (initial) speed
+ # - malfunction
+ ...
+
+ return agents_position, agents_direction, agents_target, speeds, agents_malfunction
+
+ return generator
+```
+Notice that the `rail_generator` may pass `agents_hints` to the `schedule_generator` which the latter may interpret.
+For instance, the way the `sparse_rail_generator` generates the grid, it already determines the agent's goal and target.
+Hence, `rail_generator` and `schedule_generator` have to match if `schedule_generator` presupposes some specific `agents_hints`.
+
+The environment's `reset` takes care of applying the two generators:
+```python
+ def __init__(self,
+ ...
+ rail_generator: RailGenerator = random_rail_generator(),
+ schedule_generator: ScheduleGenerator = random_schedule_generator(),
+ ...
+ ):
+ self.rail_generator: RailGenerator = rail_generator
+ self.schedule_generator: ScheduleGenerator = schedule_generator
+
+ def reset(self, regen_rail=True, replace_agents=True):
+ rail, optionals = self.rail_generator(self.width, self.height, self.get_num_agents(), self.num_resets)
+
+ ...
+
+ if replace_agents:
+ agents_hints = None
+ if optionals and 'agents_hints' in optionals:
+ agents_hints = optionals['agents_hints']
+ self.agents_static = EnvAgentStatic.from_lists(
+ *self.schedule_generator(self.rail, self.get_num_agents(), hints=agents_hints))
+```
+
+
+## Example code
+
+To see all the changes in action you can just run the `flatland_example_2_0.py` file in the examples folder. The file can be found [here](https://gitlab.aicrowd.com/flatland/flatland/blob/master/examples/flatland_2_0_example.py).
diff --git a/flatland_2.0.md b/flatland_2.0.md
index 27281911..501369bd 100644
--- a/flatland_2.0.md
+++ b/flatland_2.0.md
@@ -11,273 +11,15 @@ Thus the following changes are coming in the next version to be closer to real r
- **Stochastic Events** cause agents to stop and get stuck for different numbers of time steps.
- **Different Speed Classes** allow agents to move at different speeds and thus enhance complexity in the search for optimal solutions.
-Below we explain these changes in more detail and how you can play with their parametrization. We appreciate *your feedback* on the performance and the difficulty on these levels to help us shape the best possible **Flat**land 2.0 environment.
-## Generate levels
-
-We are currently working on different new level generators and you can expect that the levels in the submission testing will not all come from just one but rather different level generators to be sure that the controllers can handle any railway specific challenge.
-
-Let's have a look at the `sparse_rail_generator`.
-
-### Sparse Rail Generator
-
-
-The idea behind the sparse rail generator is to mimic classic railway structures where dense nodes (cities) are sparsely connected to each other and where you have to manage traffic flow between the nodes efficiently.
-The cities in this level generator are much simplified in comparison to real city networks but it mimics parts of the problems faced in daily operations of any railway company.
-
-There are a few parameters you can tune to build your own map and test different complexity levels of the levels.
-**Warning** some combinations of parameters do not go well together and will lead to infeasible level generation.
-In the worst case, the level generator currently issues a warning when it cannot build the environment according to the parameters provided.
-This will lead to a crash of the whole env.
-We are currently working on improvements here and are **happy for any suggestions from your side**.
-
-To build an environment you instantiate a `RailEnv` as follows:
-
-```python
-# Initialize the generator
-rail_generator=sparse_rail_generator(
- num_cities=10, # Number of cities in map
- num_intersections=10, # Number of interesections in map
- num_trainstations=50, # Number of possible start/targets on map
- min_node_dist=6, # Minimal distance of nodes
- node_radius=3, # Proximity of stations to city center
- num_neighb=3, # Number of connections to other cities
- seed=5, # Random seed
- grid_mode=False # Ordered distribution of nodes
-)
-
-# Build the environment
-env = RailEnv(
- width=50,
- height=50,
- rail_generator=rail_generator
- schedule_generator=sparse_schedule_generator(),
- number_of_agents=10,
- obs_builder_object=TreeObsForRailEnv(max_depth=3,predictor=shortest_path_predictor)
-)
-```
-
-You can see that you now need both a `rail_generator` and a `schedule_generator` to generate a level. These need to work nicely together. The `rail_generator` will only generate the railway infrastructure and provide hints to the `schedule_generator` about where to place agents. The `schedule_generator` will then generate a schedule, meaning it places agents at different train stations and gives them tasks by providing individual targets.
-
-You can tune the following parameters in the `sparse_rail_generator`:
-
-- `num_cities` is the number of cities on a map. Cities are the only nodes that can host start and end points for agent tasks (Train stations). Here you have to be carefull that the number is not too high as all the cities have to fit on the map. When `grid_mode=False` you have to be carefull when chosing `min_node_dist` because leves will fails if not all cities (and intersections) can be placed with at least `min_node_dist` between them.
-- `num_intersections` is the number of nodes that don't hold any trainstations. They are also the first priority that a city connects to. We use these to allow for sparse connections between cities.
-- `num_trainstations` defines the *Total* number of trainstations in the network. This also sets the max number of allowed agents in the environment. This is also a delicate parameter as there is only a limitid amount of space available around nodes and thus if the number is too high the level generation will fail. *Important*: Only the number of agents provided to the environment will actually produce active train stations. The others will just be present as dead-ends (See figures below).
-- `min_node_dist` is only used if `grid_mode=False` and represents the minimal distance between two nodes.
-- `node_radius` defines the extent of a city. Each trainstation is placed at a distance to the closes city node that is smaller or equal to this number.
-- `num_neighb`defines the number of neighbouring nodes that connect to each other. Thus this changes the connectivity and thus the amount of alternative routes in the network.
-- `grid_mode` True -> Nodes evenly distriubted in env, False-> Random distribution of nodes
-- `enhance_intersection`: True -> Extra rail elements added at intersections
-- `seed` is used to initialize the random generator
-
-
-If you run into any bugs with sets of parameters please let us know.
-
-Here is a network with `grid_mode=False` and the parameters from above.
-
-
-
-and here with `grid_mode=True`
-
-
-
-## Add Stochasticity
-
-Another area where we improved **Flat**land 2.0 are stochastic events added during the episodes.
-This is very common for railway networks where the initial plan usually needs to be rescheduled during operations as minor events such as delayed departure from trainstations, malfunctions on trains or infrastructure or just the weather lead to delayed trains.
-
-We implemted a poisson process to simulate delays by stopping agents at random times for random durations. The parameters necessary for the stochastic events can be provided when creating the environment.
-
-```python
-# Use a the malfunction generator to break agents from time to time
-
-stochastic_data = {
- 'prop_malfunction': 0.5, # Percentage of defective agents
- 'malfunction_rate': 30, # Rate of malfunction occurence
- 'min_duration': 3, # Minimal duration of malfunction
- 'max_duration': 10 # Max duration of malfunction
-}
-```
-
-The parameters are as follows:
-
-- `prop_malfunction` is the proportion of agents that can malfunction. `1.0` means that each agent can break.
-- `malfunction_rate` is the mean rate of the poisson process in number of environment steps.
-- `min_duration` and `max_duration` set the range of malfunction durations. They are sampled uniformly
-
-You can introduce stochasticity by simply creating the env as follows:
-
-```python
-env = RailEnv(
- ...
- stochastic_data=stochastic_data, # Malfunction data generator
- ...
-)
-```
-In your controller, you can check whether an agent is malfunctioning:
-```python
-obs, rew, done, info = env.step(actions)
-...
-action_dict = dict()
-for a in range(env.get_num_agents()):
- if info['malfunction'][a] == 0:
- action_dict.update({a: ...})
-
-# Custom observation builder
-tree_observation = TreeObsForRailEnv(max_depth=2, predictor=ShortestPathPredictorForRailEnv())
-
-# Different agent types (trains) with different speeds.
-speed_ration_map = {1.: 0.25, # Fast passenger train
- 1. / 2.: 0.25, # Fast freight train
- 1. / 3.: 0.25, # Slow commuter train
- 1. / 4.: 0.25} # Slow freight train
-
-env = RailEnv(width=50,
- height=50,
- rail_generator=sparse_rail_generator(num_cities=20, # Number of cities in map (where train stations are)
- num_intersections=5, # Number of intersections (no start / target)
- num_trainstations=15, # Number of possible start/targets on map
- min_node_dist=3, # Minimal distance of nodes
- node_radius=2, # Proximity of stations to city center
- num_neighb=4, # Number of connections to other cities/intersections
- seed=15, # Random seed
- grid_mode=True,
- enhance_intersection=True
- ),
- schedule_generator=sparse_schedule_generator(speed_ration_map),
- number_of_agents=10,
- stochastic_data=stochastic_data, # Malfunction data generator
- obs_builder_object=tree_observation)
-```
-
-You will quickly realize that this will lead to unforeseen difficulties which means that **your controller** needs to observe the environment at all times to be able to react to the stochastic events.
-
-## Add different speed profiles
-
-One of the main contributions to the complexity of railway network operations stems from the fact that all trains travel at different speeds while sharing a very limited railway network.
-In **Flat**land 2.0 this feature will be enabled as well and will lead to much more complex configurations. Here we count on your support if you find bugs or improvements :).
-
-The different speed profiles can be generated using the `schedule_generator`, where you can actually chose as many different speeds as you like.
-Keep in mind that the *fastest speed* is 1 and all slower speeds must be between 1 and 0.
-For the submission scoring you can assume that there will be no more than 5 speed profiles.
-
-
-
-Later versions of **Flat**land might have varying speeds during episodes. Therefore, we return the agent speeds.
-Notice that we do not guarantee that the speed will be computed at each step, but if not costly we will return it at each step.
-In your controller, you can get the agents' speed from the `info` returned by `step`:
-```python
-obs, rew, done, info = env.step(actions)
-...
-for a in range(env.get_num_agents()):
- speed = info['speed'][a]
-```
-
-## Actions and observation with different speed levels
-
-Because the different speeds are implemented as fractions the agents ability to perform actions has been updated.
-We **do not allow actions to change within the cell **.
-This means that each agent can only chose an action to be taken when entering a cell.
-This action is then executed when a step to the next cell is valid. For example
-
-- Agent enters switch and choses to deviate left. Agent fractional speed is 1/4 and thus the agent will take 4 time steps to complete its journey through the cell. On the 4th time step the agent will leave the cell deviating left as chosen at the entry of the cell.
- - All actions chosen by the agent during its travels within a cell are ignored
- - Agents can make observations at any time step. Make sure to discard observations without any information. See this [example](https://gitlab.aicrowd.com/flatland/baselines/blob/master/torch_training/training_navigation.py) for a simple implementation.
-- The environment checks if agent is allowed to move to next cell only at the time of the switch to the next cell
-
-In your controller, you can check whether an agent requires an action by checking `info`:
-```python
-obs, rew, done, info = env.step(actions)
-...
-action_dict = dict()
-for a in range(env.get_num_agents()):
- if info['action_required'][a] and info['malfunction'][a] == 0:
- action_dict.update({a: ...})
-
-```
-Notice that `info['action_required'][a]` does not mean that the action will have an effect:
-if the next cell is blocked or the agent breaks down, the action cannot be performed and an action will be required again in the next step.
-
-## Rail Generators and Schedule Generators
-The separation between rail generator and schedule generator reflects the organisational separation in the railway domain
-- Infrastructure Manager (IM): is responsible for the layout and maintenance of tracks
-- Railway Undertaking (RU): operates trains on the infrastructure
-Usually, there is a third organisation, which ensures discrimination-free access to the infrastructure for concurrent requests for the infrastructure in a **schedule planning phase**.
-However, in the **Flat**land challenge, we focus on the re-scheduling problem during live operations.
-
-Technically,
-```python
-RailGeneratorProduct = Tuple[GridTransitionMap, Optional[Any]]
-RailGenerator = Callable[[int, int, int, int], RailGeneratorProduct]
-
-AgentPosition = Tuple[int, int]
-ScheduleGeneratorProduct = Tuple[List[AgentPosition], List[AgentPosition], List[AgentPosition], List[float]]
-ScheduleGenerator = Callable[[GridTransitionMap, int, Optional[Any]], ScheduleGeneratorProduct]
-```
-
-We can then produce `RailGenerator`s by currying:
-```python
-def sparse_rail_generator(num_cities=5, num_intersections=4, num_trainstations=2, min_node_dist=20, node_radius=2,
- num_neighb=3, grid_mode=False, enhance_intersection=False, seed=0):
-
- def generator(width, height, num_agents, num_resets=0):
-
- # generate the grid and (optionally) some hints for the schedule_generator
- ...
-
- return grid_map, {'agents_hints': {
- 'num_agents': num_agents,
- 'agent_start_targets_nodes': agent_start_targets_nodes,
- 'train_stations': train_stations
- }}
-
- return generator
-```
-And, similarly, `ScheduleGenerator`s:
-```python
-def sparse_schedule_generator(speed_ratio_map: Mapping[float, float] = None) -> ScheduleGenerator:
- def generator(rail: GridTransitionMap, num_agents: int, hints: Any = None):
- # place agents:
- # - initial position
- # - initial direction
- # - (initial) speed
- # - malfunction
- ...
-
- return agents_position, agents_direction, agents_target, speeds, agents_malfunction
-
- return generator
-```
-Notice that the `rail_generator` may pass `agents_hints` to the `schedule_generator` which the latter may interpret.
-For instance, the way the `sparse_rail_generator` generates the grid, it already determines the agent's goal and target.
-Hence, `rail_generator` and `schedule_generator` have to match if `schedule_generator` presupposes some specific `agents_hints`.
-
-The environment's `reset` takes care of applying the two generators:
-```python
- def __init__(self,
- ...
- rail_generator: RailGenerator = random_rail_generator(),
- schedule_generator: ScheduleGenerator = random_schedule_generator(),
- ...
- ):
- self.rail_generator: RailGenerator = rail_generator
- self.schedule_generator: ScheduleGenerator = schedule_generator
-
- def reset(self, regen_rail=True, replace_agents=True):
- rail, optionals = self.rail_generator(self.width, self.height, self.get_num_agents(), self.num_resets)
-
- ...
-
- if replace_agents:
- agents_hints = None
- if optionals and 'agents_hints' in optionals:
- agents_hints = optionals['agents_hints']
- self.agents_static = EnvAgentStatic.from_lists(
- *self.schedule_generator(self.rail, self.get_num_agents(), hints=agents_hints))
-```
+We explain these changes in more detail and how you can play with their parametrization in Tutorials 3--5:
+* [Tutorials](https://gitlab.aicrowd.com/flatland/flatland/tree/master/docs/tutorials)
+We appreciate *your feedback* on the performance and the difficulty on these levels to help us shape the best possible **Flat**land 2.0 environment.
## Example code
-To see all the changes in action you can just run the `flatland_example_2_0.py` file in the examples folder. The file can be found [here](https://gitlab.aicrowd.com/flatland/flatland/blob/master/examples/flatland_2_0_example.py).
+To see all the changes in action you can just run the
+* [examples/flatland_example_2_0.py](https://gitlab.aicrowd.com/flatland/flatland/blob/master/examples/flatland_2_0_example.py)
+
+example.
--
GitLab