changelog.md

+Changelog
+==========
+Changes since Flatland 2.1.0
+--------------------------
+
+### Changes in 'schedule_generators'
+- Schedule generators now provide the max number of steps allowed per episode
+- Pickle files generated with older versions of Flatland need to be regenerated in order to include `_max_episode_steps`
+Changes since Flatland 2.0.0
+--------------------------
+### Changes in `EnvAgent`
+- class `EnvAgentStatic` was removed, so there is only class `EnvAgent` left which should simplify the handling of agents. The member `self.agents_static` of `RailEnv` was therefore also removed. Old Scence saved as pickle files cannot be loaded anymore.
+
+### Changes in malfunction behavior
+- agent attribute `next_malfunction`is not used anymore, it will be removed fully in future versions.
+- `break_agent()` function is introduced which induces malfunctions in agent according to poisson process
+- `_fix_agent_after_malfunction()` fixes agents after attribute `malfunction == 0`
+- Introduced the concept of malfunction generators. Here you can add different malfunction models in future updates. Currently it only loads from files and parameters.
+
+### Changes in `Environment`
+- moving of member variable `distance_map_computed` to new class `DistanceMap`
+
+### Changes in rail generator and `RailEnv`
+- renaming of `distance_maps` into `distance_map`
+- by default the reset method of RailEnv is not called in the constructor of RailEnv anymore (compliance for OpenAI Gym). Therefore the reset method needs to be called after the creation of a RailEnv object
+- renaming of parameters RailEnv.reset(): from `regen_rail` to `regenerate_rail`, from `replace_agents` to `regenerate_schedule`
+
+### Changes in schedule generation
+- return value of schedule generator has changed to the named tuple `Schedule`. From the point of view of a consumer, nothing has changed, this is just a type hint which is introduced where the attributes of `Schedule` have names.
+
+Changes since Flatland 1.0.0
+--------------------------
+### Changes in stock predictors
+The stock `ShortestPathPredictorForRailEnv` now respects the different agent speeds and updates their prediction accordingly.
+
+### Changes in stock observation biulders
+
+- `TreeObsForRailEnv` now has **11** features!
+    - 10th feature now indicates if a malfunctioning agent has been detected and how long the malfunction will still be present
+    - 11th feautre now indicates the minimal observed fractional speed of agents traveling in the same direction
+- `GlobalObsForRailEnv` now has new features!
+    - Targets and other agent targets still represented in same way
+    - `obs_agents_state` now contains 4 channels
+        - 0th channel -> agent direction at agent position
+        - 1st channel -> other agents direction at their positions
+        - 2nd channel -> all agent malfunction duration at their positions
+        - 3rd channel -> all agent fractional speeds at their positions
+- `LocalObsForRailEnv` was not update to Flatland 2.0 because it was never used by participants of the challenge.
+
+
+### Changes in level generation
+
+
+- Separation of `schedule_generator` from `rail_generator`:
+  - Renaming of `flatland/envs/generators.py` to `flatland/envs/rail_generators.py`
+  - `rail_generator` now only returns the grid and optionally hints (a python dictionary); the hints are currently use for distance_map and communication of start and goal position in complex rail generator.
+  - `schedule_generator` takes a `GridTransitionMap` and the number of agents and optionally the `agents_hints` field of the hints dictionary.
+  - Inrodcution of types hints:
+
+```python
+RailGeneratorProduct = Tuple[GridTransitionMap, Optional[Any]]
+RailGenerator = Callable[[int, int, int, int], RailGeneratorProduct]
+AgentPosition = Tuple[int, int]
+ScheduleGeneratorProduct = Tuple[List[AgentPosition], List[AgentPosition], List[AgentPosition], List[float]]
+ScheduleGenerator = Callable[[GridTransitionMap, int, Optional[Any]], ScheduleGeneratorProduct]
+```
+
+### Multi Speed
+
+- Different agent speeds are introduced. Agents now travel at a max speed which is a fraction. Meaning that they only advance parts within a cell and need several steps to move to the next cell.
+    - Fastest speed is 1. At this speed an agent can move to a new cell at each time step t.
+    - Slower speeds are smaller than one. At each time step an agent moves the fraction of its speed forward within a cell. It only changes cell when it's fractional position is greater or equal to 1.
+    - Multi-speed introduces the challenge of ordering the trains correctly when traveling in the same direction.
+- Agents always travel at their full speed when moving.
+
+To set up multiple speeds you have to modify the `agent.speed_data` within your `schedule_generator`. See [this file](https://gitlab.aicrowd.com/flatland/flatland/blob/master/flatland/envs/schedule_generators.py#L59) for a good example.
+
+**ATTENTION** multi speed means that the agents actions are not registered on every time step. Only at new cell entry can new actions be chosen! Beware to respect this with your controller as actions are only important at the specific time steps! This is shown as an example in the [navigation training](https://gitlab.aicrowd.com/flatland/baselines/blob/master/torch_training/training_navigation.py#L163)
+
+### Stochastic events
+Just like in real-worl transportation systems we introduced stochastic events to disturb normal traffic flow. Currently we implemented a malfunction process that stops agents at random time intervalls for a random time of duration.
+Currently the Flatland environment can be initiated with the following poisson process parameters:
+
+```python
+# Use a the malfunction generator to break agents from time to time
+stochastic_data = {'prop_malfunction': 0.1,  # Percentage of defective agents
+                   'malfunction_rate': 30,  # Rate of malfunction occurence
+                   'min_duration': 3,  # Minimal duration of malfunction
+                   'max_duration': 20  # Max duration of malfunction
+                   }
+```
+The duration of a malfunction is uniformly drawn from the intervall `[min_duration,max_duration0]` and the occurance of malfunctions follows a point poisson process with mean rate `malfunctin_rate`.
+
+**!!!!IMPORTANT!!!!** Once a malfunction duration has finished, the agent will **automatically** resume movement. This is important because otherwise it can get stuck in fractional positions and your code might forget to restart the agent at the first possible time. Therefore this has been automated. You can however stop the agent again at the next cell. This might in rare occasions lead to unexpected behavior, we are looking into this and will push a fix soon.
+
+
+## Baselines repository
+
+The baselines repository is not yet fully updated to handle multi-speed and stochastic events. Training needs to be modified to omitt all states inbetween the states where an agent can chose an action. Simple navigation training is already up to date. See [here](https://gitlab.aicrowd.com/flatland/baselines/blob/master/torch_training/training_navigation.py) for more details.
+
+Changes since Flatland 0.2
+--------------------------
+Please list all major changes since the last version:
+
+- Refactoring of rendering code: CamelCase functions changed to snake_case
+- Tree Observation Added a new Featuer: `unusable_switch` which indicates switches that are not branchingpoints for the observing agent
+- Updated the shortest path predictor
+- Updated conflict detection with predictor
+- Episodes length can be set as maximum number of steps allowed.

docs/03_tutorials.rst

+.. include:: tutorials/01_gettingstarted.rst
+.. include:: tutorials/02_observationbuilder.rst
+.. include:: tutorials/03_rail_and_schedule_generator.rst
+.. include:: tutorials/04_stochasticity.rst
+.. include:: tutorials/05_multispeed.rst
+.. include:: tutorials/06_round_2_starter_help.rst

docs/03_tutorials_toc.rst

+Tutorials
+=========
+
+.. toctree::
+   :maxdepth: 2
+
+   03_tutorials

docs/04_specifications.rst

+.. include:: specifications/intro.rst
+.. include:: specifications/core.rst
+.. include:: specifications/railway.rst
+.. include:: specifications/intro_observation_actions.rst
+.. include:: specifications/rendering.rst
+.. include:: specifications/visualization.rst

docs/04_specifications_toc.rst

+Specifications
+==============
+
+
+.. toctree::
+   :maxdepth: 2
+
+   04_specifications
+
+

docs/07_changes.rst

+Changes
+=======
+
+.. toctree::
+   :maxdepth: 2
+
+   07_changes_include.rst

docs/07_changes_include.rst

+.. include:: ../changelog.rst
+.. include:: ../flatland_2.0.rst

docs/authors.rst → docs/08_authors.rst

+Authors
+=======
+
 .. include:: ../AUTHORS.rst

docs/09_faq.rst

+.. include:: ../FAQ_Challenge.rst
+
+.. include:: ../FAQ_Repository.rst
+.. include:: ../FAQ_Bugs.rst

docs/09_faq_toc.rst

+FAQ
+===
+
+.. toctree::
+   :maxdepth: 2
+
+   09_faq

docs/10_interface.rst

+Multi-Agent Interface
+=======
+
+.. include:: interface/pettingzoo.rst
+.. include:: interface/wrappers.rst
+
+Multi-Agent Pettingzoo Usage
+=======
+
+We can use the PettingZoo interface by proving the rail env to the petting zoo wrapper as shown below in the example.
+
+.. literalinclude:: ../tests/test_pettingzoo_interface.py
+   :language: python
+   :start-after: __sphinx_doc_begin__
+   :end-before: __sphinx_doc_end__
+
+
+Multi-Agent Interface Stable Baseline 3 Training
+=======
+
+.. literalinclude:: ../flatland/contrib/training/flatland_pettingzoo_stable_baselines.py
+   :language: python
+   :start-after: __sphinx_doc_begin__
+   :end-before: __sphinx_doc_end__
+
+
+Multi-Agent Interface Rllib Training
+=======
+
+.. literalinclude:: ../flatland/contrib/training/flatland_pettingzoo_rllib.py
+   :language: python
+   :start-after: __sphinx_doc_begin__
+   :end-before: __sphinx_doc_end__
\ No newline at end of file

docs/10_interface_toc.rst

+Multi-Agent Interfaces
+==============
+
+
+.. toctree::
+   :maxdepth: 2
+
+   10_interface

docs/FAQ.rst

-========================================
-Frequently Asked Questions (FAQs)
-========================================
-
--   I get a runtime error with `Click` complaining about the encoding
-
-    .. code-block:: python
-
-        RuntimeError('Click will abort further execution because Python 3 \
-        was configured to use ASCII as encoding for ...sk_SK.UTF-8, \
-        sl_SI.UTF-8, sr_YU.UTF-8, sv_SE.UTF-8, tr_TR.UTF-8, \
-        uk_UA.UTF-8, zh_CN.UTF-8, zh_HK.UTF-8, zh_TW.UTF-8')
-
-    This can be solved by :
-
-    .. code-block:: bash
-
-        export LC_ALL=en_US.utf-8
-        export LANG=en_US.utf-8
-
--   We use `importlib-resources`_ to read from local files.
-    Sample usages:
-
-    .. code-block:: python
-
-        from importlib_resources import path
-
-        with path(package, resource) as file_in:
-            new_grid = np.load(file_in)
-            
-    And:
-
-    .. code-block:: python
-
-        from importlib_resources import read_binary
-        
-        load_data = read_binary(package, resource)
-        self.set_full_state_msg(load_data)
-        
-
-    .. _importlib-resources: https://importlib-resources.readthedocs.io/en/latest/
-
-    Renders the scene into a image (screenshot)
-    .. code-block:: python
-
-    renderer.gl.save_image("filename.bmp")
-

docs/Makefile

-# Minimal makefile for Sphinx documentation
-#
-
-# You can set these variables from the command line.
-# TODO fix sphinx warnings instead of suppressing them...
-SPHINXOPTS    = -Q
-SPHINXBUILD   = python -msphinx
-SPHINXPROJ    = flatland
-SOURCEDIR     = .
-BUILDDIR      = _build
-
-# Put it first so that "make" without argument is like "make help".
-help:
-	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
-
-.PHONY: help Makefile
-
-# Catch-all target: route all unknown targets to Sphinx using the new
-# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
-%: Makefile
-	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/about_flatland.rst

-About Flatland
-==============
-
-.. image:: https://i.imgur.com/rKGEmsk.gif
-  :align: center
-
-
-
-Flatland is a toolkit for developing and comparing multi agent reinforcement learning algorithms on grids.
-The base environment is a two-dimensional grid in which many agents can be placed. Each agent must solve one or more tasks in the grid world.
-In general, agents can freely navigate from cell to cell. However, cell-to-cell navigation can be restricted by transition maps.
-Each cell can hold an own transition map. By default, each cell has a default transition map defined which allows all transitions to its
-eight neighbor cells (go up and left, go up, go up and right, go right, go down and right, go down, go down and left, go left).
-So, the agents can freely move from cell to cell.
-
-The general purpose of the implementation allows to implement any kind of two-dimensional gird based environments.
-It can be used for many learning task where a two-dimensional grid could be the base of the environment.
-
-Flatland delivers a python implementation which can be easily extended. And it provides different baselines for different environments.
-Each environment enables an interesting task to solve. For example, the mutli-agent navigation task for railway train dispatching is a very exciting topic.
-It can be easily extended or adapted to the airplane landing problem. This can further be the basic implementation for many other tasks in transportation and logistics.
-
-Mapping a railway infrastructure into a grid world is an excellent example showing how the movement of an agent must be restricted.
-As trains can normally not run backwards and they have to follow rails the transition for one cell to the other depends also on train's orientation, respectively on train's travel direction.
-Trains can only change the traveling path at switches. There are two variants of switches. The first kind of switch is the splitting "switch", where trains can change rails and in consequence they can change the traveling path.
-The second kind of switch is the fusion switch, where train can change the sequence. That means two rails come together. Thus, the navigation behavior of a train is very restricted.
-The railway planning problem where many agents share same infrastructure is a very complex problem.
-
-Furthermore, trains have a departing location where they cannot depart earlier than the committed departure time.
-Then they must arrive at destination not later than the committed arrival time. This makes the whole planning problem
-very complex. In such a complex environment cooperation is essential. Thus, agents must learn to cooperate in a way that all trains (agents) arrive on time.
-
-This library was developed by `SBB <https://www.sbb.ch/en/>`_ , `AIcrowd <https://www.aicrowd.com/>`_ and numerous contributors and AIcrowd research fellows from the AIcrowd community. 
-
-This library was developed specifically for the `Flatland Challenge <https://www.aicrowd.com/challenges/flatland-challenge>`_ in which we strongly encourage you to take part in. 
-
-
-.. image:: https://i.imgur.com/pucB84T.gif
-  :align: center
-  :width: 600px
-
-.. image:: https://i.imgur.com/xgWGRse.gif
-  :align: center
-  :width: 600px
\ No newline at end of file

docs/conf.py

@@ -33,7 +33,7 @@ sys.path.insert(0, os.path.abspath('..'))
 
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
-extensions = ['sphinx.ext.autodoc', 'sphinx.ext.viewcode']
+extensions = ['sphinx.ext.autodoc', 'sphinx.ext.viewcode', 'sphinx.ext.intersphinx', 'numpydoc']
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
@@ -41,8 +41,12 @@ templates_path = ['_templates']
 # The suffix(es) of source filenames.
 # You can specify multiple suffix as a list of string:
 #
-# source_suffix = ['.rst', '.md']
-source_suffix = '.rst'
+# https://www.sphinx-doc.org/en/master/usage/markdown.html
+source_suffix = {
+    '.rst': 'restructuredtext',
+    '.txt': 'markdown',
+    '.md': 'markdown',
+}
 
 # The master toctree document.
 master_doc = 'index'
@@ -154,3 +158,6 @@ texinfo_documents = [
      'One line description of project.',
      'Miscellaneous'),
 ]
+
+# https://stackoverflow.com/questions/12206334/sphinx-autosummary-toctree-contains-reference-to-nonexisting-document-warnings
+numpydoc_show_class_members = False

docs/flatland.baselines.rst

-flatland.baselines package
-==========================
-
-Submodules
-----------
-
-flatland.baselines.dueling\_double\_dqn module
-----------------------------------------------
-
-.. automodule:: flatland.baselines.dueling_double_dqn
-    :members:
-    :undoc-members:
-    :show-inheritance:
-
-flatland.baselines.model module
--------------------------------
-
-.. automodule:: flatland.baselines.model
-    :members:
-    :undoc-members:
-    :show-inheritance:
-
-
-Module contents
----------------
-
-.. automodule:: flatland.baselines
-    :members:
-    :undoc-members:
-    :show-inheritance: