This repository allows to run Rail Environment multi agent training with the RLLib Library.

It should be clone inside the main flatland repository.

Installation:

pip install ray
pip install gin-config

To start a grid search on some parameters, you can create a folder containing a config.gin file (see example in grid_search_configs/n_agents_grid_search/config.gin.

Then, you can modify the config.gin file path at the end of the grid_search_train.py file.

The results will be stored inside the folder, and the learning curves can be visualized in tensorboard:

tensorboard --logdir=/path/to/foler_containing_config_gin_file

Gin config files

In each config.gin files, all the parameters, except local_dir of the run_experiment functions have to be specified. For example, to indicate the number of agents that have to be initialized at the beginning of each simulation, the following line should be added:

run_experiment.n_agents = 2

If several number of agents have to be explored during the experiment, one can pass the following value to the n_agents parameter:

run_experiment.n_agents = {"grid_search": [2,5]}

which is the way to indicate to the tune library to experiment several values for a parameter.

To reference a class or an object within gin, you should first register it from the train_experiment.py script adding the following line:

gin.external_configurable(TreeObsForRailEnv)

and then a TreeObsForRailEnv object can be referenced in the config.gin file:

run_experiment.obs_builder = {"grid_search": [@TreeObsForRailEnv(), @GlobalObsForRailEnv()]}
TreeObsForRailEnv.max_depth = 2

Note that @TreeObsForRailEnv references the class, while @TreeObsForRailEnv() references instantiates an object of this class.

More documentation on how to use gin-config can be found on the library github repository: https://github.com/google/gin-config