Merge branch 'cleanup' into global-density-obs

README.md

-# NeurIPS 2020 Flatland Challenge baselines
+# 🚂 Flatland Baselines
 
-📈 [**Results**](https://app.wandb.ai/masterscrat/flatland/reports/Flatland-Baselines--Vmlldzo4OTc5NA) 
+This repository contains reinforcement learning baselines for the [NeurIPS 2020 Flatland Challenge](https://www.aicrowd.com/challenges/neurips-2020-flatland-challenge/).
 
-Experiments
----
+## Provided baselines
+
+```{note}
+**Note:** looking for something simpler to start? We also provide a simpler Dueling Double DQN method implemented using PyTorch without relying on RLlib: **https://gitlab.aicrowd.com/flatland/flatland-examples** 
+```
+
+### RL Methods
+
+- Ape-X
+- PPO
+- CCPPO
+- Pure Imitation Learning: MARWIL
+- Mixed IL/RL: DQfD 
+
+### Custom observations
+
+- Density observations
+- Local conflict observations
+
+### Tricks
+
+- Action skipping
+- Action masking
+
+
+
+
+## Organisation
 
 Experiments consist of one or many RLlib YAML config files alongside a MARKDOWN file containing results, plots and a detailed description of the methodology.
 
@@ -12,31 +38,20 @@ All files are stored in a experiment folder under `experiments/<env-name>/<exper
 - [Tree observations w/ fully connected network](experiments/flatland_random_sparse_small/tree_obs_fc_net)
 - [Global observations w/ convnet](experiments/flatland_random_sparse_small/global_obs_conv_net)
 
-Setup
----
+## Setup
 
-Using conda (recommended):
+The setup uses conda, [install it](https://www.anaconda.com/products/individual) if necessary.
 
 ```
 # with GPU support:
 conda env create -f environment-gpu.yml
+conda activate flatland-baseline-gpu-env
 
 # or, without GPU support:
-#conda env create -f environment-cpu.yml
-
-conda activate flatland-env
-pip install -r requirements.txt
+conda env create -f environment-cpu.yml
+conda activate flatland-baseline-cpu-env
 ```
 
-Using pip:
-
-```
-# no GPU support:
-pip install -r requirements.txt
-```
-
-You may need to install/update bazel: [Ubuntu guide](https://docs.bazel.build/versions/master/install-ubuntu.html)
-
 ## Usage
 
 Training example:
@@ -48,9 +63,8 @@ Evaluation example:
 `python ./rollout.py /tmp/ray/checkpoint_dir/checkpoint-0 --run PPO --no-render
         --config '{"env_config": {"test": true}}' --episodes 1000 --out rollouts.pkl`
 
-Note that -f overrides all other trial-specific command-line options.
+Note that `-f` overrides all other trial-specific command-line options.
 
-Notes
----
+## Notes
 
-- The basic structure of this repository is adopted from [https://github.com/spMohanty/rl-experiments/](https://github.com/spMohanty/rl-experiments/)
\ No newline at end of file
+- The basic structure of this repository is adapted from [https://github.com/spMohanty/rl-experiments/](https://github.com/spMohanty/rl-experiments/)
\ No newline at end of file

environment-cpu.yml

-name: flatland-env
+name: flatland-baseline-cpu-env
 dependencies:
     - python=3.7
+    - pip=20.0
     - pip:
-        - flatland-rl==2.1.10
+        - flatland-rl==2.2.1
         - CairoSVG==2.4.2
-#       - pycairo==1.19.1 # fails!
         - pyhumps==1.3.1
         - gputil==1.4.0
         - pyhumps==1.3.1
-        - wandb==0.8.36
+        - wandb==0.9.2
         - ray[rllib]==0.8.5
         - tensorflow==2.1.0
\ No newline at end of file

environment-edge-cpu.yml

-name: flatland-env
-dependencies:
-    - python=3.7
-    - pip=20.0
-    - pip:
-        - git+https://gitlab.aicrowd.com/flatland/flatland.git@master
-        - CairoSVG==2.4.2
-#       - pycairo==1.19.1 # fails!
-        - pyhumps==1.3.1
-        - gputil==1.4.0
-        - pyhumps==1.3.1
-        - wandb==0.8.36
-        - ray[rllib]==0.8.5
-        - tensorflow==2.1.0
\ No newline at end of file

environment-edge-gpu.yml

-name: flatland-env
-dependencies:
-    - python=3.7
-    - tensorflow-gpu=2.1.0
-    - pip=20.0
-    - pip:
-        - git+https://gitlab.aicrowd.com/flatland/flatland.git@master
-        - CairoSVG==2.4.2
-#       - pycairo==1.19.1 # fails!
-        - pyhumps==1.3.1
-        - gputil==1.4.0
-        - pyhumps==1.3.1
-        - wandb==0.8.36
-        - ray[rllib]==0.8.5
\ No newline at end of file

environment-gpu.yml

-name: flatland-env
+name: flatland-baseline-gpu-env
 dependencies:
     - python=3.7
     - tensorflow-gpu=2.1.0
     - pip=20.0
     - pip:
-        - flatland-rl==2.1.10
+        - flatland-rl==2.2.1
         - CairoSVG==2.4.2
-#       - pycairo==1.19.1 # fails!
         - pyhumps==1.3.1
         - gputil==1.4.0
         - pyhumps==1.3.1
-        - wandb==0.8.36
+        - wandb==0.9.2
         - ray[rllib]==0.8.5
\ No newline at end of file

gen-envs-actions.sh

-#!/bin/bash
-
-if [ $# != 4 ]; then
-	echo "Need 4 args: sName, nStart, nEnd. nSize"
-	echo "eg: gen-envs-actions.sh small-100-120 100 120 0"
-	echo "Creates envs under ./envs/small-100-120/envs with sequence 100..119, nSizebias=0"
-	exit 1
-fi
-
-sName=$1
-nStart=$2
-nEnd=$3
-nSize=$4
-
-dirRoot=./saved-envs/$sName/
-
-echo "Settings - name $sName start $nStart end $nEnv size $nSize dirRoot $dirRoot"
-
-mkdir -p $dirRoot
-echo -e "params: $*\nsName=$sName\nnStart=$nStart\nnEnd=$nEnd\nnSize=$nSize\n" > $dirRoot/params
-
-export RLBASEDIR=$HOME/projects/aicrowd/rl-trains/
-export AICROWD_TESTS_FOLDER=$dirRoot
-
-echo "================ remove flatland keys from redis ================="
-redis-cli keys "flatland*" 
-redis-cli keys "flatland*" | xargs redis-cli del
-
-echo "=============== Generate Envs ===================="
-python -m flatland_contrib.cli.gen_envs --createEnvs $nStart $nEnd  -d $dirRoot/envs -i $dirRoot/images  -s 3
- 
-
-echo "=============== kill any existing pythons on this terminal ===================="
-ps -f | grep python | egrep -v jupyter | awk '{print $2}' | xargs kill
-
-echo "=============== Start service ===================="
-python evaluator2/service.py --test_folder $dirRoot --mergeDir $dirRoot/merge --actionDir $dirRoot/actions --pickle --noShuffle --missingOnly &
-
-sleep 3
-
-echo "=============== Starting client to create actions ===================="
-python $RLBASEDIR/mugurel/run.py 
-
-
-
-
-
-