# Flatland Challenge Starter Kit
[](https://gitter.im/AIcrowd-HQ/flatland-rl)
Instructions to make submissions to the [SBB CFF Flatland Challenge](https://www.aicrowd.com/challenges/flatland-challenge).
Participants will have to submit their code, with packaging specifications, and the evaluator will automatically build a docker image and execute their agent against an arbitrary number of pre-generated flatland environments.
### Dependencies
- **Anaconda** (By following instructions [here](https://www.anaconda.com/download)) At least version `4.5.11` is required to correctly populate `environment.yml`.
- **flatland-rl** (By following instructions [here](http://flatland-rl-docs.s3-website.eu-central-1.amazonaws.com/readme.html))
**IMPORTANT** : Please note that you will need `flatland-rl` version `>=2.1.7` to be able to submit.
### Setup
* Clone the repository
```
git clone git@github.com:AIcrowd/flatland-challenge-starter-kit.git
cd flatland-challenge-starter-kit
```
* Create a conda environment from the provided `environment.yml`
```sh
conda env create -f environment.yml
```
* Activate the conda environment and install your code specific dependencies
```sh
conda activate flatland-rl
# If say you want to install PyTorch
# conda install pytorch torchvision -c pytorch
#
# or you can even use pip to install any additional packages
# for example :
# pip install -U flatland-rl
# which updates the flatland-rl package to the latest version
```
### Test Submission Locally
* First lets begin by downloading a small set of test envs, and put them at a location of your choice.
In this exercise, we assume that you will download the test-envs provided at : [https://www.aicrowd.com/challenges/flatland-challenge/dataset_files](https://www.aicrowd.com/challenges/flatland-challenge/dataset_files), and will untar them inside `./scratch/test-envs`, so that you have a directory structure similar to :
```
./scratch
└── test-envs
├── Test_0
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_1
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_2
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_3
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_4
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_5
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_6
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_7
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_8
│ ├── Level_0.pkl
│ └── Level_1.pkl
└── Test_9
├── Level_0.pkl
└── Level_1.pkl
```
* `redis-server` : **NOTE** : Please ensure that you have a redis-server running on localhost.
You can find more instructions on how to run redis [here](https://redis.io/topics/quickstart)
* **Run evaluator**
```
# In a separate tab : run local grader
flatland-evaluator --tests <path_to_your_tests_directory>
# If you downloaded the files to the location we specified above, then you should be running :
flatland-evaluator --tests ./scratch/test-envs/
```
* **Run Agent(s)**
```
# In a separate tab :
export AICROWD_TESTS_FOLDER=<path_to_your_tests_directory>
# or on Windows :
#
# SET AICROWD_TESTS_FOLDER=<path_to_your_tests_directory>
python run.py
```
# How do I specify my software runtime ?
The software runtime is specified by exporting your `conda` env to the root
of your repository by doing :
```
# The included environment.yml is generated by the command below, and you do not need to run it again
# if you did not add any custom dependencies
conda env export --no-build > environment.yml
# Note the `--no-build` flag, which is important if you want your anaconda env to be replicable across all
```
This `environment.yml` file will be used to recreate the `conda environment` inside the Docker container.
This repository includes an example `environment.yml`
You can specify your software environment by using all the [available configuration options of repo2docker](https://repo2docker.readthedocs.io/en/latest/config_files.html). (But please remember to use [aicrowd-repo2docker](https://pypi.org/project/aicrowd-repo2docker/) to have GPU support)
# What should my code structure be like ?
Please follow the structure documented in the included [run.py](https://github.com/AIcrowd/flatland-challenge-starter-kit/blob/master/run.py) to adapt
your already existing code to the required structure for this round.
## Important Concepts
### Repository Structure
- `aicrowd.json`
Each repository should have a `aicrowd.json` with the following content :
```json
{
"challenge_id": "aicrowd_flatland_challenge_2019",
"grader_id": "aicrowd_flatland_challenge_2019",
"authors": ["your-aicrowd-username"],
"description": "sample description about your awesome agent",
"license": "MIT",
"debug": false
}
```
This is used to map your submission to the said challenge, so please remember to use the correct `challenge_id` and `grader_id` as specified above.
If you set `debug` to `true`, then the evaluation will run on a separate set of 20 environments, and the logs from your submitted code (if it fails), will be made available to you to help you debug.
**NOTE** : **IMPORTANT** : By default we have set `debug:false`, so when you have done the basic integration testing of your code, and are ready to make a final submission, please do make sure to set `debug` to `true` in `aicrowd.json`.
### Code Entrypoint
The evaluator will use `/home/aicrowd/run.sh` as the entrypoint, so please remember to have a `run.sh` at the root, which can instantitate any necessary environment variables, and also start executing your actual code. This repository includes a sample `run.sh` file.
If you are using a Dockerfile to specify your software environment, please remember to create a `aicrowd` user, and place the entrypoint code at `run.sh`.
If you are unsure what this is all about, you can let `run.sh` be as is, and instead focus on the `run.py` which is being called from within `run.sh`.
## Submission
To make a submission, you will have to create a private repository on [https://gitlab.aicrowd.com/](https://gitlab.aicrowd.com/).
You will have to add your SSH Keys to your GitLab account by following the instructions [here](https://docs.gitlab.com/ee/gitlab-basics/create-your-ssh-keys.html).
If you do not have SSH Keys, you will first need to [generate one](https://docs.gitlab.com/ee/ssh/README.html#generating-a-new-ssh-key-pair).
Then you can create a submission by making a _tag push_ to your repository on [https://gitlab.aicrowd.com/](https://gitlab.aicrowd.com/).
**Any tag push (where the tag name begins with "submission-") to your private repository is considered as a submission**
Then you can add the correct git remote, and finally submit by doing :
```
cd flatland-challenge-starter-kit
# Add AIcrowd git remote endpoint
git remote add aicrowd git@gitlab.aicrowd.com/:<YOUR_AICROWD_USER_NAME>/flatland-challenge-starter-kit.git
git push aicrowd master
# Create a tag for your submission and push
git tag -am "submission-v0.1" submission-v0.1
git push aicrowd master
git push aicrowd submission-v0.1
# Note : If the contents of your repository (latest commit hash) does not change,
# then pushing a new tag will **not** trigger a new evaluation.
```
You now should be able to see the details of your submission at :
[gitlab.aicrowd.com//<YOUR_AICROWD_USER_NAME>/flatland-challenge-starter-kit/issues](gitlab.aicrowd.com//<YOUR_AICROWD_USER_NAME>/flatland-challenge-starter-kit/issues)
**NOTE**: Remember to update your username in the link above :wink:
In the link above, you should start seeing something like this take shape (the whole evaluation can take a bit of time, so please be a bit patient too :wink: ) :
**Best of Luck** :tada: :tada:
# Author
**[Sharada Mohanty](https://twitter.com/MeMohanty)**