From 68fc310c0c48dabdc9a491ccbf9f069a548b6fe2 Mon Sep 17 00:00:00 2001 From: flaurent <florian.laurent@gmail.com> Date: Thu, 4 Jun 2020 12:25:31 +0200 Subject: [PATCH] Moved documentation to flatland.aicrowd.com --- FAQ.md | 38 -------- README.md | 204 ++---------------------------------------- windows_submission.md | 180 ------------------------------------- 3 files changed, 9 insertions(+), 413 deletions(-) delete mode 100644 FAQ.md delete mode 100644 windows_submission.md diff --git a/FAQ.md b/FAQ.md deleted file mode 100644 index 98de4a2..0000000 --- a/FAQ.md +++ /dev/null @@ -1,38 +0,0 @@ -# Frequently Asked Questions - -A non-exhaustive list of Frequently Asked Questions for the [Flatland Challenge](https://www.aicrowd.com/challenges/flatland-challenge). - -### How do I locally build a docker image out of my submission ? - -* Install Dependencies -- **docker** : By following the instructions [here](https://docs.docker.com/install/linux/docker-ce/ubuntu/) -- **aicrowd-repo2docker** - -```sh -pip install aicrowd-repo2docker -``` - -* Build Image -``` -sudo aicrowd-repo2docker --no-run \ - --user-id 1001 \ - --user-name aicrowd \ - --image-name my-random-agent \ - --debug . -``` - -### Debugging the packaged software environment - -If you have issues with your submission because of your software environment and dependencies, you can debug them, by first building the docker image, and then getting a shell inside the image by : - -``` -# After ensuring that you have build the docker image by following the -# instructions here : https://github.com/AIcrowd/flatland-challenge-starter-kit/blob/master/FAQ.md#how-do-i-locally-build-a-docker-image-out-of-my-submission- - -docker run --net=host -it my-random-agent /bin/bash -``` - -and then exploring to find the cause of the issue. - -# Author -Sharada Mohanty <https://twitter.com/MeMohanty> \ No newline at end of file diff --git a/README.md b/README.md index 097434e..7c69cd9 100644 --- a/README.md +++ b/README.md @@ -2,205 +2,19 @@ # Flatland Challenge Starter Kit -[](https://gitter.im/AIcrowd-HQ/flatland-rl) +**[Follow the instructions in the documentation to submit your solutions!](http://flatland.aicrowd.com/first-submission.html)** - -Instructions to make submissions to the [SBB CFF Flatland Challenge](https://www.aicrowd.com/challenges/flatland-challenge). + -Participants will have to submit their code, with packaging specifications, and the evaluator will automatically build a docker image and execute their agent against an arbitrary number of pre-generated flatland environments. +Communication +--- -### Dependencies +* [Discord Channel](https://discord.com/invite/hCR3CZG) +* [Discussion Forum](https://discourse.aicrowd.com/c/neurips-2020-flatland-challenge) +* [Issue Tracker](https://gitlab.aicrowd.com/flatland/flatland/issues/) -- **Anaconda** (By following instructions [here](https://www.anaconda.com/download)) At least version `4.5.11` is required to correctly populate `environment.yml`. -- **flatland-rl** (By following instructions [here](http://flatland-rl-docs.s3-website.eu-central-1.amazonaws.com/readme.html)) - **IMPORTANT** : Please note that you will need `flatland-rl` version `>=2.1.7` to be able to submit. - - -### Setup - -* Clone the repository - -``` -git clone git@github.com:AIcrowd/flatland-challenge-starter-kit.git -cd flatland-challenge-starter-kit -``` - -* Create a conda environment from the provided `environment.yml` - -```sh -conda env create -f environment.yml -``` - -* Activate the conda environment and install your code specific dependencies - -```sh -conda activate flatland-rl -# If say you want to install PyTorch -# conda install pytorch torchvision -c pytorch -# -# or you can even use pip to install any additional packages -# for example : -# pip install -U flatland-rl -# which updates the flatland-rl package to the latest version -``` - - -### Test Submission Locally - -* First lets begin by downloading a small set of test envs, and put them at a location of your choice. -In this exercise, we assume that you will download the test-envs provided at : [https://www.aicrowd.com/challenges/flatland-challenge/dataset_files](https://www.aicrowd.com/challenges/flatland-challenge/dataset_files), and will untar them inside `./scratch/test-envs`, so that you have a directory structure similar to : - -``` -./scratch -└── test-envs - ├── Test_0 - │  ├── Level_0.pkl - │  └── Level_1.pkl - ├── Test_1 - │  ├── Level_0.pkl - │  └── Level_1.pkl - ├── Test_2 - │  ├── Level_0.pkl - │  └── Level_1.pkl - ├── Test_3 - │  ├── Level_0.pkl - │  └── Level_1.pkl - ├── Test_4 - │  ├── Level_0.pkl - │  └── Level_1.pkl - ├── Test_5 - │  ├── Level_0.pkl - │  └── Level_1.pkl - ├── Test_6 - │  ├── Level_0.pkl - │  └── Level_1.pkl - ├── Test_7 - │  ├── Level_0.pkl - │  └── Level_1.pkl - ├── Test_8 - │  ├── Level_0.pkl - │  └── Level_1.pkl - └── Test_9 - ├── Level_0.pkl - └── Level_1.pkl -``` - -* `redis-server` : **NOTE** : Please ensure that you have a redis-server running on localhost. -You can find more instructions on how to run redis [here](https://redis.io/topics/quickstart) - -* **Run evaluator** - -``` - -# In a separate tab : run local grader -flatland-evaluator --tests <path_to_your_tests_directory> - -# If you downloaded the files to the location we specified above, then you should be running : -flatland-evaluator --tests ./scratch/test-envs/ -``` - -* **Run Agent(s)** - -``` -# In a separate tab : -export AICROWD_TESTS_FOLDER=<path_to_your_tests_directory> -# or on Windows : -# -# SET AICROWD_TESTS_FOLDER=<path_to_your_tests_directory> -python run.py -``` - -# How do I specify my software runtime ? - -The software runtime is specified by exporting your `conda` env to the root -of your repository by doing : - -``` -# The included environment.yml is generated by the command below, and you do not need to run it again -# if you did not add any custom dependencies - -conda env export --no-build > environment.yml - -# Note the `--no-build` flag, which is important if you want your anaconda env to be replicable across all -``` - -This `environment.yml` file will be used to recreate the `conda environment` inside the Docker container. -This repository includes an example `environment.yml` - -You can specify your software environment by using all the [available configuration options of repo2docker](https://repo2docker.readthedocs.io/en/latest/config_files.html). (But please remember to use [aicrowd-repo2docker](https://pypi.org/project/aicrowd-repo2docker/) to have GPU support) - - -# What should my code structure be like ? - -Please follow the structure documented in the included [run.py](https://github.com/AIcrowd/flatland-challenge-starter-kit/blob/master/run.py) to adapt -your already existing code to the required structure for this round. - -## Important Concepts - -### Repository Structure - -- `aicrowd.json` - Each repository should have a `aicrowd.json` with the following content : - -```json -{ - "challenge_id": "aicrowd_flatland_challenge_2019", - "grader_id": "aicrowd_flatland_challenge_2019", - "authors": ["your-aicrowd-username"], - "description": "sample description about your awesome agent", - "license": "MIT", - "debug": false -} -``` - -This is used to map your submission to the said challenge, so please remember to use the correct `challenge_id` and `grader_id` as specified above. - -If you set `debug` to `true`, then the evaluation will run on a separate set of 20 environments, and the logs from your submitted code (if it fails), will be made available to you to help you debug. -**NOTE** : **IMPORTANT** : By default we have set `debug:false`, so when you have done the basic integration testing of your code, and are ready to make a final submission, please do make sure to set `debug` to `true` in `aicrowd.json`. - -### Code Entrypoint - -The evaluator will use `/home/aicrowd/run.sh` as the entrypoint, so please remember to have a `run.sh` at the root, which can instantitate any necessary environment variables, and also start executing your actual code. This repository includes a sample `run.sh` file. -If you are using a Dockerfile to specify your software environment, please remember to create a `aicrowd` user, and place the entrypoint code at `run.sh`. -If you are unsure what this is all about, you can let `run.sh` be as is, and instead focus on the `run.py` which is being called from within `run.sh`. - -## Submission - -To make a submission, you will have to create a private repository on [https://gitlab.aicrowd.com/](https://gitlab.aicrowd.com/). - -You will have to add your SSH Keys to your GitLab account by following the instructions [here](https://docs.gitlab.com/ee/gitlab-basics/create-your-ssh-keys.html). -If you do not have SSH Keys, you will first need to [generate one](https://docs.gitlab.com/ee/ssh/README.html#generating-a-new-ssh-key-pair). - -Then you can create a submission by making a _tag push_ to your repository on [https://gitlab.aicrowd.com/](https://gitlab.aicrowd.com/). -**Any tag push (where the tag name begins with "submission-") to your private repository is considered as a submission** -Then you can add the correct git remote, and finally submit by doing : - -``` -cd flatland-challenge-starter-kit -# Add AIcrowd git remote endpoint -git remote add aicrowd git@gitlab.aicrowd.com:<YOUR_AICROWD_USER_NAME>/flatland-challenge-starter-kit.git -git push aicrowd master - -# Create a tag for your submission and push -git tag -am "submission-v0.1" submission-v0.1 -git push aicrowd master -git push aicrowd submission-v0.1 - -# Note : If the contents of your repository (latest commit hash) does not change, -# then pushing a new tag will **not** trigger a new evaluation. -``` - -You now should be able to see the details of your submission at : -[gitlab.aicrowd.com//<YOUR_AICROWD_USER_NAME>/flatland-challenge-starter-kit/issues](gitlab.aicrowd.com//<YOUR_AICROWD_USER_NAME>/flatland-challenge-starter-kit/issues) - -**NOTE**: Remember to update your username in the link above :wink: - -In the link above, you should start seeing something like this take shape (the whole evaluation can take a bit of time, so please be a bit patient too :wink: ) : - - -**Best of Luck** :tada: :tada: - -# Author +Author +--- **[Sharada Mohanty](https://twitter.com/MeMohanty)** diff --git a/windows_submission.md b/windows_submission.md deleted file mode 100644 index e759741..0000000 --- a/windows_submission.md +++ /dev/null @@ -1,180 +0,0 @@ - - -# Flatland submission from Windows -If you are working on windows locally there is a few steps you will need to do in order to make your flatland challenge submission work. Here is a step-by-step instruction on how to achieve this. - -## Installing WSL - -1. Enable [WSL](https://docs.microsoft.com/en-us/windows/wsl/install-win10) on Windows. -2. Get [Ubuntu](https://www.microsoft.com/en-us/p/ubuntu/9nblggh4msv6?SilentAuth=1&wa=wsignin1.0&activetab=pivot:overviewtab) for Windows. -3. Run your **Ubuntu** system on your computer -4. Now let us install the **Dependencies**. From within the **Ubuntu-Shell** run: - -### Dependencies - -First download **Anaconda** by running this in the **Ubuntu Shell** if you are on a 64bit machine, otherwise update link.( **ATTENTION:** You need the **Linux** version and not Windows version!!!) - -``` -wget https://repo.anaconda.com/archive/Anaconda3-2019.07-Linux-x86_64.sh -``` - -Now we can install **Anaconda** for Ubuntu - -``` -chmod +x Anaconda3-2019.07-Linux-x86_64.sh - ./Anaconda3-2019.07-Linux-x86_64.sh -``` - -For all changes to take affect usually you will have to restart Ubuntu, just close the terminal and run Ubuntu again. - -### Setup -Setup you `ssh` keys for your gitlab account by running. You can install in the default directory and use key without password if you like. - -```sh -ssh-keygen -``` - -Now you need to copy your key to your gitlab account. Find the key - -```sh -cd .ssh -cat id_rsa.pub -``` -Copy all the output and add it to your [gitlab keys](https://gitlab.aicrowd.com/profile/keys) - -Clone your own repository remeber to use ssh cloning and change to the directory where you would like to clone it to! Enter `yes` when promted. If you don't have an own repository yet, start by cloning the official starter kit - -``` -git clone git@gitlab.aicrowd.com:<YOUR_USER_NAME>/flatland-challenge-starter-kit.git -``` - -or official starter kit - -``` -git clone git@github.com:AIcrowd/flatland-challenge-starter-kit.git -cd flatland-challenge-starter-kit -``` - -Create a conda environment from the provided `environment.yml` - -```sh -conda env create -f environment.yml -``` - -This might take a little bit of time to finish. - -Activate the conda environment and install your code specific dependencies - -```sh -conda activate flatland-rl -# If say you want to install PyTorch -# conda install pytorch torchvision -c pytorch -# -# or you can even use pip to install any additional packages -# for example : -# pip install -U flatland-rl -# which updates the flatland-rl package to the latest version -``` - -# How do I specify my software runtime ? - -The software runtime is specified by exporting your `conda` env to the root -of your repository by doing : - -``` -# The included environment.yml is generated by the command below, and you do not need to run it again -# if you did not add any custom dependencies - -conda env export --no-build > environment.yml - -# Note the `--no-build` flag, which is important if you want your anaconda env to be replicable across all -``` - -This `environment.yml` file will be used to recreate the `conda environment` inside the Docker container. -This repository includes an example `environment.yml` - -You can specify your software environment by using all the [available configuration options of repo2docker](https://repo2docker.readthedocs.io/en/latest/config_files.html). (But please remember to use [aicrowd-repo2docker](https://pypi.org/project/aicrowd-repo2docker/) to have GPU support) - - -# What should my code structure be like ? - -Please follow the structure documented in the included [run.py](https://github.com/AIcrowd/flatland-challenge-starter-kit/blob/master/run.py) to adapt -your already existing code to the required structure for this round. - -## Important Concepts - -### Repository Structure - -- `aicrowd.json` - Each repository should have a `aicrowd.json` with the following content : - -```json -{ - "challenge_id": "aicrowd_flatland_challenge_2019", - "grader_id": "aicrowd_flatland_challenge_2019", - "authors": ["your-aicrowd-username"], - "description": "sample description about your awesome agent", - "license": "MIT", - "debug": false -} -``` - -If you are not familiar with working in the shell. Use - -```sh -nano aicrowd.json -``` - -to edit the JSON file. - -This is used to map your submission to the said challenge, so please remember to use the correct `challenge_id` and `grader_id` as specified above. - -If you set `debug` to `true`, then the evaluation will run on a separate set of 20 environments, and the logs from your submitted code (if it fails), will be made available to you to help you debug. -**NOTE** : **IMPORTANT** : By default we have set `debug:false`, so when you have done the basic integration testing of your code, and are ready to make a final submission, please do make sure to set `debug` to `true` in `aicrowd.json`. - -### Code Entrypoint - -The evaluator will use `/home/aicrowd/run.sh` as the entrypoint, so please remember to have a `run.sh` at the root, which can instantitate any necessary environment variables, and also start executing your actual code. This repository includes a sample `run.sh` file. -If you are using a Dockerfile to specify your software environment, please remember to create a `aicrowd` user, and place the entrypoint code at `run.sh`. -If you are unsure what this is all about, you can let `run.sh` be as is, and instead focus on the `run.py` which is being called from within `run.sh`. - -## Submission - -To make a submission, you will have to create a private repository on [https://gitlab.aicrowd.com/](https://gitlab.aicrowd.com/). - -You will have to add your SSH Keys to your GitLab account if you haven't already by following the instructions [here](https://docs.gitlab.com/ee/gitlab-basics/create-your-ssh-keys.html). -If you do not have SSH Keys, you will first need to [generate one](https://docs.gitlab.com/ee/ssh/README.html#generating-a-new-ssh-key-pair). - -Then you can create a submission by making a _tag push_ to your repository on [https://gitlab.aicrowd.com/](https://gitlab.aicrowd.com/). -**Any tag push (where the tag name begins with "submission-") to your private repository is considered as a submission** . Be sure that all changes where added to the commit before pushing. -Then you can add the correct git remote, and finally submit by doing : - -``` -cd flatland-challenge-starter-kit -# Add AIcrowd git remote endpoint -git remote add aicrowd git@gitlab.aicrowd.com:<YOUR_AICROWD_USER_NAME>/flatland-challenge-starter-kit.git -git push aicrowd master - -# Create a tag for your submission and push -git tag -am "submission-v0.1" submission-v0.1 -git push aicrowd master -git push aicrowd submission-v0.1 - -# Note : If the contents of your repository (latest commit hash) does not change, -# then pushing a new tag will **not** trigger a new evaluation. -``` - -You now should be able to see the details of your submission at : -[gitlab.aicrowd.com//<YOUR_AICROWD_USER_NAME>/flatland-challenge-starter-kit/issues](gitlab.aicrowd.com//<YOUR_AICROWD_USER_NAME>/flatland-challenge-starter-kit/issues) - -**NOTE**: Remember to update your username in the link above :wink: - -In the link above, you should start seeing something like this take shape (the whole evaluation can take a bit of time, so please be a bit patient too :wink: ) : - - -**Best of Luck** :tada: :tada: - -# Author - -Sharada Mohanty <https://twitter.com/MeMohanty> -Erik Nygren -- GitLab