Skip to content
Snippets Groups Projects
Commit 68fc310c authored by MasterScrat's avatar MasterScrat
Browse files

Moved documentation to flatland.aicrowd.com

parent 3806e5a2
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
FAQ.md deleted 100644 → 0
+ 0
− 38
View file @ 3806e5a2
# Frequently Asked Questions
A non-exhaustive list of Frequently Asked Questions for the [Flatland Challenge](https://www.aicrowd.com/challenges/flatland-challenge).
### How do I locally build a docker image out of my submission ?
* Install Dependencies
- **docker** : By following the instructions [here](https://docs.docker.com/install/linux/docker-ce/ubuntu/)
- **aicrowd-repo2docker**
```sh
pip install aicrowd-repo2docker
```
* Build Image
```
sudo aicrowd-repo2docker --no-run \
--user-id 1001 \
--user-name aicrowd \
--image-name my-random-agent \
--debug .
```
### Debugging the packaged software environment
If you have issues with your submission because of your software environment and dependencies, you can debug them, by first building the docker image, and then getting a shell inside the image by :
```
# After ensuring that you have build the docker image by following the
# instructions here : https://github.com/AIcrowd/flatland-challenge-starter-kit/blob/master/FAQ.md#how-do-i-locally-build-a-docker-image-out-of-my-submission-
docker run --net=host -it my-random-agent /bin/bash
```
and then exploring to find the cause of the issue.
# Author
Sharada Mohanty <https://twitter.com/MeMohanty>
\ No newline at end of file
README.md
+ 9
− 195
View file @ 68fc310c
...@@ -2,205 +2,19 @@ ...@@ -2,205 +2,19 @@
# Flatland Challenge Starter Kit # Flatland Challenge Starter Kit
[![gitter-badge](https://badges.gitter.im/AIcrowd-HQ/flatland-rl.png)](https://gitter.im/AIcrowd-HQ/flatland-rl) **[Follow the instructions in the documentation to submit your solutions!](http://flatland.aicrowd.com/first-submission.html)**
![](https://i.imgur.com/9cNtWjs.gif)
Instructions to make submissions to the [SBB CFF Flatland Challenge](https://www.aicrowd.com/challenges/flatland-challenge). ![flatland](https://i.imgur.com/9cNtWjs.gif)
Participants will have to submit their code, with packaging specifications, and the evaluator will automatically build a docker image and execute their agent against an arbitrary number of pre-generated flatland environments. Communication
---
### Dependencies * [Discord Channel](https://discord.com/invite/hCR3CZG)
* [Discussion Forum](https://discourse.aicrowd.com/c/neurips-2020-flatland-challenge)
* [Issue Tracker](https://gitlab.aicrowd.com/flatland/flatland/issues/)
- **Anaconda** (By following instructions [here](https://www.anaconda.com/download)) At least version `4.5.11` is required to correctly populate `environment.yml`. Author
- **flatland-rl** (By following instructions [here](http://flatland-rl-docs.s3-website.eu-central-1.amazonaws.com/readme.html)) ---
**IMPORTANT** : Please note that you will need `flatland-rl` version `>=2.1.7` to be able to submit.
### Setup
* Clone the repository
```
git clone git@github.com:AIcrowd/flatland-challenge-starter-kit.git
cd flatland-challenge-starter-kit
```
* Create a conda environment from the provided `environment.yml`
```sh
conda env create -f environment.yml
```
* Activate the conda environment and install your code specific dependencies
```sh
conda activate flatland-rl
# If say you want to install PyTorch
# conda install pytorch torchvision -c pytorch
#
# or you can even use pip to install any additional packages
# for example :
# pip install -U flatland-rl
# which updates the flatland-rl package to the latest version
```
### Test Submission Locally
* First lets begin by downloading a small set of test envs, and put them at a location of your choice.
In this exercise, we assume that you will download the test-envs provided at : [https://www.aicrowd.com/challenges/flatland-challenge/dataset_files](https://www.aicrowd.com/challenges/flatland-challenge/dataset_files), and will untar them inside `./scratch/test-envs`, so that you have a directory structure similar to :
```
./scratch
└── test-envs
├── Test_0
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_1
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_2
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_3
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_4
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_5
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_6
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_7
│ ├── Level_0.pkl
│ └── Level_1.pkl
├── Test_8
│ ├── Level_0.pkl
│ └── Level_1.pkl
└── Test_9
├── Level_0.pkl
└── Level_1.pkl
```
* `redis-server` : **NOTE** : Please ensure that you have a redis-server running on localhost.
You can find more instructions on how to run redis [here](https://redis.io/topics/quickstart)
* **Run evaluator**
```
# In a separate tab : run local grader
flatland-evaluator --tests <path_to_your_tests_directory>
# If you downloaded the files to the location we specified above, then you should be running :
flatland-evaluator --tests ./scratch/test-envs/
```
* **Run Agent(s)**
```
# In a separate tab :
export AICROWD_TESTS_FOLDER=<path_to_your_tests_directory>
# or on Windows :
#
# SET AICROWD_TESTS_FOLDER=<path_to_your_tests_directory>
python run.py
```
# How do I specify my software runtime ?
The software runtime is specified by exporting your `conda` env to the root
of your repository by doing :
```
# The included environment.yml is generated by the command below, and you do not need to run it again
# if you did not add any custom dependencies
conda env export --no-build > environment.yml
# Note the `--no-build` flag, which is important if you want your anaconda env to be replicable across all
```
This `environment.yml` file will be used to recreate the `conda environment` inside the Docker container.
This repository includes an example `environment.yml`
You can specify your software environment by using all the [available configuration options of repo2docker](https://repo2docker.readthedocs.io/en/latest/config_files.html). (But please remember to use [aicrowd-repo2docker](https://pypi.org/project/aicrowd-repo2docker/) to have GPU support)
# What should my code structure be like ?
Please follow the structure documented in the included [run.py](https://github.com/AIcrowd/flatland-challenge-starter-kit/blob/master/run.py) to adapt
your already existing code to the required structure for this round.
## Important Concepts
### Repository Structure
- `aicrowd.json`
Each repository should have a `aicrowd.json` with the following content :
```json
{
"challenge_id": "aicrowd_flatland_challenge_2019",
"grader_id": "aicrowd_flatland_challenge_2019",
"authors": ["your-aicrowd-username"],
"description": "sample description about your awesome agent",
"license": "MIT",
"debug": false
}
```
This is used to map your submission to the said challenge, so please remember to use the correct `challenge_id` and `grader_id` as specified above.
If you set `debug` to `true`, then the evaluation will run on a separate set of 20 environments, and the logs from your submitted code (if it fails), will be made available to you to help you debug.
**NOTE** : **IMPORTANT** : By default we have set `debug:false`, so when you have done the basic integration testing of your code, and are ready to make a final submission, please do make sure to set `debug` to `true` in `aicrowd.json`.
### Code Entrypoint
The evaluator will use `/home/aicrowd/run.sh` as the entrypoint, so please remember to have a `run.sh` at the root, which can instantitate any necessary environment variables, and also start executing your actual code. This repository includes a sample `run.sh` file.
If you are using a Dockerfile to specify your software environment, please remember to create a `aicrowd` user, and place the entrypoint code at `run.sh`.
If you are unsure what this is all about, you can let `run.sh` be as is, and instead focus on the `run.py` which is being called from within `run.sh`.
## Submission
To make a submission, you will have to create a private repository on [https://gitlab.aicrowd.com/](https://gitlab.aicrowd.com/).
You will have to add your SSH Keys to your GitLab account by following the instructions [here](https://docs.gitlab.com/ee/gitlab-basics/create-your-ssh-keys.html).
If you do not have SSH Keys, you will first need to [generate one](https://docs.gitlab.com/ee/ssh/README.html#generating-a-new-ssh-key-pair).
Then you can create a submission by making a _tag push_ to your repository on [https://gitlab.aicrowd.com/](https://gitlab.aicrowd.com/).
**Any tag push (where the tag name begins with "submission-") to your private repository is considered as a submission**
Then you can add the correct git remote, and finally submit by doing :
```
cd flatland-challenge-starter-kit
# Add AIcrowd git remote endpoint
git remote add aicrowd git@gitlab.aicrowd.com:<YOUR_AICROWD_USER_NAME>/flatland-challenge-starter-kit.git
git push aicrowd master
# Create a tag for your submission and push
git tag -am "submission-v0.1" submission-v0.1
git push aicrowd master
git push aicrowd submission-v0.1
# Note : If the contents of your repository (latest commit hash) does not change,
# then pushing a new tag will **not** trigger a new evaluation.
```
You now should be able to see the details of your submission at :
[gitlab.aicrowd.com//<YOUR_AICROWD_USER_NAME>/flatland-challenge-starter-kit/issues](gitlab.aicrowd.com//<YOUR_AICROWD_USER_NAME>/flatland-challenge-starter-kit/issues)
**NOTE**: Remember to update your username in the link above :wink:
In the link above, you should start seeing something like this take shape (the whole evaluation can take a bit of time, so please be a bit patient too :wink: ) :
![](https://i.imgur.com/4HWf1jU.png)
**Best of Luck** :tada: :tada:
# Author
**[Sharada Mohanty](https://twitter.com/MeMohanty)** **[Sharada Mohanty](https://twitter.com/MeMohanty)**
windows_submission.md deleted 100644 → 0
+ 0
− 180
View file @ 3806e5a2
![AIcrowd-Logo](https://raw.githubusercontent.com/AIcrowd/AIcrowd/master/app/assets/images/misc/aicrowd-horizontal.png)
# Flatland submission from Windows
If you are working on windows locally there is a few steps you will need to do in order to make your flatland challenge submission work. Here is a step-by-step instruction on how to achieve this.
## Installing WSL
1. Enable [WSL](https://docs.microsoft.com/en-us/windows/wsl/install-win10) on Windows.
2. Get [Ubuntu](https://www.microsoft.com/en-us/p/ubuntu/9nblggh4msv6?SilentAuth=1&wa=wsignin1.0&activetab=pivot:overviewtab) for Windows.
3. Run your **Ubuntu** system on your computer
4. Now let us install the **Dependencies**. From within the **Ubuntu-Shell** run:
### Dependencies
First download **Anaconda** by running this in the **Ubuntu Shell** if you are on a 64bit machine, otherwise update link.( **ATTENTION:** You need the **Linux** version and not Windows version!!!)
```
wget https://repo.anaconda.com/archive/Anaconda3-2019.07-Linux-x86_64.sh
```
Now we can install **Anaconda** for Ubuntu
```
chmod +x Anaconda3-2019.07-Linux-x86_64.sh
./Anaconda3-2019.07-Linux-x86_64.sh
```
For all changes to take affect usually you will have to restart Ubuntu, just close the terminal and run Ubuntu again.
### Setup
Setup you `ssh` keys for your gitlab account by running. You can install in the default directory and use key without password if you like.
```sh
ssh-keygen
```
Now you need to copy your key to your gitlab account. Find the key
```sh
cd .ssh
cat id_rsa.pub
```
Copy all the output and add it to your [gitlab keys](https://gitlab.aicrowd.com/profile/keys)
Clone your own repository remeber to use ssh cloning and change to the directory where you would like to clone it to! Enter `yes` when promted. If you don't have an own repository yet, start by cloning the official starter kit
```
git clone git@gitlab.aicrowd.com:<YOUR_USER_NAME>/flatland-challenge-starter-kit.git
```
or official starter kit
```
git clone git@github.com:AIcrowd/flatland-challenge-starter-kit.git
cd flatland-challenge-starter-kit
```
Create a conda environment from the provided `environment.yml`
```sh
conda env create -f environment.yml
```
This might take a little bit of time to finish.
Activate the conda environment and install your code specific dependencies
```sh
conda activate flatland-rl
# If say you want to install PyTorch
# conda install pytorch torchvision -c pytorch
#
# or you can even use pip to install any additional packages
# for example :
# pip install -U flatland-rl
# which updates the flatland-rl package to the latest version
```
# How do I specify my software runtime ?
The software runtime is specified by exporting your `conda` env to the root
of your repository by doing :
```
# The included environment.yml is generated by the command below, and you do not need to run it again
# if you did not add any custom dependencies
conda env export --no-build > environment.yml
# Note the `--no-build` flag, which is important if you want your anaconda env to be replicable across all
```
This `environment.yml` file will be used to recreate the `conda environment` inside the Docker container.
This repository includes an example `environment.yml`
You can specify your software environment by using all the [available configuration options of repo2docker](https://repo2docker.readthedocs.io/en/latest/config_files.html). (But please remember to use [aicrowd-repo2docker](https://pypi.org/project/aicrowd-repo2docker/) to have GPU support)
# What should my code structure be like ?
Please follow the structure documented in the included [run.py](https://github.com/AIcrowd/flatland-challenge-starter-kit/blob/master/run.py) to adapt
your already existing code to the required structure for this round.
## Important Concepts
### Repository Structure
- `aicrowd.json`
Each repository should have a `aicrowd.json` with the following content :
```json
{
"challenge_id": "aicrowd_flatland_challenge_2019",
"grader_id": "aicrowd_flatland_challenge_2019",
"authors": ["your-aicrowd-username"],
"description": "sample description about your awesome agent",
"license": "MIT",
"debug": false
}
```
If you are not familiar with working in the shell. Use
```sh
nano aicrowd.json
```
to edit the JSON file.
This is used to map your submission to the said challenge, so please remember to use the correct `challenge_id` and `grader_id` as specified above.
If you set `debug` to `true`, then the evaluation will run on a separate set of 20 environments, and the logs from your submitted code (if it fails), will be made available to you to help you debug.
**NOTE** : **IMPORTANT** : By default we have set `debug:false`, so when you have done the basic integration testing of your code, and are ready to make a final submission, please do make sure to set `debug` to `true` in `aicrowd.json`.
### Code Entrypoint
The evaluator will use `/home/aicrowd/run.sh` as the entrypoint, so please remember to have a `run.sh` at the root, which can instantitate any necessary environment variables, and also start executing your actual code. This repository includes a sample `run.sh` file.
If you are using a Dockerfile to specify your software environment, please remember to create a `aicrowd` user, and place the entrypoint code at `run.sh`.
If you are unsure what this is all about, you can let `run.sh` be as is, and instead focus on the `run.py` which is being called from within `run.sh`.
## Submission
To make a submission, you will have to create a private repository on [https://gitlab.aicrowd.com/](https://gitlab.aicrowd.com/).
You will have to add your SSH Keys to your GitLab account if you haven't already by following the instructions [here](https://docs.gitlab.com/ee/gitlab-basics/create-your-ssh-keys.html).
If you do not have SSH Keys, you will first need to [generate one](https://docs.gitlab.com/ee/ssh/README.html#generating-a-new-ssh-key-pair).
Then you can create a submission by making a _tag push_ to your repository on [https://gitlab.aicrowd.com/](https://gitlab.aicrowd.com/).
**Any tag push (where the tag name begins with "submission-") to your private repository is considered as a submission** . Be sure that all changes where added to the commit before pushing.
Then you can add the correct git remote, and finally submit by doing :
```
cd flatland-challenge-starter-kit
# Add AIcrowd git remote endpoint
git remote add aicrowd git@gitlab.aicrowd.com:<YOUR_AICROWD_USER_NAME>/flatland-challenge-starter-kit.git
git push aicrowd master
# Create a tag for your submission and push
git tag -am "submission-v0.1" submission-v0.1
git push aicrowd master
git push aicrowd submission-v0.1
# Note : If the contents of your repository (latest commit hash) does not change,
# then pushing a new tag will **not** trigger a new evaluation.
```
You now should be able to see the details of your submission at :
[gitlab.aicrowd.com//<YOUR_AICROWD_USER_NAME>/flatland-challenge-starter-kit/issues](gitlab.aicrowd.com//<YOUR_AICROWD_USER_NAME>/flatland-challenge-starter-kit/issues)
**NOTE**: Remember to update your username in the link above :wink:
In the link above, you should start seeing something like this take shape (the whole evaluation can take a bit of time, so please be a bit patient too :wink: ) :
![](https://i.imgur.com/4HWf1jU.png)
**Best of Luck** :tada: :tada:
# Author
Sharada Mohanty <https://twitter.com/MeMohanty>
Erik Nygren
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment