Commit b7e1a791 authored by pfrl_rainbow's avatar pfrl_rainbow
Browse files

Initial commit

parents
data/
shared/
logs/
.gradle/
*.pyc
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
.pytest_cache/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
.python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# celery beat schedule file
celerybeat-schedule
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# NeurIPS 2020: MineRL Competition Starter Kit
[![Discourse status](https://img.shields.io/discourse/https/discourse.aicrowd.com/status.svg)](https://discourse.aicrowd.com/) [![Discord](https://img.shields.io/discord/565639094860775436.svg)](https://discord.gg/BT9uegr)
This repository is the main MineRL Competition **submission template and starter kit**! Compete to solve `MineRLObtainDiamondVectorObf-v0` now!
**This repository contains**:
* **Documentation** on how to submit your agent to the leaderboard
* **The procedure** for Round 1 (how long you should train your agent, how we evaluate and re-train your agent, etc.)
* **Starter code** for you to base your submission!
**Other Resources**:
- [MineRL Competition Page](https://www.aicrowd.com/challenges/neurips-2020-minerl-competition) - Main registration page & leaderboard.
- [MineRL Documentation](http://minerl.io/docs) - Documentation for the `minerl` package and dataset!
- [Example Baselines](https://github.com/minerllabs/baselines) - A set of competition and non-competition baselines for `minerl`.
![](https://i.imgur.com/XB1WORT.gif)
# Competition Procedure: Round 1
Welcome to Round 1! The main task of the competition is solving the `MineRLObtainDiamondVectorObf-v0` environment. In this environment, the agent begins in a random starting location without any items, and is tasked with obtaining a diamond. This task can only be accomplished by navigating the complex item hierarchy of Minecraft.
In this round you will train your agents locally with a limited number of samples and then upload them to AIcrowd (via git) to be evaluated and retrained by the organizers!
**The following is a high level description of how this round works**
![](http://minerl.io/assets/images/round1_procedure.png)
1. **Sign up** to join the competition [on the AIcrowd website.](https://www.aicrowd.com/challenges/neurips-2020-minerl-competition)
2. **Clone** this repo and start developing your submissions.
3. **Train** your models against `MineRLObtainDiamondVectorObf-v0` using the `train_locally.sh` or on Azure with **only 8,000,000 samples** in less than **four days** using hardware **no powerful than a NG6v2 instance** (6 CPU cores, 112 GiB RAM, 736 GiB SDD, and a single NVIDIA P100 GPU - to be confirmed)
4. [**Submit**](https://github.com/minerllabs/competition_submission_starter_template#how-to-submit-a-model) your trained models to [AIcrowd Gitlab](https://gitlab.aicrowd.com) for evaluation [(full instructions below)](#how-to-submit-a-model). The automated evaluation setup will evaluate the submissions against the validation environment, to compute and report the metrics on the leaderboard of the competition.
Once Round 1 is complete, the organizers will:
1. **Examine** the code repositories of the top submissions on the leaderboard to ensure compliance with the competition rules.
2. **Retrain** the top submissions from scratch to ensure reproducibility of the leaderboard score! **NOTE: Make sure that you train your models in UNDER 8,000,000 samples using a similar (or worse) hardware spec than above** so that you are **not disqualified** for a score mismatch!
3. **Evaluate** the resulting models again over several hundred episodes to determine the final ranking.
The code repositories associated with the corresponding submissions will be forked and scrubbed of any files larger than 15MB to ensure that participants are not using any pre-trained models in the subsequent round.
# How to Submit a Model!
## Setup
1. **Clone the github repository** or press the "Use this Template" button on GitHub!
```
git clone https://github.com/minerllabs/competition_submission_starter_template.git
```
2. **Install** competition specific dependencies! **Make sure you have the [JDK 8 installed first](http://minerl.io/docs/tutorials/getting_started.html)!**
```
# 1. Make sure to install the JDK first
# -> Go to http://minerl.io/docs/tutorials/getting_started.html
# 2. Install the `minerl` package and the dependencies for the competition
cd competition_submission_starter_template
pip3 install -r requirements.txt
```
3. **Specify** your specific submission dependencies (PyTorch, Tensorflow, kittens, etc.)
* (Optional) **Anaconda Environment**. If you would like to use anaconda to manage your environment, make sure at least version `4.5.11` is required to correctly populate `environment.yml` (By following instructions [here](https://www.anaconda.com/download)). Then:
* **Create your new conda environment**
```sh
conda create --name minerl_challenge
conda activate minerl_challenge
```
* **Your code specific dependencies**
```sh
conda install <your-package>
```
* **Pip Packages** If you are using specific Python packages **make sure to add them to** `requirements.txt`! Here's an example:
```
# requirements.txt
minerl>=0.3.5
matplotlib
tensorflow
```
* **Apt Packages** If your training procedure or agent depends on specific Debian (Ubuntu, etc.) packages, add them to `apt.txt`.
## How do I specify my software runtime ?
As mentioned above, **the software runtime is specified in 3 places**:
* `environment.yml` -- The _optional_ Anaconda environment specification.
As you add new requirements you can export your `conda` environment to this file!
```
conda env export --no-build > environment.yml
```
* `requirements.txt` -- The `pip3` packages used by your agent to train. **Note that dependencies specified by `environment.yml` take precedence over `requirements.txt`.** As you add new pip3 packages to your training procedure either manually add them to `requirements.txt` or if your software runtime is simple, perform:
```
# Put ALL of the current pip3 packages on your system in the submission
pip3 freeze > requirements.txt
```
* `apt.txt` -- The Debian packages (via aptitude) used by your training procedure!
These files are used to construct both the **local and AICrowd docker containers** in which your agent will train.
## What should my code structure be like ?
Please follow the example structure shared in the starter kit for the code structure.
The different files and directories have following meaning:
```
.
├── aicrowd.json # Submission meta information like your username
├── apt.txt # Packages to be installed inside docker image
├── data # The downloaded data, the path to directory is also available as `MINERL_DATA_ROOT` env variable
├── requirements.txt # Python packages to be installed
├── test.py # IMPORTANT: Your testing/inference phase code, must include main() method
├── train # Your trained model MUST be saved inside this directory, must include main() method
├── train.py # IMPORTANT: Your training phase code
└── utility # The utility scripts to provide smoother experience to you.
├── debug_build.sh
├── docker_run.sh
├── environ.sh
├── evaluation_locally.sh
├── parser.py
├── train_locally.sh
└── verify_or_download_data.sh
```
Finally, **you must specify an AIcrowd submission JSON in `aicrowd.json` to be scored!**
The `aicrowd.json` of each submission should contain the following content:
```json
{
"challenge_id": "aicrowd-neurips-2020-minerl-challenge",
"grader_id": "aicrowd-neurips-2020-minerl-challenge",
"authors": ["your-aicrowd-username"],
"tags": ["change-me"],
"description": "sample description about your awesome agent",
"license": "MIT",
"gpu": true
}
```
This JSON is used to map your submission to the said challenge, so please remember to use the correct `challenge_id` and `grader_id` as specified above.
Please specify if your code will use a GPU or not for the evaluation of your model. If you specify `true` for the GPU, a **NVIDIA Tesla K80 GPU** will be provided and used for the evaluation.
**Remember: You need to specify "tags" in aicrowd.json, which need to be one of `"RL"`, `"IL"`, `["RL", "IL"]`.**
### Dataset location
You **don't** need to upload the data set in submission and it will be provided in online submissions at `MINERL_DATA_ROOT` path. For local training and evaluations, you can download it once in your system via `python /utility/verify_or_download_data.py` or place manually into `data/` folder.
## Training and Testing Code Entrypoint (where you write your code!)
The evaluator will use `train.py` and `test.py` as the entrypoint for training and testing/inference stage respectively, so please remember to include the files in your submission!
The inline documentation in these files will guide you in interfacing with evaluator properly.
## IMPORTANT: Saving Models during Training!
Before you sbumit make sure that your code does the following.
* **During training** (`train.py`) **save your models to the `train/` folder.**
* **During testing** (`test.py`) **load your model from the `train/` folder.**
It is absolutely imperative **that you save your models during training** (`train.py`) so that they can be used in the evaluation phase (`test.py`) on AICrowd, and so the oraganizers can retrain your models from scratch at the end of Round 1 and during Round 2!
## How to submit a trained agent!
To make a submission, you will have to create a private repository on [https://gitlab.aicrowd.com/](https://gitlab.aicrowd.com/).
You will have to add your SSH Keys to your GitLab account by following the instructions [here](https://docs.gitlab.com/ee/gitlab-basics/create-your-ssh-keys.html).
If you do not have SSH Keys, you will first need to [generate one](https://docs.gitlab.com/ee/ssh/README.html#generating-a-new-ssh-key-pair).
Then you can create a submission by making a _tag push_ to your repository on [https://gitlab.aicrowd.com/](https://gitlab.aicrowd.com/).
**Any tag push (where the tag name begins with "submission-") to your private repository is considered as a submission**
Then you can add the correct git remote, and finally submit by doing :
```
cd competition_submission_starter_template
# Add AIcrowd git remote endpoint
git remote add aicrowd git@gitlab.aicrowd.com:<YOUR_AICROWD_USER_NAME>/competition_submission_starter_template.git
git push aicrowd master
# Create a tag for your submission and push
git tag -am "submission-v0.1" submission-v0.1
git push aicrowd master
git push aicrowd submission-v0.1
# Note : If the contents of your repository (latest commit hash) does not change,
# then pushing a new tag will **not** trigger a new evaluation.
```
You now should be able to see the details of your submission at :
[gitlab.aicrowd.com/<YOUR_AICROWD_USER_NAME>/competition_submission_starter_template/issues](gitlab.aicrowd.com//<YOUR_AICROWD_USER_NAME>/competition_submission_starter_template/issues)
**NOTE**: Remember to update your username in the link above :wink:
In the link above, you should start seeing something like this take shape (each of the steps can take a bit of time, so please be patient too :wink: ) :
![](https://i.imgur.com/FqScw4m.png)
and if everything works out correctly, then you should be able to see the final scores like this :
![](https://i.imgur.com/u00qcif.png)
**Best of Luck** :tada: :tada:
# Other Concepts
## Time constraints
### Round 1
You have to train your models locally **with under 8,000,000 samples** and with **worse or comprable hardware to that above** and upload the trained model in `train/` directory. But, to make sure, your training code is compatible with further round's interface, the training code will be executed in this round as well. The constraints will be timeout of 5 minutes.
### Round 2
You are expected to train your model online using the training phase docker container and output the trained model in `train/` directory. You need to ensure that your submission is trained in under 8,000,000 samples and within 4 days period. Otherwise, the container will be killed
## Local evaluation
You can perform local training and evaluation using utility scripts shared in this directory. To mimic the online training phase you can run `./utility/train_locally.sh` from repository root, you can specify `--verbose` for complete logs.
```
aicrowd_minerl_starter_kit❯ ./utility/train_locally.sh --verbose
2019-07-22 07:58:38 root[77310] INFO Training Start...
2019-07-22 07:58:38 crowdai_api.events[77310] DEBUG Registering crowdAI API Event : CROWDAI_EVENT_INFO training_started {'event_type': 'minerl_challenge:training_started'} # with_oracle? : False
2019-07-22 07:58:40 minerl.env.malmo.instance.17c149[77310] INFO Starting Minecraft process: ['/var/folders/82/wsds_18s5dq321scc1j531m40000gn/T/tmpnyzpjrsc/Minecraft/launchClient.sh', '-port', '9001', '-env', '-runDir', '/var/folders/82/wsds_18s5dq321scc1j531m40000gn/T/tmpnyzpjrsc/Minecraft/run']
2019-07-22 07:58:40 minerl.env.malmo.instance.17c149[77310] INFO Starting process watcher for process 77322 @ localhost:9001
2019-07-22 07:58:48 minerl.env.malmo.instance.17c149[77310] DEBUG This mapping 'snapshot_20161220' was designed for MC 1.11! Use at your own peril.
2019-07-22 07:58:48 minerl.env.malmo.instance.17c149[77310] DEBUG #################################################
2019-07-22 07:58:48 minerl.env.malmo.instance.17c149[77310] DEBUG ForgeGradle 2.2-SNAPSHOT-3966cea
2019-07-22 07:58:48 minerl.env.malmo.instance.17c149[77310] DEBUG https://github.com/MinecraftForge/ForgeGradle
2019-07-22 07:58:48 minerl.env.malmo.instance.17c149[77310] DEBUG #################################################
2019-07-22 07:58:48 minerl.env.malmo.instance.17c149[77310] DEBUG Powered by MCP unknown
2019-07-22 07:58:48 minerl.env.malmo.instance.17c149[77310] DEBUG http://modcoderpack.com
2019-07-22 07:58:48 minerl.env.malmo.instance.17c149[77310] DEBUG by: Searge, ProfMobius, Fesh0r,
2019-07-22 07:58:48 minerl.env.malmo.instance.17c149[77310] DEBUG R4wk, ZeuX, IngisKahn, bspkrs
2019-07-22 07:58:48 minerl.env.malmo.instance.17c149[77310] DEBUG #################################################
2019-07-22 07:58:48 minerl.env.malmo.instance.17c149[77310] DEBUG Found AccessTransformer: malmomod_at.cfg
2019-07-22 07:58:49 minerl.env.malmo.instance.17c149[77310] DEBUG :deobfCompileDummyTask
2019-07-22 07:58:49 minerl.env.malmo.instance.17c149[77310] DEBUG :deobfProvidedDummyTask
...
```
For local evaluation of your code, you can use `./utility/evaluation_locally.sh`, add `--verbose` if you want to view complete logs.
```
aicrowd_minerl_starter_kit❯ ./utility/evaluation_locally.sh
{'state': 'RUNNING', 'score': {'score': '0.0', 'score_secondary': 0.0}, 'instances': {'1': {'totalNumberSteps': 1001, 'totalNumberEpisodes': 0, 'currentEnvironment': 'MineRLObtainDiamondVectorObf-v0', 'state': 'IN_PROGRESS', 'episodes': [{'numTicks': 1001, 'environment': 'MineRLObtainDiamondVectorObf-v0', 'rewards': 0.0, 'state': 'IN_PROGRESS'}], 'score': {'score': '0.0', 'score_secondary': 0.0}}}}
{'state': 'RUNNING', 'score': {'score': '0.0', 'score_secondary': 0.0}, 'instances': {'1': {'totalNumberSteps': 2001, 'totalNumberEpisodes': 0, 'currentEnvironment': 'MineRLObtainDiamondVectorObf-v0', 'state': 'IN_PROGRESS', 'episodes': [{'numTicks': 2001, 'environment': 'MineRLObtainDiamondVectorObf-v0', 'rewards': 0.0, 'state': 'IN_PROGRESS'}], 'score': {'score': '0.0', 'score_secondary': 0.0}}}}
{'state': 'RUNNING', 'score': {'score': '0.0', 'score_secondary': 0.0}, 'instances': {'1': {'totalNumberSteps': 3001, 'totalNumberEpisodes': 0, 'currentEnvironment': 'MineRLObtainDiamondVectorObf-v0', 'state': 'IN_PROGRESS', 'episodes': [{'numTicks': 3001, 'environment': 'MineRLObtainDiamondVectorObf-v0', 'rewards': 0.0, 'state': 'IN_PROGRESS'}], 'score': {'score': '0.0', 'score_secondary': 0.0}}}}
{'state': 'RUNNING', 'score': {'score': '0.0', 'score_secondary': 0.0}, 'instances': {'1': {'totalNumberSteps': 4001, 'totalNumberEpisodes': 0, 'currentEnvironment': 'MineRLObtainDiamondVectorObf-v0', 'state': 'IN_PROGRESS', 'episodes': [{'numTicks': 4001, 'environment': 'MineRLObtainDiamondVectorObf-v0', 'rewards': 0.0, 'state': 'IN_PROGRESS'}], 'score': {'score': '0.0', 'score_secondary': 0.0}}}}
{'state': 'RUNNING', 'score': {'score': '0.0', 'score_secondary': 0.0}, 'instances': {'1': {'totalNumberSteps': 5001, 'totalNumberEpisodes': 0, 'currentEnvironment': 'MineRLObtainDiamondVectorObf-v0', 'state': 'IN_PROGRESS', 'episodes': [{'numTicks': 5001, 'environment': 'MineRLObtainDiamondVectorObf-v0', 'rewards': 0.0, 'state': 'IN_PROGRESS'}], 'score': {'score': '0.0', 'score_secondary': 0.0}}}}
{'state': 'RUNNING', 'score': {'score': '0.0', 'score_secondary': 0.0}, 'instances': {'1': {'totalNumberSteps': 6001, 'totalNumberEpisodes': 0, 'currentEnvironment': 'MineRLObtainDiamondVectorObf-v0', 'state': 'IN_PROGRESS', 'episodes': [{'numTicks': 6001, 'environment': 'MineRLObtainDiamondVectorObf-v0', 'rewards': 0.0, 'state': 'IN_PROGRESS'}], 'score': {'score': '0.0', 'score_secondary': 0.0}}}}
...
```
For running/testing your submission in a docker environment (ideantical to online submission), you can use `./utility/docker_train_locally.sh` and `./utility/docker_evaluation_locally.sh`. You can also run docker image with bash entrypoint for debugging on the go with the help of `./utility/docker_run.sh`. These scripts respect following parameters:
* `--no-build`: To skip docker image build and use the last build image
* `--nvidia`: To use `nvidia-docker` instead of `docker` which include your nvidia related drivers inside docker image
# Team
The quick-start kit was authored by
**[Shivam Khandelwal](https://twitter.com/skbly7)** with help from [William H. Guss](http://wguss.ml)
The competition is organized by the following team:
* [William H. Guss](http://wguss.ml) (Carnegie Mellon University)
* Mario Ynocente Castro (Preferred Networks)
* Cayden Codel (Carnegie Mellon University)
* Katja Hofmann (Microsoft Research)
* Brandon Houghton (Carnegie Mellon University)
* Noboru Kuno (Microsoft Research)
* Crissman Loomis (Preferred Networks)
* Keisuke Nakata (Preferred Networks)
* Stephanie Milani (University of Maryland, Baltimore County and Carnegie Mellon University)
* Sharada Mohanty (AIcrowd)
* Diego Perez Liebana (Queen Mary University of London)
* Ruslan Salakhutdinov (Carnegie Mellon University)
* Shinya Shiroshita (Preferred Networks)
* Nicholay Topin (Carnegie Mellon University)
* Avinash Ummadisingu (Preferred Networks)
* Manuela Veloso (Carnegie Mellon University)
* Phillip Wang (Carnegie Mellon University)
<img src="https://d3000t1r8yrm6n.cloudfront.net/images/challenge_partners/image_file/35/CMU_wordmark_1500px-min.png" width="50%">
<img src="https://d3000t1r8yrm6n.cloudfront.net/images/challenge_partners/image_file/34/MSFT_logo_rgb_C-Gray.png" width="20%" style="margin-top:10px">
<img src="https://raw.githubusercontent.com/AIcrowd/AIcrowd/master/app/assets/images/misc/aicrowd-horizontal.png" width="20%"> <img src="https://d3000t1r8yrm6n.cloudfront.net/images/challenge_partners/image_file/38/PFN_logo.png" width="15%" style="margin-top:10px">
{
"challenge_id": "aicrowd-neurips-2020-minerl-challenge",
"grader_id": "aicrowd-neurips-2020-minerl-challenge",
"authors": ["aicrowd-bot"],
"tags": "change-me",
"description": "Test Model for MineRL Challenge",
"gpu": false
}
#!/usr/bin/env python
import crowdai_api
import os
import logging
########################################################################
# Instatiate Event Notifier
########################################################################
crowdai_events = crowdai_api.events.CrowdAIEvents()
current_phase = None
training_progress = 0.0
def inference_start():
########################################################################
# Register Inference Start event
########################################################################
logging.info("Inference Start...")
global current_phase
current_phase = "inference"
crowdai_events.register_event(
event_type=crowdai_events.CROWDAI_EVENT_INFO,
message="inference_started",
payload={ #Arbitrary Payload
"event_type": "minerl_challenge:inference_started"
}
)
def inference_end():
########################################################################
# Register Inference End event
########################################################################
logging.info("Inference End...")
global current_phase
current_phase = None
crowdai_events.register_event(
event_type=crowdai_events.CROWDAI_EVENT_INFO,
message="inference_ended",
payload={ #Arbitrary Payload
"event_type": "minerl_challenge:inference_ended"
}
)
def inference_error():
########################################################################
# Register Inference Error event
########################################################################
logging.error("Inference Failed...")
crowdai_events.register_event(
event_type=crowdai_events.CROWDAI_EVENT_INFO,
message="inference_error",
payload={ #Arbitrary Payload
"event_type": "minerl_challenge:inference_error"
}
)
def training_start():
########################################################################
# Register Training Start event
########################################################################
logging.info("Training Start...")
global current_phase
current_phase = "training"
crowdai_events.register_event(
event_type=crowdai_events.CROWDAI_EVENT_INFO,
message="training_started",
payload={ #Arbitrary Payload
"event_type": "minerl_challenge:training_started"
}
)
def training_end():
########################################################################
# Register Training End event
########################################################################
logging.info("Training End...")
register_progress(1.0)
global current_phase
current_phase = None
crowdai_events.register_event(
event_type=crowdai_events.CROWDAI_EVENT_INFO,
message="training_ended",
payload={ #Arbitrary Payload
"event_type": "minerl_challenge:training_ended"
}
)
def training_error():
########################################################################
# Register Training Error event
########################################################################
logging.error("Training Failed...")
crowdai_events.register_event(
event_type=crowdai_events.CROWDAI_EVENT_INFO,
message="training_error",
payload={ #Arbitrary Payload
"event_type": "minerl_challenge:training_error"
}
)
def register_progress(progress):
########################################################################
# Register Evaluation Progress event
# progress : float [0, 1]
########################################################################
logging.info("Progress : {}".format(progress))
global training_progress, current_phase
if current_phase is None:
raise Exception('Please register current phase by calling `training_start` \
or `inference_start` before sending progress.')
if current_phase == "training":
if progress < training_progress:
logging.warn('Invalid progress update to %f while you are already \
at %f. Skipping it...', progress, training_progress)
return
training_progress = progress
crowdai_events.register_event(
event_type=crowdai_events.CROWDAI_EVENT_INFO,
message="register_progress",
payload={ #Arbitrary Payload
"event_type": "minerl_challenge:register_progress",
"training_progress" : training_progress
}
)
def submit(payload={}):
########################################################################
# Register Evaluation Complete event
########################################################################
logging.info("AIcrowd Submit")
crowdai_events.register_event(
event_type=crowdai_events.CROWDAI_EVENT_SUCCESS,
message="submit",
payload={ #Arbitrary Payload
"event_type": "minerl_challenge:submit",
},
blocking=True
)
def execution_error(error):
########################################################################
# Register Evaluation Complete event
########################################################################
crowdai_events.register_event(
event_type=crowdai_events.CROWDAI_EVENT_ERROR,
message="execution_error",
payload={ #Arbitrary Payload
"event_type": "minerl_challenge:execution_error",
"error" : error
},
blocking=True
)
curl
git
vim
ssh
gcc
python-dev
libsm6
libxext6
libxrender-dev
libglib2.0-0
openjdk-8-jdk
xvfb
x11vnc
freeglut3-dev
libx11-6
python-opengl
x11-xserver-utils
import aicrowd_helper
import train
import test
import os
EVALUATION_RUNNING_ON = os.getenv('EVALUATION_RUNNING_ON', None)
EVALUATION_STAGE = os.getenv('EVALUATION_STAGE', 'all')