README.md 8.83 KB
Newer Older
Jyotish P's avatar
Jyotish P committed
1 2
![](https://i.imgur.com/jbZ1HlW.png)
<h1 align="center"><a href="https://www.aicrowd.com/challenges/the-neural-mmo-challenge">The Neural MMO Challenge</a> - Starter Kit</h1>
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
3

Jyotish P's avatar
Jyotish P committed
4 5 6
<p align="center">
 <a href="https://discord.gg/N3cWFhsk8K"><img src="https://img.shields.io/discord/657211973435392011?style=for-the-badge" alt="chat on Discord"></a>
</p>
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
7

Jyotish P's avatar
Jyotish P committed
8
This is the starter kit for the [Neural MMO challenge](https://www.aicrowd.com/challenges/the-neural-mmo-challenge) hosted on [AIcrowd](https://www.aicrowd.com). Clone the repository to compete now!
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
9

Jyotish P's avatar
Jyotish P committed
10
This repository contains:
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
11

Jyotish P's avatar
Jyotish P committed
12 13 14
- **Documentation** on how to submit your models to the leaderboard.
- Information on **evaluating your agents locally**, **baselines** and some best practises to have hassle free submissions.
- **Starter code** for you to get started!
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
15

Jyotish P's avatar
Jyotish P committed
16
# Table of contents
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
17

Jyotish P's avatar
Jyotish P committed
18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
- [📚 Competition procedure](#-competition-procedure)
- [💪 Getting started](#-getting-started)
- [🛠 Preparing your submission](#-preparing-your-submission)
  * [Write your agents](#write-your-agents)
  * [Evaluate your agents locally](#evaluate-your-agents-locally)
- [📨 Submission](#-submission)
  * [Repository Structure](#repository-structure)
  * [Runtime configuration](#runtime-configuration)
  * [🚀 Submitting to AIcrowd](#-submitting-to-aicrowd)
    + [`aicrowd.json`](#aicrowdjson)
    + [Configuring the submission repository](#configuring-the-submission-repository)
    + [Pushing the code to AIcrowd](#pushing-the-code-to-aicrowd)
- [📝 Submission checklist](#-submission-checklist)
- [📎 Important links](#-important-links)
- [✨ Contributors](#-contributors)
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
33

Jyotish P's avatar
Jyotish P committed
34
# 📚 Competition procedure
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
35

Jyotish P's avatar
Jyotish P committed
36
The Neural MMO Challenge is an opportunity for researchers and machine learning enthusiasts to test their skills by designing and building agents that can survive and thrive in a massively multiagent environment full of potential adversaries.
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
37

Jyotish P's avatar
Jyotish P committed
38
In this challenge, you will train your models locally and then upload them to AIcrowd (via git) to be evaluated.
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
39

Jyotish P's avatar
Jyotish P committed
40
**The following is a high level description of how this process works.**
41

Jyotish P's avatar
Jyotish P committed
42
![](https://i.imgur.com/xzQkwKV.jpg)
43

Jyotish P's avatar
Jyotish P committed
44 45 46 47
1. **Sign up** to join the competition [on the AIcrowd website](https://www.aicrowd.com/challenges/the-neural-mmo-challenge).
2. **Clone** this repo and start developing your solution.
3. **Design and build** your agents that can compete in Neural MMO environment and implement an agent class as described in [writing your agents](#write-your-agents) section.
4. [**Submit**](#-submission) your agents to [AIcrowd Gitlab](https://gitlab.aicrowd.com) for evaluation. [[Refer this for detailed instructions]](#-submission).
48

Jyotish P's avatar
Jyotish P committed
49
# 💪 Getting started
50

Jyotish P's avatar
Jyotish P committed
51
> We recommend using `python 3.8`. If you are using Miniconda/Anaconda, you can install it using `conda install python=3.8`.
52

Jyotish P's avatar
Jyotish P committed
53
Clone the starter kit repository and install the dependencies.
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
54 55

```bash
Jyotish P's avatar
Jyotish P committed
56 57 58
git clone https://gitlab.aicrowd.com/neural-mmo/neural-mmo-starter-kit
cd neural-mmo-starter-kit
pip install -U -r requirements.txt
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
59
```
Jyotish P's avatar
Jyotish P committed
60 61 62
   
Generate the Neural MMO environment maps.
   
63
```bash
Jyotish P's avatar
Jyotish P committed
64
neural-mmo-forge generate
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
65 66
```

Jyotish P's avatar
Jyotish P committed
67
# 🛠 Preparing your submission
68

Jyotish P's avatar
Jyotish P committed
69
## Write your agents
70

Jyotish P's avatar
Jyotish P committed
71
Your agents need to implement the [`NeuralMMOAgent`](evaluator/base_agent.py#L4) class from [`evaluator/base_agent.py`](evaluator/base_agent.py). You can check the code in [`agents`](agents) directory for examples.
72

Jyotish P's avatar
Jyotish P committed
73
**Note:** If your agent doesn't inherit the `NeuralMMOAgent` class, the evaluation will fail.
74

Jyotish P's avatar
Jyotish P committed
75
Once your agent class is ready, you can specify the class to use as the player agent in your [`config.py`](config.py). The starter kit comes with a machine learning based baseline. The [`config.py`](config.py) in the starter kit points to this class. You should update it to use your class.
76

Jyotish P's avatar
Jyotish P committed
77
For information on tweaking and training the baseline agents, please refer [`training baselines`](docs/training-baselines.md).
78

Jyotish P's avatar
Jyotish P committed
79
## Evaluate your agents locally
80

Jyotish P's avatar
Jyotish P committed
81
We have provided [`evaluator/rollout.py`](evaluator/rollout.py) to test your agents locally. This file reads the configuration options specified in [`config.py`](config.py), runs the rollouts and reports the rewards obtained by the agents.
82

Jyotish P's avatar
Jyotish P committed
83
To run the evaluation locally, run the following command.
84

Jyotish P's avatar
Jyotish P committed
85 86 87
```bash
python evaluator/rollout.py
```
88

Jyotish P's avatar
Jyotish P committed
89
**Note:** Please note that the changes you make to any file inside [`evaluator`](evaluator) directory will be dropped during evaluation. We will use a slightly modified version of the [`evaluator/rollout.py`](evaluator/rollout.py) during the evaluation.
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
90

Jyotish P's avatar
Jyotish P committed
91
# 📨 Submission
92

Jyotish P's avatar
Jyotish P committed
93
## Repository Structure
94

Jyotish P's avatar
Jyotish P committed
95 96 97 98 99 100 101 102 103
**File/Directory** | **Description**
--- | ---
[`agents`](agents) | Directory containing different scripted bots, baseline agent and bots performing random actions. We recommend that you add your agents to this directory.
[`config.py`](config.py) | File containing the configuration options for local evaluation. We will use the same player agent you specify here during the evaluation.
[`utils/submit.sh`](utils/submit.sh) | Helper script to submit your repository to [AIcrowd GitLab](https://gitlab.aicrowd.com).
[`Dockerfile`](Dockerfile) | Docker config for your submission. Refer [runtime configuration](#runtime-configuration) for more information.
[`requirements.txt`](requirements.txt) | File containing the list of python packages you want to install for the submission to run. Refer [runtime configuration](#runtime-configuration) for more information.
[`apt.txt`](apt.txt) | File containing the list of packages you want to install for submission to run. Refer [runtime configuration](#runtime-configuration) for more information.
[`evaluator`](evaluator) | Directory containing scripts for local evaluation. **Do not make changes to the files in this directory. Any changes made will be dropped during evaluation.**
104

Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
105

Jyotish P's avatar
Jyotish P committed
106
## Runtime configuration
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
107

Jyotish P's avatar
Jyotish P committed
108
You can specify the list of python packages needed for your code to run in your [`requirements.txt`](requirements.txt) file. We will install the packages using `pip install` command.
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
109

Jyotish P's avatar
Jyotish P committed
110
You can also specify the OS packages needed using [`apt.txt`](apt.txt) file. We install these packages using `apt-get install` command.
111

Jyotish P's avatar
Jyotish P committed
112
For more information on how you can configure the evaluation runtime, please refer [`RUNTIME.md`](docs/RUNTIME.md).
113

Jyotish P's avatar
Jyotish P committed
114
## 🚀 Submitting to AIcrowd
115

Jyotish P's avatar
Jyotish P committed
116
### `aicrowd.json`
117

Jyotish P's avatar
Jyotish P committed
118
Your repository should have an `aicrowd.json` file with following fields:
119 120 121

```json
{
Jyotish P's avatar
Jyotish P committed
122 123 124 125
    "challenge_id" : "the-neural-mmo-challenge",
    "grader_id" : "the-neural-mmo-challenge",
    "authors" : ["Your Name"],
    "description" : "Brief description for your submission"
126 127 128
}
```

Jyotish P's avatar
Jyotish P committed
129
This file is used to identify your submission as a part of the Neural MMO Challenge. You must use the `challenge_id`, and `grader_id` as specified above.
130

Jyotish P's avatar
Jyotish P committed
131
### Configuring the submission repository
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
132

Jyotish P's avatar
Jyotish P committed
133 134
```bash
git remote add aicrowd git@gitlab.aicrowd.com:<username>/neural-mmo-starter-kit.git
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
135 136
```

Jyotish P's avatar
Jyotish P committed
137
**Note:** This needs to be done only once. This configuration will be saved in your repository for future use.
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
138

Jyotish P's avatar
Jyotish P committed
139
### Pushing the code to AIcrowd
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
140

Jyotish P's avatar
Jyotish P committed
141 142 143
```bash
./utils/submit.sh "some description"
```
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
144

Jyotish P's avatar
Jyotish P committed
145
If you want to submit without the helper script, please refer [`SUBMISSION.md`](docs/SUBMISSION.md).
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
146 147


Jyotish P's avatar
Jyotish P committed
148
# 📝 Submission checklist
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
149

Jyotish P's avatar
Jyotish P committed
150 151 152 153 154
- [x] **Accept the challenge rules**. You can do this by going to the [challenge overview page](https://www.aicrowd.com/challenges/the-neural-mmo-challenge) and clicking the "Participate" button. You only need to do this once.
- [x] **Add your agent code** that implements the `NeuralMMOBaseAgent` class from `evaluator/base_agent`.
- [x] **Add your model checkpoints** (if any) to the repo. The `utils/submit.sh` will automatically detect large files and add them to git LFS. If you are using the script, please refer to [this post explaining how to add your models](https://discourse.aicrowd.com/t/how-to-upload-large-files-size-to-your-submission/2304).
- [x] **Evaluate your agents locally** to know that they work as expected.
- [x] **Update runtime configuration** using `requirements.txt`, `apt.txt` and/or `Dockerfile` as necessary. Please make sure that you specified the same package versions that you use locally on your machine.
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
155

Jyotish P's avatar
Jyotish P committed
156
# 📎 Important links
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
157

Jyotish P's avatar
Jyotish P committed
158 159 160 161 162 163 164 165 166 167
- 💪 Challenge information
   * [Challenge page](https://www.aicrowd.com/challenges/the-neural-mmo-challenge)
   * [Leaderboard](https://www.aicrowd.com/challenges/the-neural-mmo-challenge)
 - 🗣 Community
    * [Neural MMO discord server](https://discord.gg/N3cWFhsk8K)
    * [Challenge discussion forum](https://discourse.aicrowd.com/c/the-neural-mmo-challenge/)
- 🎮 Neural MMO resources
    * [Neural MMO documentation](https://jsuarez5341.github.io)
    * [Neural MMO GitHub repository](https://github.com/jsuarez5341/neural-mmo)
    
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
168

Jyotish P's avatar
Jyotish P committed
169
# ✨ Contributors
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
170

171 172 173 174
- [Siddhartha Laghuvarapu](https://www.aicrowd.com/participants/siddhartha)
- [Jyotish Poonganam](https://www.aicrowd.com/participants/jyotish)
- [Joseph Suarez](https://www.aicrowd.com/participants/joseph_suarez)
- [Dipam Chakraborty](https://www.aicrowd.com/participants/dipam)
Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
175

Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
176

Siddhartha Laghuvarapu's avatar
Siddhartha Laghuvarapu committed
177
**Best of Luck** 🎉