![](https://i.imgur.com/jbZ1HlW.png)

The Neural MMO Challenge - Starter Kit

chat on Discord

This is the starter kit for the [Neural MMO challenge](https://www.aicrowd.com/challenges/the-neural-mmo-challenge) hosted on [AIcrowd](https://www.aicrowd.com). Clone the repository to compete now! This repository contains: - **Documentation** on how to submit your models to the leaderboard. - Information on **evaluating your agents locally**, **baselines** and some best practises to have hassle free submissions. - **Starter code** for you to get started! [IMPORTANT - Accept the rules before you submit](https://www.aicrowd.com/challenges/the-neural-mmo-challenge/challenge_rules) # Table of contents - [📚 Competition procedure](#-competition-procedure) - [💪 Getting started](#-getting-started) - [🛠 Preparing your submission](#-preparing-your-submission) * [Write your agents](#write-your-agents) * [Evaluate your agents locally](#evaluate-your-agents-locally) - [📨 Submission](#-submission) * [Repository Structure](#repository-structure) * [Runtime configuration](#runtime-configuration) * [🚀 Submitting to AIcrowd](#-submitting-to-aicrowd) + [`aicrowd.json`](#aicrowdjson) + [Configuring the submission repository](#configuring-the-submission-repository) + [Pushing the code to AIcrowd](#pushing-the-code-to-aicrowd) - [📝 Submission checklist](#-submission-checklist) - [📎 Important links](#-important-links) - [✨ Contributors](#-contributors) # 📚 Competition procedure The Neural MMO Challenge is an opportunity for researchers and machine learning enthusiasts to test their skills by designing and building agents that can survive and thrive in a massively multiagent environment full of potential adversaries. In this challenge, you will train your models locally and then upload them to AIcrowd (via git) to be evaluated. **The following is a high level description of how this process works.** ![](https://i.imgur.com/xzQkwKV.jpg) 1. **Sign up** to join the competition [on the AIcrowd website](https://www.aicrowd.com/challenges/the-neural-mmo-challenge). 2. **Clone** this repo and start developing your solution. 3. **Design and build** your agents that can compete in Neural MMO environment and implement an agent class as described in [writing your agents](#write-your-agents) section. 4. [**Submit**](#-submission) your agents to [AIcrowd Gitlab](https://gitlab.aicrowd.com) for evaluation. [[Refer this for detailed instructions]](#-submission). # 💪 Getting started > We recommend using `python 3.8`. If you are using Miniconda/Anaconda, you can install it using `conda install python=3.8`. Clone the starter kit repository and install the dependencies. ```bash git clone https://gitlab.aicrowd.com/neural-mmo/neural-mmo-starter-kit cd neural-mmo-starter-kit pip install neural-mmo #Optional: Install ML dependencies pip install -U -r requirements.txt ``` Generate the Neural MMO environment maps. ```bash cd neuralmmo/ python Forge.py generate --config=CompetitionRound1 ``` # 🛠 Preparing your submission ## Write your agents Your agents need to implement the [`NeuralMMOAgent`](evaluator/base_agent.py#L4) class from [`evaluator/base_agent.py`](evaluator/base_agent.py). You can check the code in [`agents`](agents) directory for examples. **Note:** If your agent doesn't inherit the `NeuralMMOAgent` class, the evaluation will fail. Once your agent class is ready, you can specify the class to use as the player agent in your [`config.py`](config.py). The starter kit comes with a machine learning based baseline. The [`config.py`](config.py) in the starter kit points to this class. You should update it to use your class. For information on tweaking and training the baseline agents, please refer [`training baselines`](docs/training-baselines.md). ## Evaluate your agents locally We have provided [`evaluator/rollout.py`](evaluator/rollout.py) to test your agents locally. This file reads the configuration options specified in [`config.py`](config.py), runs the rollouts and reports the rewards obtained by the agents. To run the evaluation locally, run the following command. ```bash python evaluator/rollout.py ``` **Note:** Please note that the changes you make to any file inside [`evaluator`](evaluator) directory will be dropped during evaluation. We will use a slightly modified version of the [`evaluator/rollout.py`](evaluator/rollout.py) during the evaluation. # 📨 Submission ## Repository Structure **File/Directory** | **Description** --- | --- [`agents`](agents) | Directory containing different scripted bots, baseline agent and bots performing random actions. We recommend that you add your agents to this directory. [`config.py`](config.py) | File containing the configuration options for local evaluation. We will use the same player agent you specify here during the evaluation. [`utils/submit.sh`](utils/submit.sh) | Helper script to submit your repository to [AIcrowd GitLab](https://gitlab.aicrowd.com). [`Dockerfile`](Dockerfile) | Docker config for your submission. Refer [runtime configuration](#runtime-configuration) for more information. [`requirements.txt`](requirements.txt) | File containing the list of python packages you want to install for the submission to run. Refer [runtime configuration](#runtime-configuration) for more information. [`apt.txt`](apt.txt) | File containing the list of packages you want to install for submission to run. Refer [runtime configuration](#runtime-configuration) for more information. [`evaluator`](evaluator) | Directory containing scripts for local evaluation. **Do not make changes to the files in this directory. Any changes made will be dropped during evaluation.** [`neuralmmo/projekt`](projekt) | Per-round environment configs and RLlib demo. [`neuralmmo/Forge.py`](Forge.py) | Main file for baselines and demos [`neuralmmo/resource`](projekt) | Path where maps are generated. [`baselines`](Forge.py) | Path where models are saved ## Runtime configuration You can specify the list of python packages needed for your code to run in your [`requirements.txt`](requirements.txt) file. We will install the packages using `pip install` command. You can also specify the OS packages needed using [`apt.txt`](apt.txt) file. We install these packages using `apt-get install` command. For more information on how you can configure the evaluation runtime, please refer [`RUNTIME.md`](docs/RUNTIME.md). ## 🚀 Submitting to AIcrowd ### `aicrowd.json` Your repository should have an `aicrowd.json` file with following fields: ```json { "challenge_id" : "the-neural-mmo-challenge", "grader_id" : "the-neural-mmo-challenge", "authors" : ["Your Name"], "description" : "Brief description for your submission" } ``` This file is used to identify your submission as a part of the Neural MMO Challenge. You must use the `challenge_id`, and `grader_id` as specified above. ### Configuring the submission repository ```bash git remote add aicrowd git@gitlab.aicrowd.com:/neural-mmo-starter-kit.git ``` **Note:** This needs to be done only once. This configuration will be saved in your repository for future use. ### Pushing the code to AIcrowd ```bash ./utils/submit.sh "some description" ``` If you want to submit without the helper script, please refer [`SUBMISSION.md`](docs/SUBMISSION.md). # 📝 Submission checklist - [x] **Accept the challenge rules**. You can do this by going to the [challenge overview page](https://www.aicrowd.com/challenges/the-neural-mmo-challenge) and clicking the "Participate" button. You only need to do this once. - [x] **Add your agent code** that implements the `NeuralMMOBaseAgent` class from `evaluator/base_agent`. Make sure your `agent_type` is filled. - [x] **Add your model checkpoints** (if any) to the repo. The `utils/submit.sh` will automatically detect large files and add them to git LFS. If you are using the script, please refer to [this post explaining how to add your models](https://discourse.aicrowd.com/t/how-to-upload-large-files-size-to-your-submission/2304). - [x] **Evaluate your agents locally** to know that they work as expected. - [x] **Update runtime configuration** using `requirements.txt`, `apt.txt` and/or `Dockerfile` as necessary. Please make sure that you specified the same package versions that you use locally on your machine. # 📎 Important links - 💪 Challenge information * [Challenge page](https://www.aicrowd.com/challenges/the-neural-mmo-challenge) * [Leaderboard](https://www.aicrowd.com/challenges/the-neural-mmo-challenge) - 🗣 Community * [Neural MMO discord server](https://discord.gg/N3cWFhsk8K) * [Challenge discussion forum](https://discourse.aicrowd.com/c/the-neural-mmo-challenge/) - 🎮 Neural MMO resources * [Neural MMO documentation](https://jsuarez5341.github.io) * [Neural MMO GitHub repository](https://github.com/jsuarez5341/neural-mmo) # ✨ Contributors - [Siddhartha Laghuvarapu](https://www.aicrowd.com/participants/siddhartha) - [Jyotish Poonganam](https://www.aicrowd.com/participants/jyotish) - [Joseph Suarez](https://www.aicrowd.com/participants/joseph_suarez) - [Dipam Chakraborty](https://www.aicrowd.com/participants/dipam) **Best of Luck** 🎉