Commit 7ba0f38c authored by nilabha's avatar nilabha
Browse files

Merge branch 'flatland-3-updates' into 'flatland3-pettingzoo'

# Conflicts:
#   flatland/contrib/interface/flatland_env.py
#   flatland/contrib/requirements_training.txt
#   tests/test_pettingzoo_interface.py
parents 62e02366 a578d2a1
Pipeline #8492 failed with stages
in 6 minutes and 25 seconds
...@@ -119,3 +119,5 @@ test_save.dat ...@@ -119,3 +119,5 @@ test_save.dat
.visualizations .visualizations
playground/ playground/
**/tmp
**/TEMP
\ No newline at end of file
...@@ -10,6 +10,7 @@ image: themattrix/tox ...@@ -10,6 +10,7 @@ image: themattrix/tox
## - AWS_SECRET_ACCESS_KEY ## - AWS_SECRET_ACCESS_KEY
stages: stages:
- build_wheel
- tests - tests
- integration_testing - integration_testing
- profiling - profiling
...@@ -149,4 +150,18 @@ test_conda_setup: ...@@ -149,4 +150,18 @@ test_conda_setup:
script: script:
- xvfb-run bash getting_started/getting_started.sh - xvfb-run bash getting_started/getting_started.sh
build_wheel:
image: "python:3.7-slim"
stage: build_wheel
before_script:
- apt update
- apt install -y make
- pip install -r requirements_dev.txt
script:
- make dist
- export WHEEL_NAME="$( find dist -name 'flatland_rl*.whl' )"
- mv "${WHEEL_NAME}" "${WHEEL_NAME/-py2.py3-/-py3-}"
artifacts:
paths:
- dist/flatland_rl*.whl
expire_in: 2 mos
...@@ -5,20 +5,22 @@ ...@@ -5,20 +5,22 @@
<p style="text-align:center"> <p style="text-align:center">
<img alt="repository" src="https://gitlab.aicrowd.com/flatland/flatland/badges/master/pipeline.svg"> <img alt="repository" src="https://gitlab.aicrowd.com/flatland/flatland/badges/master/pipeline.svg">
<img alt="discord" src="https://gitlab.aicrowd.com/flatland/flatland/badges/master/coverage.svg"> <img alt="coverage" src="https://gitlab.aicrowd.com/flatland/flatland/badges/master/coverage.svg">
</p> </p>
Flatland is a open-source toolkit for developing and comparing Multi Agent Reinforcement Learning algorithms in little (or ridiculously large!) gridworlds. Flatland is a open-source toolkit for developing and comparing Multi Agent Reinforcement Learning algorithms in little (or ridiculously large!) gridworlds.
[The official documentation](http://flatland.aicrowd.com/) contains full details about the environment and problem statement [The official documentation](http://flatland.aicrowd.com/) contains full details about the environment and problem statement
Flatland is tested with Python 3.6 and 3.7 on modern versions of macOS, Linux and Windows. You may encounter problems with graphical rendering if you use WSL. Your [contribution is welcome](https://flatland.aicrowd.com/misc/contributing.html) if you can help with this! Flatland is tested with Python 3.6, 3.7 and 3.8 on modern versions of macOS, Linux and Windows. You may encounter problems with graphical rendering if you use WSL. Your [contribution is welcome](https://flatland.aicrowd.com/misc/contributing.html) if you can help with this!
🏆 Challenges 🏆 Challenges
--- ---
This library was developed specifically for the AIcrowd [Flatland challenges](http://flatland.aicrowd.com/research/top-challenge-solutions.html) in which we strongly encourage you to take part in! This library was developed specifically for the AIcrowd [Flatland challenges](http://flatland.aicrowd.com/research/top-challenge-solutions.html) in which we strongly encourage you to take part in!
- [Flatland 3 Challenge](https://www.aicrowd.com/challenges/flatland-3) - ONGOING!
- [AMLD 2021 Challenge](https://www.aicrowd.com/challenges/flatland)
- [NeurIPS 2020 Challenge](https://www.aicrowd.com/challenges/neurips-2020-flatland-challenge/) - [NeurIPS 2020 Challenge](https://www.aicrowd.com/challenges/neurips-2020-flatland-challenge/)
- [2019 Challenge](https://www.aicrowd.com/challenges/flatland-challenge) - [2019 Challenge](https://www.aicrowd.com/challenges/flatland-challenge)
...@@ -30,7 +32,7 @@ This library was developed specifically for the AIcrowd [Flatland challenges](ht ...@@ -30,7 +32,7 @@ This library was developed specifically for the AIcrowd [Flatland challenges](ht
Install [Anaconda](https://www.anaconda.com/distribution/) and create a new conda environment: Install [Anaconda](https://www.anaconda.com/distribution/) and create a new conda environment:
```console ```console
$ conda create python=3.6 --name flatland-rl $ conda create python=3.7 --name flatland-rl
$ conda activate flatland-rl $ conda activate flatland-rl
``` ```
...@@ -57,7 +59,7 @@ $ git clone git@gitlab.aicrowd.com:flatland/flatland.git ...@@ -57,7 +59,7 @@ $ git clone git@gitlab.aicrowd.com:flatland/flatland.git
Once you have a copy of the source, install it with: Once you have a copy of the source, install it with:
```console ```console
$ python setup.py install $ pip install -e .
``` ```
### Test installation ### Test installation
...@@ -77,7 +79,7 @@ python setup.py test ...@@ -77,7 +79,7 @@ python setup.py test
👥 Credits 👥 Credits
--- ---
This library was developed by [SBB](https://www.sbb.ch/en/), [Deutsche Bahn](https://www.deutschebahn.com/), [AIcrowd](https://www.aicrowd.com/) and [numerous contributors](http://flatland.aicrowd.com/misc/credits.html) and AIcrowd research fellows from the AIcrowd community. This library was developed by [SBB](https://www.sbb.ch/en/), [Deutsche Bahn](https://www.deutschebahn.com/), [SNCF](https://www.sncf.com/en), [AIcrowd](https://www.aicrowd.com/) and [numerous contributors](http://flatland.aicrowd.com/misc/credits.html) and AIcrowd research fellows from the AIcrowd community.
➕ Contributions ➕ Contributions
--- ---
...@@ -93,6 +95,7 @@ Please follow the [Contribution Guidelines](https://flatland.aicrowd.com/misc/co ...@@ -93,6 +95,7 @@ Please follow the [Contribution Guidelines](https://flatland.aicrowd.com/misc/co
🔗 Partners 🔗 Partners
--- ---
<a href="https://sbb.ch" target="_blank" style="margin-right:25px"><img src="https://i.imgur.com/OSCXtde.png" alt="SBB" width="200"/></a> <a href="https://sbb.ch" target="_blank" style="margin-right:30px"><img src="https://annpr2020.ch/wp-content/uploads/2020/06/SBB.png" alt="SBB" width="140"/></a>
<a href="https://www.deutschebahn.com/" target="_blank" style="margin-right:25px"><img src="https://i.imgur.com/pjTki15.png" alt="DB" width="200"/></a> <a href="https://www.deutschebahn.com/" target="_blank" style="margin-right:30px"><img src="https://i.imgur.com/pjTki15.png" alt="DB" width="140"/></a>
<a href="https://www.aicrowd.com" target="_blank"><img src="https://avatars1.githubusercontent.com/u/44522764?s=200&v=4" alt="AICROWD" width="200"/></a> <a href="https://www.sncf.com/en" target="_blank" style="margin-right:30px"><img src="https://iconape.com/wp-content/png_logo_vector/logo-sncf.png" alt="SNCF" width="140"/></a>
<a href="https://www.aicrowd.com" target="_blank"><img src="https://i.imgur.com/kBZQGI9.png" alt="AIcrowd" width="140"/></a>
...@@ -4,4 +4,4 @@ ...@@ -4,4 +4,4 @@
__author__ = """S.P. Mohanty""" __author__ = """S.P. Mohanty"""
__email__ = 'mohanty@aicrowd.com' __email__ = 'mohanty@aicrowd.com'
__version__ = '2.2.2' __version__ = '3.0.0rc1'
...@@ -9,8 +9,8 @@ import numpy as np ...@@ -9,8 +9,8 @@ import numpy as np
import redis import redis
from flatland.envs.rail_env import RailEnv from flatland.envs.rail_env import RailEnv
from flatland.envs.rail_generators import complex_rail_generator from flatland.envs.rail_generators import sparse_rail_generator
from flatland.envs.line_generators import complex_line_generator from flatland.envs.line_generators import sparse_line_generator
from flatland.evaluators.service import FlatlandRemoteEvaluationService from flatland.evaluators.service import FlatlandRemoteEvaluationService
from flatland.utils.rendertools import RenderTool from flatland.utils.rendertools import RenderTool
...@@ -18,35 +18,42 @@ from flatland.utils.rendertools import RenderTool ...@@ -18,35 +18,42 @@ from flatland.utils.rendertools import RenderTool
@click.command() @click.command()
def demo(args=None): def demo(args=None):
"""Demo script to check installation""" """Demo script to check installation"""
env = RailEnv(width=15, height=15, rail_generator=complex_rail_generator( env = RailEnv(
nr_start_goal=10, width=30,
nr_extra=1, height=30,
min_dist=8, rail_generator=sparse_rail_generator(
max_dist=99999), line_generator=complex_line_generator(), number_of_agents=5) max_num_cities=3,
grid_mode=False,
max_rails_between_cities=4,
max_rail_pairs_in_city=2,
seed=0
),
line_generator=sparse_line_generator(),
number_of_agents=5)
env._max_episode_steps = int(15 * (env.width + env.height)) env._max_episode_steps = int(15 * (env.width + env.height))
env_renderer = RenderTool(env) env_renderer = RenderTool(env)
while True: obs, info = env.reset()
obs, info = env.reset() _done = False
_done = False # Run a single episode here
# Run a single episode here step = 0
step = 0 while not _done:
while not _done: # Compute Action
# Compute Action _action = {}
_action = {} for _idx, _ in enumerate(env.agents):
for _idx, _ in enumerate(env.agents): _action[_idx] = np.random.randint(0, 5)
_action[_idx] = np.random.randint(0, 5) obs, all_rewards, done, _ = env.step(_action)
obs, all_rewards, done, _ = env.step(_action) _done = done['__all__']
_done = done['__all__'] step += 1
step += 1 env_renderer.render_env(
env_renderer.render_env( show=True,
show=True, frames=False,
frames=False, show_observations=False,
show_observations=False, show_predictions=False
show_predictions=False )
) time.sleep(0.1)
time.sleep(0.3)
return 0 return 0
......
...@@ -310,4 +310,4 @@ class RailEnvPersister(object): ...@@ -310,4 +310,4 @@ class RailEnvPersister(object):
self.height, self.width = self.rail.grid.shape self.height, self.width = self.rail.grid.shape
self.rail.height = self.height self.rail.height = self.height
self.rail.width = self.width self.rail.width = self.width
self.dones = dict.fromkeys(list(range(self.get_num_agents())) + ["__all__"], False) self.dones = dict.fromkeys(list(range(self.get_num_agents())) + ["__all__"], False)
\ No newline at end of file
...@@ -15,7 +15,7 @@ from flatland.core.grid.grid4 import Grid4TransitionsEnum, Grid4Transitions ...@@ -15,7 +15,7 @@ from flatland.core.grid.grid4 import Grid4TransitionsEnum, Grid4Transitions
from flatland.core.grid.grid4_utils import get_new_position from flatland.core.grid.grid4_utils import get_new_position
from flatland.core.grid.grid_utils import IntVector2D from flatland.core.grid.grid_utils import IntVector2D
from flatland.core.transition_map import GridTransitionMap from flatland.core.transition_map import GridTransitionMap
from flatland.envs.agent_utils import EnvAgent, RailAgentStatus from flatland.envs.agent_utils import Agent, EnvAgent, RailAgentStatus
from flatland.envs.distance_map import DistanceMap from flatland.envs.distance_map import DistanceMap
from flatland.envs.rail_env_action import RailEnvActions from flatland.envs.rail_env_action import RailEnvActions
...@@ -477,6 +477,34 @@ class RailEnv(Environment): ...@@ -477,6 +477,34 @@ class RailEnv(Environment):
return return
def _handle_end_reward(self, agent: EnvAgent) -> int:
'''
Handles end-of-episode reward for a particular agent.
Parameters
----------
agent : EnvAgent
'''
reward = None
# agent done? (arrival_time is not None)
if agent.status == RailAgentStatus.DONE or agent.status == RailAgentStatus.DONE_REMOVED:
# if agent arrived earlier or on time = 0
# if agent arrived later = -ve reward based on how late
reward = min(agent.latest_arrival - agent.arrival_time, 0)
# Agents not done (arrival_time is None)
else:
# CANCELLED check (never departed)
if (agent.status == RailAgentStatus.READY_TO_DEPART):
reward = -1 * self.cancellation_factor * \
(agent.get_travel_time_on_shortest_path(self.distance_map) + self.cancellation_time_buffer)
# Departed but never reached
if (agent.status == RailAgentStatus.ACTIVE):
reward = agent.get_current_delay(self._elapsed_steps, self.distance_map)
return reward
def step(self, action_dict_: Dict[int, RailEnvActions]): def step(self, action_dict_: Dict[int, RailEnvActions]):
""" """
Updates rewards for the agents at a step. Updates rewards for the agents at a step.
...@@ -565,27 +593,8 @@ class RailEnv(Environment): ...@@ -565,27 +593,8 @@ class RailEnv(Environment):
for i_agent, agent in enumerate(self.agents): for i_agent, agent in enumerate(self.agents):
# agent done? (arrival_time is not None) reward = self._handle_end_reward(agent)
if agent.status == RailAgentStatus.DONE or agent.status == RailAgentStatus.DONE_REMOVED: self.rewards_dict[i_agent] += reward
# if agent arrived earlier or on time = 0
# if agent arrived later = -ve reward based on how late
reward = min(agent.latest_arrival - agent.arrival_time, 0)
self.rewards_dict[i_agent] += reward
# Agents not done (arrival_time is None)
else:
# CANCELLED check (never departed)
if (agent.status == RailAgentStatus.READY_TO_DEPART):
reward = -1 * self.cancellation_factor * \
(agent.get_travel_time_on_shortest_path(self.distance_map) + 0) # 0 replaced with buffer
self.rewards_dict[i_agent] += reward
# Departed but never reached
if (agent.status == RailAgentStatus.ACTIVE):
reward = agent.get_current_delay(self._elapsed_steps, self.distance_map)
self.rewards_dict[i_agent] += reward
self.dones[i_agent] = True self.dones[i_agent] = True
......
...@@ -162,7 +162,7 @@ def sparse_rail_generator(*args, **kwargs): ...@@ -162,7 +162,7 @@ def sparse_rail_generator(*args, **kwargs):
class SparseRailGen(RailGen): class SparseRailGen(RailGen):
def __init__(self, max_num_cities: int = 5, grid_mode: bool = False, max_rails_between_cities: int = 4, def __init__(self, max_num_cities: int = 2, grid_mode: bool = False, max_rails_between_cities: int = 2,
max_rail_pairs_in_city: int = 2, seed=0) -> RailGenerator: max_rail_pairs_in_city: int = 2, seed=0) -> RailGenerator:
""" """
Generates railway networks with cities and inner city rails Generates railway networks with cities and inner city rails
......
[bumpversion] [bumpversion]
current_version = 2.2.2 current_version = 3.0.0rc1
commit = True commit = True
tag = True tag = True
......
...@@ -80,6 +80,6 @@ setup( ...@@ -80,6 +80,6 @@ setup(
test_suite='tests', test_suite='tests',
tests_require=test_requirements, tests_require=test_requirements,
url='https://gitlab.aicrowd.com/flatland/flatland', url='https://gitlab.aicrowd.com/flatland/flatland',
version='3.0.0', version='3.0.0rc1',
zip_safe=False, zip_safe=False,
) )
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment