BUG: pkl files (test-env) are broken with latest flatland version
Local evaluation is broken. The provided dataset doesn't fit the version.
(1) pip install flatland-rl - U (2) download https://www.aicrowd.com/challenges/flatland-challenge/dataset_files
see also : https://gitlab.aicrowd.com/flatland/flatland/issues/291
(flatland-rl) adrian@K57261:~/Innovation/flatland-challenge-starter-kit$ pip show flatland-rl Name: flatland-rl Version: 2.1.10 Summary: Multi Agent Reinforcement Learning on Trains Home-page: https://gitlab.aicrowd.com/flatland/flatland Author: S.P. Mohanty Author-email: mohanty@aicrowd.com License: UNKNOWN Location: /home/adrian/miniconda3/envs/flatland-rl/lib/python3.6/site-packages Requires: importlib-metadata, timeout-decorator, importlib-resources, msgpack, gym, pyarrow, tox, Click, pytest-runner, crowdai-api, ushlex, msgpack-numpy, svgutils, pytest, matplotlib, Pillow, attrs, CairoSVG, six, xarray, numpy, recordtype Required-by:
ERROR TRACE:
(flatland-rl) adrian@K57261:~$ redis-server
(flatland-rl) adrian@K57261:~/Innovation/flatland-challenge-starter-kit$ flatland-evaluator --tests ./scratch/test-envs/ ['Test_6/Level_1.pkl', 'Test_9/Level_1.pkl', 'Test_6/Level_3.pkl', 'Test_4/Level_0.pkl', 'Test_4/Level_2.pkl', 'Test_3/Level_2.pkl', 'Test_5/Level_3.pkl', 'Test_8/Level_3.pkl', 'Test_7/Level_2.pkl', 'Test_3/Level_0.pkl', 'Test_2/Level_2.pkl', 'Test_2/Level_3.pkl', 'Test_3/Level_3.pkl', 'Test_5/Level_2.pkl', 'Test_1/Level_2.pkl', 'Test_8/Level_1.pkl', 'Test_0/Level_1.pkl', 'Test_1/Level_0.pkl', 'Test_0/Level_2.pkl', 'Test_8/Level_0.pkl', 'Test_6/Level_0.pkl', 'Test_1/Level_1.pkl', 'Test_4/Level_3.pkl', 'Test_9/Level_2.pkl', 'Test_0/Level_3.pkl', 'Test_7/Level_3.pkl', 'Test_2/Level_1.pkl', 'Test_9/Level_3.pkl', 'Test_2/Level_0.pkl', 'Test_5/Level_0.pkl', 'Test_4/Level_1.pkl', 'Test_0/Level_0.pkl', 'Test_5/Level_1.pkl', 'Test_7/Level_1.pkl', 'Test_6/Level_2.pkl', 'Test_7/Level_0.pkl', 'Test_8/Level_2.pkl', 'Test_3/Level_1.pkl', 'Test_1/Level_3.pkl', 'Test_9/Level_0.pkl'] Listening at : flatland-rl::FLATLAND_RL_SERVICE_ID::commands Evaluating : Test_6/Level_1.pkl Error : 'malfunction' Traceback (most recent call last): File "/home/adrian/miniconda3/envs/flatland-rl/lib/python3.6/site-packages/flatland/evaluators/service.py", line 637, in run self.handle_env_create(command) File "/home/adrian/miniconda3/envs/flatland-rl/lib/python3.6/site-packages/flatland/evaluators/service.py", line 341, in handle_env_create malfunction_generator_and_process_data=malfunction_from_file(test_env_file_path), File "/home/adrian/miniconda3/envs/flatland-rl/lib/python3.6/site-packages/flatland/envs/malfunction_generators.py", line 45, in malfunction_from_file data['malfunction'] = MalfunctionProcessData._make(data['malfunction']) KeyError: 'malfunction'
(flatland-rl) adrian@K57261:~/Innovation/flatland-challenge-starter-kit$ python run.py Traceback (most recent call last): File "run.py", line 73, in obs_builder_object=my_observation_builder File "/home/adrian/miniconda3/envs/flatland-rl/lib/python3.6/site-packages/flatland/evaluators/client.py", line 187, in env_create _response = self._remote_request(_request) File "/home/adrian/miniconda3/envs/flatland-rl/lib/python3.6/site-packages/flatland/evaluators/client.py", line 152, in _remote_request raise Exception(str(_response["payload"])) Exception: {'type': 'FLATLAND_RL.ERROR', 'payload': "'malfunction'"}