Ray integration problem with 2.1.8
I'm having a problem using the new version in ray.
since you are setting the trains do Done and Removed in the same timestep you are setting them to DONE in https://gitlab.aicrowd.com/flatland/flatland/blob/master/flatland/envs/rail_env.py#L635
The get_observations function is never called for DONE agents. Since it only return values for DONE and ACTIVE agents, The first observation we get for agents with its done
as True is a None observation. This does not work with ray as you need a final observation that has done set to True.
Could I suggest splitting the Done check in line 525 and removing the agents there? That would give you something like
`
if agent.status is RailAgentStatus.DONE_REMOVED: # this agent has already completed...
return
if agent.status is RailAgentStatus.DONE: # this agent has already completed...
self._remove_agent_from_scene(agent)
return
`
and remove line 635. I think that could solve the problem and not change the states for anyone that does not use ray.