Format as RLLib and OpenAIGym. observation_space size and type should be forwarded to the observation builder.