README.md 3.19 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55
# Saving Recorded Video

We use `gym monitor` for video recording

To use human rendering with recording and uploading videos, update the env_cnfig as follows
```
env_config:
    render: human
```
This saves videos to the folder `flatland` in the same location as the running script.

To add a custom location, set the config `video_dir` under env_config

```
        env_config:
            render: human
            # For saving videos in custom folder and to wandb.
            # By default if not specified folder is flatland 
            video_dir: small_tree_video 
```

By default, the folder is cleared before saving the new videos. This is done as all videos with the pattern `*0.mp4` are used for uploading to wandb and we do not want to upload videos from older runs

This behavior can be changed using the config parameter `resume: True`

Also, note that 

When we process videos, we sort videos which are more recent (based on the latest modified time). Hence videos would appear in descending order in wandb under the section media

## Sample video uploaded to wandb
![](recording/wandbvideos.png)

## Video Folder Names
flatland-1.1250.0.0.11475.video000000.1590223601.6230516

Currently video naming convention is 
`flatland-{iteration}-{steps}.{percentage_completition_mean}.{episode_id}.{pid}.video000000.{last_modified_time}`

The first part comprising of `iteration,steps,percentage_completition_mean` are the env stats from when the first result occurs just immediately after the video was first created. Look at the section [TODO](#TODO) for more details regarding this choice.

The second part `{episode_id}.{pid}.video000000` is basically as per the convention used for video names in the openai.gym monitor. episode_id is the current worker's episode number, pid is the process ID for the process which saved the video recording. Often, the last frames are recorded as `*video000001.mp4` which we ignore and don't upload to wandb to reduce the number of videos uploaded to wandb.

We also finally save the last modified time as often there can be multiple versions of same video in different stages of recording.

# Evaluation
We can enable rendering only during evalaution and not in the normal training process. This will take the new config settings and videos will be saved. All videos will be saved in the `video_dir` or the default `flatland` folder. In the same wandb process we only resume videos and we don't overwrite any content in the same folder. Caution should be taken to manually delete the existing videos in the folder or specify a new `video_dir` folder in such cases.

# CAUTION
If we are running multiple wandb process, caution should be taken to avoid writing to the same folder. The recommended practice is to create a different `video_dir` for each run to avoid any conflicts and accidently deleting other run files.

# TODO
- Cannot match exact env details on which video was created and hence defaulting to the env details from when the video was first created. To help identify we must record the video file with the env iteration or/and steps etc. 
- Using the env details when the video was created may be useful when recording video during evaluation
 where we are more interested in the current training state