add multi nodes distributed test support (#1399)

* add multi nodes distributed test support

* fix bug in htc.py when keep_all_stages turn on

* remove package imported but unused in test.py

* reformat code in test.py

* support both cpu & gpu for gathering

* reformat

* clean code, add doc

* add docstring

* reformat doc string