CONTRIBUTING.rst 12.4 KB
 spmohanty committed Apr 03, 2019 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 .. highlight:: shell ============ Contributing ============ Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given. You can contribute in many ways: Types of Contributions ---------------------- Report Bugs ~~~~~~~~~~~  spmohanty committed Apr 03, 2019 18 Report bugs at https://gitlab.aicrowd.com/flatland/flatland/issues.  spmohanty committed Apr 03, 2019 19 20 21 22 23 24 25 26 27 28  If you are reporting a bug, please include: * Your operating system name and version. * Any details about your local setup that might be helpful in troubleshooting. * Detailed steps to reproduce the bug. Fix Bugs ~~~~~~~~  spmohanty committed Apr 03, 2019 29 Look through the Repository Issue Tracker for bugs. Anything tagged with "bug" and "help  spmohanty committed Apr 03, 2019 30 31 32 33 34 wanted" is open to whoever wants to implement it. Implement Features ~~~~~~~~~~~~~~~~~~  spmohanty committed Apr 03, 2019 35 Look through the Repository Issue Tracker for features. Anything tagged with "enhancement"  spmohanty committed Apr 03, 2019 36 37 38 39 40 41 42 and "help wanted" is open to whoever wants to implement it. Write Documentation ~~~~~~~~~~~~~~~~~~~ flatland could always use more documentation, whether as part of the official flatland docs, in docstrings, or even on the web in blog posts,  spmohanty committed Apr 03, 2019 43 articles, and such. A quick reference for writing good docstrings is available at : https://docs.python-guide.org/writing/documentation/#writing-docstrings  spmohanty committed Apr 03, 2019 44 45 46 47  Submit Feedback ~~~~~~~~~~~~~~~  spmohanty committed Apr 03, 2019 48 The best way to send feedback is to file an issue at https://gitlab.aicrowd.com/flatland/flatland/issues.  spmohanty committed Apr 03, 2019 49 50 51 52 53 54 55 56 57 58 59 60 61  If you are proposing a feature: * Explain in detail how it would work. * Keep the scope as narrow as possible, to make it easier to implement. * Remember that this is a volunteer-driven project, and that contributions are welcome :) Get Started! ------------ Ready to contribute? Here's how to set up flatland for local development.  spmohanty committed Apr 03, 2019 62 1. Fork the flatland repo on https://gitlab.aicrowd.com/flatland/flatland .  spmohanty committed Apr 03, 2019 63 64 2. Clone your fork locally::  spmohanty committed Apr 03, 2019 65  $git clone git@gitlab.aicrowd.com:flatland/flatland.git  spmohanty committed Apr 03, 2019 66   u214892 committed Jul 11, 2019 67 3. Install the software dependencies via Anaconda-3 or Miniconda-3. (This assumes you have Anaconda installed by following the instructions here _)  spmohanty committed Jul 03, 2019 68   u214892 committed Jul 11, 2019 69 70 71 $ conda install -c conda-forge tox-conda $conda install tox$ tox -v --recreate  u214892 committed Jun 20, 2019 72   u214892 committed Jul 11, 2019 73 74  This will create a virtual env you can then use.  u214892 committed Jul 08, 2019 75 76 77 78  These steps are performed if you run $getting_started/getting_started.bat/.sh  u214892 committed Jul 11, 2019 79  from Anaconda prompt.  u214892 committed Jul 08, 2019 80   u214892 committed Jun 20, 2019 81   spmohanty committed Apr 03, 2019 82 83 84 85 86 87 88 89 90 4. Create a branch for local development::$ git checkout -b name-of-your-bugfix-or-feature Now you can make your changes locally. 5. When you're done making changes, check that your changes pass flake8 and the tests, including testing other Python versions with tox::  u214892 committed May 21, 2019 91  $flake8 flatland tests examples benchmarks  spmohanty committed Apr 03, 2019 92 93 94 95 96 $ python setup.py test or py.test $tox To get flake8 and tox, just pip install them into your virtualenv.  spmohanty committed Apr 03, 2019 97 6. Commit your changes and push your branch to Gitlab::  spmohanty committed Apr 03, 2019 98 99 $ git add .  spmohanty committed Apr 03, 2019 100  $git commit -m "Addresses # Your detailed description of your changes."  spmohanty committed Apr 03, 2019 101 102 $ git push origin name-of-your-bugfix-or-feature  spmohanty committed Apr 03, 2019 103 7. Submit a merge request through the Gitlab repository website.  spmohanty committed Apr 03, 2019 104   spmohanty committed Apr 03, 2019 105 Merge Request Guidelines  spmohanty committed Apr 03, 2019 106 -------------------------  spmohanty committed Apr 03, 2019 107   spmohanty committed Apr 03, 2019 108 Before you submit a merge request, check that it meets these guidelines:  spmohanty committed Apr 03, 2019 109   spmohanty committed Apr 03, 2019 110 1. The merge request should include tests.  u214892 committed Nov 06, 2019 111 112 2. The could must be formatted (PyCharm) 3. If the merge request adds functionality, the docs should be updated. Put  spmohanty committed Apr 03, 2019 113 114  your new functionality into a function with a docstring, and add the feature to the list in README.rst.  u214892 committed Nov 06, 2019 115 4. The merge request should work for Python 3.6, 3.7 and for PyPy. Check  spmohanty committed Apr 03, 2019 116  https://gitlab.aicrowd.com/flatland/flatland/pipelines  spmohanty committed Apr 03, 2019 117  and make sure that the tests pass for all supported Python versions.  u214892 committed Nov 06, 2019 118  We force pipelines to be run successfully for merge requests to be merged.  u214892 committed Nov 06, 2019 119 5. Although we cannot enforce it technically, we ask for merge requests to be reviewed by at least one core member  u214892 committed Nov 06, 2019 120 121  in order to ensure that the Technical Guidelines below are respected and that the code is well tested:  u214892 committed Nov 06, 2019 122 5.1. The remarks from the review should be resolved/implemented and communicated using the 'discussions resolved':  u214892 committed Nov 06, 2019 123 124 125  .. image:: images/DiscussionsResolved.png  u214892 committed Nov 06, 2019 126 5.2. When a merge request is merged, source branches should be deleted and commits squashed:  u214892 committed Nov 06, 2019 127 128  .. image:: images/SourceBranchSquash.png  spmohanty committed Apr 03, 2019 129 130 131 132 133 134 135 136 137 138 139 140 141  Tips ---- To run a subset of tests:: $py.test tests.test_flatland Deploying --------- A reminder for the maintainers on how to deploy.  spmohanty committed Jul 03, 2019 142 Make sure all your changes are committed .  spmohanty committed Apr 03, 2019 143 144 145 146 147 148 Then run::$ bumpversion patch # possible: major / minor / patch $git push$ git push --tags  spmohanty committed Jul 03, 2019 149 TODO: Travis will then deploy to PyPI if tests pass. (To be configured properly by Mohanty)  u214892 committed Sep 25, 2019 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216  Local Evaluation ---------------- This document explains you how to locally evaluate your submissions before making an official submission to the competition. Requirements ~~~~~~~~~~~~ * **flatland-rl** : We expect that you have flatland-rl installed by following the instructions in [README.md](README.md). * **redis** : Additionally you will also need to have redis installed _ and **should have it running in the background.** Test Data ~~~~~~~~~ * **test env data** : You can download and untar the test-env-data , at a location of your choice, lets say /path/to/test-env-data/. After untarring the folder, the folder structure should look something like: .. code-block:: console . └── test-env-data ├── Test_0 │   ├── Level_0.pkl │   └── Level_1.pkl ├── Test_1 │   ├── Level_0.pkl │   └── Level_1.pkl ├.................. ├.................. ├── Test_8 │   ├── Level_0.pkl │   └── Level_1.pkl └── Test_9 ├── Level_0.pkl └── Level_1.pkl Evaluation Service ~~~~~~~~~~~~~~~~~~ * **start evaluation service** : Then you can start the evaluator by running : .. code-block:: console flatland-evaluator --tests /path/to/test-env-data/ RemoteClient ~~~~~~~~~~~~ * **run client** : Some sample submission code can be found in the starter-kit _, but before you can run your code locally using FlatlandRemoteClient, you will have to set the AICROWD_TESTS_FOLDER environment variable to the location where you previous untarred the folder with the test-env-data: .. code-block:: console export AICROWD_TESTS_FOLDER="/path/to/test-env-data/" # or on Windows : # # set AICROWD_TESTS_FOLDER "\path\to\test-env-data\" # and then finally run your code python run.py  u214892 committed Nov 06, 2019 217 218 219 Technical Guidelines --------------------  u214892 committed Nov 07, 2019 220 221 222 223 Clean Code ~~~~~~~~~~ Please adhere to the general Clean Code _ principles, for instance we write short and concise functions and use appropriate naming to ensure readability.  u214892 committed Nov 06, 2019 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244  Naming Conventions ~~~~~~~~~~~~~~~~~~ We use the pylint naming conventions: module_name, package_name, ClassName, method_name, ExceptionName, function_name, GLOBAL_CONSTANT_NAME, global_var_name, instance_var_name, function_parameter_name, local_var_name. numpydoc ~~~~~~~~ Docstrings should be formatted using numpydoc_. .. _numpydoc: https://numpydoc.readthedocs.io/en/latest/format.html Acessing resources ~~~~~~~~~~~~~~~~~~  u214892 committed Nov 07, 2019 245 We use importlib-resources _ to read from local files.  u214892 committed Nov 06, 2019 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273  Sample usages: .. code-block:: python from importlib_resources import path with path(package, resource) as file_in: new_grid = np.load(file_in) And: .. code-block:: python from importlib_resources import read_binary load_data = read_binary(package, resource) self.set_full_state_msg(load_data) Renders the scene into a image (screenshot) .. code-block:: python renderer.gl.save_image("filename.bmp") Type Hints ~~~~~~~~~~  u214892 committed Nov 07, 2019 274 275  We use Type Hints (PEP 484 _) for better readability and better IDE support.  u214892 committed Nov 06, 2019 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294  .. code-block:: python # This is how you declare the type of a variable type in Python 3.6 age: int = 1 # In Python 3.5 and earlier you can use a type comment instead # (equivalent to the previous definition) age = 1 # type: int # You don't need to initialize a variable to annotate it a: int # Ok (no value at runtime until assigned) # The latter is useful in conditional branches child: bool if age < 18: child = True else: child = False  u214892 committed Nov 07, 2019 295 Have a look at the Type Hints Cheat Sheet _ to get started with Type Hints.  u214892 committed Nov 06, 2019 296   u214892 committed Nov 07, 2019 297 Caveat: We discourage the usage of Type Aliases for structured data since its members remain unnamed (see Issue #284 _).  u214892 committed Nov 06, 2019 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315  .. code-block:: python # Discouraged: Type Alias with unnamed members Tuple[int, int] # Better: use NamedTuple from typing import NamedTuple Position = NamedTuple('Position', [ ('r', int), ('c', int) ] NamedTuple ~~~~~~~~~~  u214892 committed Nov 07, 2019 316   u214892 committed Nov 06, 2019 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 For structured data containers for which we do not write methods that have to ensure some (class) invariant over multiple members, we use NamedTuples instead of plain Dicts to ensure better readability by .. code-block:: python from typing import NamedTuple RailEnvNextAction = NamedTuple('RailEnvNextAction', [ ('action', RailEnvActions), ('next_position', RailEnvGridPos), ('next_direction', Grid4TransitionsEnum) ]) Members of NamedTuple can then be accessed through . instead of ['']. Class Attributes ~~~~~~~~~~~~~~~~  u214892 committed Nov 07, 2019 335   u214892 committed Nov 06, 2019 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 We use classes for data structures if we need to write methods that ensure (class) invariants over multiple members. We use the attrs_ class decorator and a way to declaratively define the attributes on that class: .. code-block:: python @attrs class Replay(object): position = attrib(type=Tuple[int, int]) .. _attrs: https://github.com/python-attrs/attrs Abstract Base Classes ~~~~~~~~~~~~~~~~~~~~~ We use the abc_ class decorator and a way to declaratively define the attributes on that class: .. code-block:: python # abc_base.py import abc class PluginBase(metaclass=abc.ABCMeta): @abc.abstractmethod def load(self, input): """Retrieve data from the input source and return an object. """ @abc.abstractmethod def save(self, output, data): """Save the data object to the output."""  u214892 committed Nov 06, 2019 369 370 371   u214892 committed Nov 06, 2019 372 And then  u214892 committed Nov 06, 2019 373   u214892 committed Nov 06, 2019 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402  .. code-block:: python # abc_subclass.py import abc from abc_base import PluginBase class SubclassImplementation(PluginBase): def load(self, input): return input.read() def save(self, output, data): return output.write(data) if __name__ == '__main__': print('Subclass:', issubclass(SubclassImplementation, PluginBase)) print('Instance:', isinstance(SubclassImplementation(), PluginBase)) .. _abc: https://pymotw.com/3/abc/ Currying ~~~~~~~~  u214892 committed Nov 07, 2019 403 We discourage currying to encapsulate state since we often want the stateful object to have multiple methods (but the curried function only has its one and only interface).  u214892 committed Nov 06, 2019 404   u214892 committed Nov 07, 2019 405 Thus, we should refactor our generators and use classes instead (see Issue #283 _).  u214892 committed Nov 06, 2019 406 407 408 409 410 411 412 413 414 415 416 417 418  .. code-block:: python # Type Alias RailGeneratorProduct = Tuple[GridTransitionMap, Optional[Dict]] RailGenerator = Callable[[int, int, int, int], RailGeneratorProduct] # Currying: a function that returns a confectioned function with internal state def complex_rail_generator(nr_start_goal=1, nr_extra=100, min_dist=20, max_dist=99999, seed=1) -> RailGenerator:  u214892 committed Nov 08, 2019 419