CONTRIBUTING.rst 12.6 KB
Newer Older
spmohanty's avatar
spmohanty committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
.. highlight:: shell

============
Contributing
============

Contributions are welcome, and they are greatly appreciated! Every little bit
helps, and credit will always be given.

You can contribute in many ways:

Types of Contributions
----------------------

Report Bugs
~~~~~~~~~~~

spmohanty's avatar
spmohanty committed
18
Report bugs at https://gitlab.aicrowd.com/flatland/flatland/issues.
spmohanty's avatar
spmohanty committed
19
20
21
22
23
24
25
26
27
28

If you are reporting a bug, please include:

* Your operating system name and version.
* Any details about your local setup that might be helpful in troubleshooting.
* Detailed steps to reproduce the bug.

Fix Bugs
~~~~~~~~

spmohanty's avatar
spmohanty committed
29
Look through the Repository Issue Tracker for bugs. Anything tagged with "bug" and "help
spmohanty's avatar
spmohanty committed
30
31
32
33
34
wanted" is open to whoever wants to implement it.

Implement Features
~~~~~~~~~~~~~~~~~~

spmohanty's avatar
spmohanty committed
35
Look through the Repository Issue Tracker for features. Anything tagged with "enhancement"
spmohanty's avatar
spmohanty committed
36
37
38
39
40
41
42
and "help wanted" is open to whoever wants to implement it.

Write Documentation
~~~~~~~~~~~~~~~~~~~

flatland could always use more documentation, whether as part of the
official flatland docs, in docstrings, or even on the web in blog posts,
spmohanty's avatar
spmohanty committed
43
articles, and such. A quick reference for writing good docstrings is available at : https://docs.python-guide.org/writing/documentation/#writing-docstrings
spmohanty's avatar
spmohanty committed
44
45
46
47

Submit Feedback
~~~~~~~~~~~~~~~

spmohanty's avatar
spmohanty committed
48
The best way to send feedback is to file an issue at https://gitlab.aicrowd.com/flatland/flatland/issues.
spmohanty's avatar
spmohanty committed
49
50
51
52
53
54
55
56
57
58
59
60
61

If you are proposing a feature:

* Explain in detail how it would work.
* Keep the scope as narrow as possible, to make it easier to implement.
* Remember that this is a volunteer-driven project, and that contributions
  are welcome :)

Get Started!
------------

Ready to contribute? Here's how to set up `flatland` for local development.

spmohanty's avatar
spmohanty committed
62
1. Fork the `flatland` repo on https://gitlab.aicrowd.com/flatland/flatland .
spmohanty's avatar
spmohanty committed
63
64
2. Clone your fork locally::

spmohanty's avatar
spmohanty committed
65
    $ git clone git@gitlab.aicrowd.com:flatland/flatland.git
spmohanty's avatar
spmohanty committed
66

67
3. Install the software dependencies via Anaconda-3 or Miniconda-3. (This assumes you have Anaconda installed by following the instructions `here <https://www.anaconda.com/distribution>`_)
68

69
70
71
    $ conda install -c conda-forge tox-conda
    $ conda install tox
    $ tox -v --recreate
72

73
74
    This will create a virtual env you can then use.

75
76
77
78
    These steps are performed if you run

    $ getting_started/getting_started.bat/.sh

79
    from Anaconda prompt.
80

81

spmohanty's avatar
spmohanty committed
82
83
84
85
86
87
88
89
90
4. Create a branch for local development::

    $ git checkout -b name-of-your-bugfix-or-feature

   Now you can make your changes locally.

5. When you're done making changes, check that your changes pass flake8 and the
   tests, including testing other Python versions with tox::

u214892's avatar
u214892 committed
91
    $ flake8 flatland tests examples benchmarks
spmohanty's avatar
spmohanty committed
92
93
94
95
96
    $ python setup.py test or py.test
    $ tox

   To get flake8 and tox, just pip install them into your virtualenv.

spmohanty's avatar
spmohanty committed
97
6. Commit your changes and push your branch to Gitlab::
spmohanty's avatar
spmohanty committed
98
99

    $ git add .
spmohanty's avatar
spmohanty committed
100
    $ git commit -m "Addresses #<issue-number> Your detailed description of your changes."
spmohanty's avatar
spmohanty committed
101
102
    $ git push origin name-of-your-bugfix-or-feature

spmohanty's avatar
spmohanty committed
103
7. Submit a merge request through the Gitlab repository website.
spmohanty's avatar
spmohanty committed
104

spmohanty's avatar
spmohanty committed
105
Merge Request Guidelines
spmohanty's avatar
spmohanty committed
106
-------------------------
spmohanty's avatar
spmohanty committed
107

spmohanty's avatar
spmohanty committed
108
Before you submit a merge request, check that it meets these guidelines:
spmohanty's avatar
spmohanty committed
109

spmohanty's avatar
spmohanty committed
110
1. The merge request should include tests.
u214892's avatar
u214892 committed
111
2. The code must be formatted (PyCharm)
u214892's avatar
u214892 committed
112
3. If the merge request adds functionality, the docs should be updated. Put
spmohanty's avatar
spmohanty committed
113
114
   your new functionality into a function with a docstring, and add the
   feature to the list in README.rst.
u214892's avatar
u214892 committed
115
4. The merge request should work for Python 3.6, 3.7 and for PyPy. Check
spmohanty's avatar
spmohanty committed
116
   https://gitlab.aicrowd.com/flatland/flatland/pipelines
spmohanty's avatar
spmohanty committed
117
   and make sure that the tests pass for all supported Python versions.
u214892's avatar
u214892 committed
118
   We force pipelines to be run successfully for merge requests to be merged.
u214892's avatar
u214892 committed
119
5. Although we cannot enforce it technically, we ask for merge requests to be reviewed by at least one core member
u214892's avatar
u214892 committed
120
121
   in order to ensure that the Technical Guidelines below are respected and that the code is well tested:

u214892's avatar
u214892 committed
122
5.1.  The remarks from the review should be resolved/implemented and communicated using the 'discussions resolved':
u214892's avatar
u214892 committed
123
124
125

.. image:: images/DiscussionsResolved.png

u214892's avatar
u214892 committed
126
5.2.  When a merge request is merged, source branches should be deleted and commits squashed:
u214892's avatar
u214892 committed
127
128

.. image:: images/SourceBranchSquash.png
spmohanty's avatar
spmohanty committed
129
130
131
132
133
134
135
136
137
138
139
140
141

Tips
----

To run a subset of tests::

$ py.test tests.test_flatland


Deploying
---------

A reminder for the maintainers on how to deploy.
142
Make sure all your changes are committed .
spmohanty's avatar
spmohanty committed
143
144
145
146
147
148
Then run::

$ bumpversion patch # possible: major / minor / patch
$ git push
$ git push --tags

149
TODO: Travis will then deploy to PyPI if tests pass. (To be configured properly by Mohanty)
u214892's avatar
u214892 committed
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216


Local Evaluation
----------------

This document explains you how to locally evaluate your submissions before making
an official submission to the competition.

Requirements
~~~~~~~~~~~~

* **flatland-rl** : We expect that you have `flatland-rl` installed by following the instructions in  [README.md](README.md).

* **redis** : Additionally you will also need to have  `redis installed <https://redis.io/topics/quickstart>`_ and **should have it running in the background.**

Test Data
~~~~~~~~~

* **test env data** : You can `download and untar the test-env-data <https://www.aicrowd.com/challenges/flatland-challenge/dataset_files>`, at a location of your choice, lets say `/path/to/test-env-data/`. After untarring the folder, the folder structure should look something like:


.. code-block:: console

    .
    └── test-env-data
        ├── Test_0
        │   ├── Level_0.pkl
        │   └── Level_1.pkl
        ├── Test_1
        │   ├── Level_0.pkl
        │   └── Level_1.pkl
        ├..................
        ├..................
        ├── Test_8
        │   ├── Level_0.pkl
        │   └── Level_1.pkl
        └── Test_9
            ├── Level_0.pkl
            └── Level_1.pkl

Evaluation Service
~~~~~~~~~~~~~~~~~~

* **start evaluation service** : Then you can start the evaluator by running :

.. code-block:: console

    flatland-evaluator --tests /path/to/test-env-data/

RemoteClient
~~~~~~~~~~~~

* **run client** : Some `sample submission code can be found in the starter-kit <https://github.com/AIcrowd/flatland-challenge-starter-kit/>`_, but before you can run your code locally using `FlatlandRemoteClient`, you will have to set the `AICROWD_TESTS_FOLDER` environment variable to the location where you previous untarred the folder with `the test-env-data`:


.. code-block:: console

    export AICROWD_TESTS_FOLDER="/path/to/test-env-data/"

    # or on Windows :
    #
    # set AICROWD_TESTS_FOLDER "\path\to\test-env-data\"

    # and then finally run your code
    python run.py


u214892's avatar
u214892 committed
217
218
219
Technical Guidelines
--------------------

220
221
222
223
Clean Code
~~~~~~~~~~
Please adhere to the general `Clean Code <https://www.planetgeek.ch/wp-content/uploads/2014/11/Clean-Code-V2.4.pdf>`_ principles,
for instance we write short and concise functions and use appropriate naming to ensure readability.
u214892's avatar
u214892 committed
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244

Naming Conventions
~~~~~~~~~~~~~~~~~~

We use the pylint naming conventions:

`module_name`, `package_name`, `ClassName`, `method_name`, `ExceptionName`, `function_name`, `GLOBAL_CONSTANT_NAME`, `global_var_name`, `instance_var_name`, `function_parameter_name`, `local_var_name`.


numpydoc
~~~~~~~~

Docstrings should be formatted using numpydoc_.


.. _numpydoc: https://numpydoc.readthedocs.io/en/latest/format.html


Acessing resources
~~~~~~~~~~~~~~~~~~

245
We use `importlib-resources <https://importlib-resources.readthedocs.io/en/latest/>`_ to read from local files.
u214892's avatar
u214892 committed
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
    Sample usages:

    .. code-block:: python

        from importlib_resources import path

        with path(package, resource) as file_in:
            new_grid = np.load(file_in)

    And:

    .. code-block:: python

        from importlib_resources import read_binary

        load_data = read_binary(package, resource)
        self.set_full_state_msg(load_data)



    Renders the scene into a image (screenshot)

    .. code-block:: python

        renderer.gl.save_image("filename.bmp")

Type Hints
~~~~~~~~~~
274
275

We use Type Hints (`PEP 484 <https://www.python.org/dev/peps/pep-0484/>`_) for better readability and better IDE support.
u214892's avatar
u214892 committed
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294

    .. code-block:: python
        # This is how you declare the type of a variable type in Python 3.6
        age: int = 1

        # In Python 3.5 and earlier you can use a type comment instead
        # (equivalent to the previous definition)
        age = 1  # type: int

        # You don't need to initialize a variable to annotate it
        a: int  # Ok (no value at runtime until assigned)

        # The latter is useful in conditional branches
        child: bool
        if age < 18:
            child = True
        else:
            child = False

295
Have a look at the `Type Hints Cheat Sheet <https://mypy.readthedocs.io/en/latest/cheat_sheet_py3.html>`_ to get started with Type Hints.
u214892's avatar
u214892 committed
296

297
Caveat: We discourage the usage of Type Aliases for structured data since its members remain unnamed (see `Issue #284 <https://gitlab.aicrowd.com/flatland/flatland/issues/284/>`_).
u214892's avatar
u214892 committed
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315

    .. code-block:: python
        # Discouraged: Type Alias with unnamed members
        Tuple[int, int]

        # Better: use NamedTuple
        from typing import NamedTuple

        Position = NamedTuple('Position',
            [
                ('r', int),
                ('c', int)
            ]



NamedTuple
~~~~~~~~~~
u214892's avatar
u214892 committed
316
317
For structured data containers for which we do not write additional methods, we use
`NamedTuple` instead of plain `Dict` to ensure better readability by
u214892's avatar
u214892 committed
318
319
320
321
322
323
324
325
326
327
328
329
330

    .. code-block:: python
        from typing import NamedTuple

        RailEnvNextAction = NamedTuple('RailEnvNextAction',
            [
                ('action', RailEnvActions),
                ('next_position', RailEnvGridPos),
                ('next_direction', Grid4TransitionsEnum)
            ])

Members of NamedTuple can then be accessed through `.<member>` instead of `['<key>']`.

u214892's avatar
u214892 committed
331
332
333
334
If we have to ensure some (class) invariant over multiple members
(for instance, `o.A` always changes at the same time as `o.B`),
then we should uses classes instead, see the next section.

u214892's avatar
u214892 committed
335
336
Class Attributes
~~~~~~~~~~~~~~~~
337

u214892's avatar
u214892 committed
338
339
We use classes for data structures if we need to write methods that ensure (class) invariants over multiple members,
for instance, `o.A` always changes at the same time as `o.B`.
u214892's avatar
u214892 committed
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
We use the attrs_ class decorator and a way to declaratively define the attributes on that class:

    .. code-block:: python
        @attrs
        class Replay(object):
            position = attrib(type=Tuple[int, int])

.. _attrs: https://github.com/python-attrs/attrs


Abstract Base Classes
~~~~~~~~~~~~~~~~~~~~~
We use the abc_ class decorator and a way to declaratively define the attributes on that class:

    .. code-block:: python
        # abc_base.py

        import abc


        class PluginBase(metaclass=abc.ABCMeta):

            @abc.abstractmethod
            def load(self, input):
                """Retrieve data from the input source
                and return an object.
                """

            @abc.abstractmethod
            def save(self, output, data):
                """Save the data object to the output."""

u214892's avatar
u214892 committed
372
373
374



u214892's avatar
u214892 committed
375
And then
u214892's avatar
u214892 committed
376

u214892's avatar
u214892 committed
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
    .. code-block:: python

        # abc_subclass.py

        import abc
        from abc_base import PluginBase


        class SubclassImplementation(PluginBase):

            def load(self, input):
                return input.read()

            def save(self, output, data):
                return output.write(data)


        if __name__ == '__main__':
            print('Subclass:', issubclass(SubclassImplementation,
                                          PluginBase))
            print('Instance:', isinstance(SubclassImplementation(),
                                          PluginBase))

.. _abc: https://pymotw.com/3/abc/



Currying
~~~~~~~~
u214892's avatar
u214892 committed
406
407
We discourage currying to encapsulate state since we often want the stateful object to have multiple methods
(but the curried function has only its signature and abusing params to switch behaviour is not very readable).
u214892's avatar
u214892 committed
408

409
Thus, we should refactor our generators and use classes instead (see `Issue #283 <https://gitlab.aicrowd.com/flatland/flatland/issues/283>`_).
u214892's avatar
u214892 committed
410
411
412
413
414
415
416
417
418
419
420
421
422

    .. code-block:: python
        # Type Alias
        RailGeneratorProduct = Tuple[GridTransitionMap, Optional[Dict]]
        RailGenerator = Callable[[int, int, int, int], RailGeneratorProduct]

        # Currying: a function that returns a confectioned function with internal state
        def complex_rail_generator(nr_start_goal=1,
                                   nr_extra=100,
                                   min_dist=20,
                                   max_dist=99999,
                                   seed=1) -> RailGenerator:

u214892's avatar
u214892 committed
423