###########################
Data Lab Python Style Guide
###########################

.. note::
   This document was derived from the version 6.0 of the LSST/DM Python Coding
   Standard (https://github.com/lsst-dm/dm_dev_guide/blob/master/python/style.rst).
   Changes have been made to reflect the coding styles allowed within Data Lab
   that are to be used as the basis for code reviews.  External documents
   referenced in the original LSST/DM document have been partially imported as
   needed for clarity, or else now reference similarly modified Data Lab documents.
   Similar documents for coding style guides in other languages will be prepared.

.. contents::
   :depth: 4

.. _style-guide-py-version:


Introduction
============

Stringency Levels
-----------------

In this style guide we use RFC-2119-style vocabulary to rank the importance of
conforming to a specific recommendation.

REQUIRED
  The Rule is an absolute requirement of the specification.  Explicit approval
  to contravene the Rule must be sought by the developer before proceeding
  and the exception must be noted in code documentaation and unit tests.

MUST and SHALL
  Mean that there may exist valid reasons in particular circumstances to
  ignore a particular Rule, but the full implications must be understood and
  carefully weighed before choosing a different course.  Explicit approval
  to contravene the Rule must be sought by the developer before proceeding
  and the exception must be noted in code documentaation and unit tests.

SHOULD, RECOMMENDED and MAY
  There are valid reasons in particular circumstances to ignore a particular
  Rule. The developer is expected to personally consider the full implications
  before choosing a different course.

PROHIBITED
  The opposite of REQUIRED.

MUST NOT and SHALL NOT
  The opposites of MUST and SHALL.

SHOULD NOT, NOT RECOMMENDED and MAY NOT
  The opposites of SHOULD, RECOMMENDED and MAY.


0. Python Version
=================

.. _style-guide-py-version-py3:

All Data Lab Python client-side code MUST work with Python 3
------------------------------------------------------------

All Python *client-side* code (application interfaces and commandline tools)
written by Data Lab MUST be runnable using Python 3 (Py3).  Where possible,
support for Python 2 (Py2) SHOULD be provided, otherwise features that are
Py3-only should be clearly documented and the code structured so as not to
cripple an entire interface or application (e.g. isolate :command:`import`
statements or methods to separate files or with tests of
:command:`sys.version_info.major`).

The current baseline versions for client code are Python 3.6 and Python 2.7.

Data Lab Python server-side code SHOULD work with Python 3
-----------------------------------------------------------

Existing server-side Python code written by Data Lab may have package
dependencies that prevent it from running under Python 3, in which case 
the service can be run using the baseline version of Python 2.  A Python 3
migration plan for existing code SHOULD be developed and scheduled as part
of the normal development and release cycle.

All *new* service code MUST be written to be runnable under Python 3 and avoid
these Py2 dependency packages.  Where possible, support for Python 2 SHOULD
be provided but is not required if the service can be deployed using Python 3.

The current baseline versions for service code are Python 3.6 and Python 2.7.

.. _style-guide-py-pep8-baseline:

1. PEP 8 is the Baseline Coding Style
=====================================

Data Lab's Python Coding Style is based on the `PEP 8 Style Guide for Python
Code <https://www.python.org/dev/peps/pep-0008/>`_ with modifications and
exceptions specified in this document.

:pep:`8` is used throughout the Python community and should feel familiar
to Python developers.  Deviations from :pep:`8` style are intended to 
permit flexibility in personal coding styles so not all of the :pep:`8`
guidlines are enforced.  Additional guidelines are included in this document
to address specific requirements of code as needed.

.. _style-guide-py-ignored-errors:

Exceptions to PEP 8
-------------------

The following table summarizes all :pep:`8` guidelines that are
**not followed** by this style guide.  These exceptions are organized by
error codes that may be ignored by the flake8_ linter
(see :ref:`style-guide-py-flake8`).

E133
   Closing bracket is missing indentation.
   This `pycodestyle error`_ (via flake8_) is not part of :pep:`8`.

E211
   Whitespace before '('.
   See :ref:`style-guide-py-operator-whitespace`.

E221
   Multiple whitespaces before operator.
   See :ref:`style-guide-py-operator-whitespace`.

E223
   Tab before operator.
   See :ref:`style-guide-py-operator-whitespace`.

E226
   Missing whitespace around arithmetic operator.
   See :ref:`style-guide-py-operator-whitespace`.

E228
   Missing whitespace around bitwise or shift operator.
   See :ref:`style-guide-py-operator-whitespace`.

Maximum line length
   Line length MUST be less than or equal to 79 columns.

The style checker in ``pycodestyle`` also provides warnings that can be used
to request a specific style that is ambiguous in :pep:`8`.
These codes should be ignored to choose the Data Lab preferred style:

W504
   Line break after binary operator.
   Disabling this enables W503 that checks that line breaks do not occur
   before binary operators.

Additionally, packages should disable the following rules:

N802
   Function name should be lowercase.

N803
   Argument name should be lowercase.

N806
   Variable in function should be lowercase.

.. _pycodestyle error: http://pep8.readthedocs.io/en/latest/intro.html#error-codes

.. _style-guide-py-flake8:

Code MAY be validated with flake8
---------------------------------

The flake8_ tool may be used to validate Python source code against the
portion of :pep:`8` adopted by Data Lab.  Additionally, flake8_ statically
checks Python for code errors.  The separate `pep8-naming`_ plugin validates
names according to this style guide.

.. note::

   Flake8 only validates code against PEP 8 specifications.
   This style guide includes additional guidelines that *are not* automatically linted.

.. _flake8: https://flake8.readthedocs.io
.. _pep8-naming: http://pypi.python.org/pypi/pep8-naming

.. _style-guide-py-flake8-install:

Flake8 installation
^^^^^^^^^^^^^^^^^^^

Linters are installable with :command:`pip`:

.. code-block:: bash

   pip install flake8
   pip install pep8-naming

.. _style-guide-py-flake8-invoke:

Flake8 command line invocation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: bash

   flake8 --ignore=E133,E211,E221,E223,E226,E228 .

This command lints all Python files in the current directory.
Alternatively, individual files can be specified in place of ``.``.

The ignored error codes are :ref:`explained above <style-guide-py-ignored-errors>`.
N802, N803, and N806 can be added to this list for some packages.

.. _style-guide-py-flake8-config:

Flake8 configuration files
^^^^^^^^^^^^^^^^^^^^^^^^^^

:command:`flake8` can be invoked without arguments when a configuration file
is present.  This configuration, included in a :file:`setup.cfg` file at the
root of code repositories, is consistent with the style guide:

.. code-block:: ini

   [flake8]
   max-line-length = 79
   ignore = E133, E211, E221, E223, E226, E228, N802, N803, N806, W504
   exclude =
     bin,
     doc,
     lib,
     tests,
     **/*/__init__.py,
     **/*/__version__.py,
     **/*/version.py

The ``exclude`` field lists paths that are not usefully linted by
:command:`flake8` in Data Lab code repositories.  Auto-generated Python should
not be linted.  We also discourage linting :file:`__init__.py` modules due
to the abundance of :pep:`8` exceptions typically involved.

.. _style-guide-py-noqa:

Lines that intentionally deviate from DM's PEP 8 MUST include a ``noqa`` comment
--------------------------------------------------------------------------------

Lines of code may intentionally deviate from our application of PEP 8 because
of limitations in flake8_.  In such cases, authors must append a ``# noqa``
comment to the line that includes the specific error code being ignored.
`See the flake8 documentation for details <https://flake8.readthedocs.io/en/latest/user/ignoring-errors.html#in-line-ignoring-errors>`__ .
This prevents the line from triggering false flake8_ warnings to other
developers, while also linting unexpected errors.

For example, to import a module without using it (to build a namespace, as in
a :file:`__init__.py`):

.. code-block:: py

   from .module import AClass  # noqa: F401

.. seealso::

   - `flake8 error codes <https://flake8.readthedocs.io/en/latest/user/error-codes.html>`_
   - `pycodestyle error codes <https://pycodestyle.readthedocs.io/en/latest/intro.html#error-codes>`_
   - `pep8-naming error codes <https://github.com/PyCQA/pep8-naming#plugin-for-flake8>`_

.. _style-guide-py-autopep8:

autopep8 MAY be used to fix PEP 8 compliance
--------------------------------------------

Many :pep:`8` issues in existing code can be fixed with `autopep8`_ version
1.2 or newer:

.. code-block:: bash

   autopep8 . --in-place --recursive \
       --ignore E133,E211,E221,E223,E226,E228,N802,N803,N806,W504

The ``.`` specifies the current directory.
Together with ``--recursive``, the full tree of Python files will be processed
by :command:`autopep8`.  Alternatively, a single file can be specified in
place of ``.``.

:command:`autopep8`\ ʼs changes must always be validated before committing.

Style changes SHOULD be encapsulated in a distinct commit.

.. note::

   :command:`autopep8` only fixes PEP 8 issues and does not address other guidelines listed here.

.. _autopep8: https://pypi.python.org/pypi/autopep8

.. _style-guide-py-layout:

2. Layout
=========

.. seealso::

   :doc:`numpydoc` provides guidelines for the :ref:`layout of docstrings <py-docstring-basics>`.

.. _style-guide-py-svc-config-info:

Service Configuration Information MUST be stored externally
-----------------------------------------------------------

Service code that requires configuration information MUST use an external file
to store any data specific to the deployment of the service. This is intended
to allow code to be distributed from public repositories without releasing
local configuration details.

A default configuration (or path to a configuration file) SHOULD be specified
in the source file to prevent errors resulting from undefined values, however
these MUST be overridden by configuration data read from the external file
when the code is deployed if the default parameters are insufficient to define
a fully working service.  Service code MAY allow alternate configuration
files to be specified at runtime, e.g. such as when used in a development
environment.


.. _style-guide-py-client-config-info:

Client-side Configuration Information
-------------------------------------

Client-side code requiring configuration information MUST have the
configuration parameters set automatically by the package installer.
User-configurable data (e.g. Preferences) SHALL be allowed provided an
adequate description of each option is included with the software 
documentation.


.. _style-guide-py-necessary-imports:

Packages SHOULD import only what is necessary
---------------------------------------------

In order to minimize startup times, Data Lab code SHOULD import only the
necessary modules from 3rd-party packages. For example,

Yes:

.. code-block:: py

   from bigpackage.utils import convert

No:

.. code-block:: py

   import bigpackage


.. _style-guide-license:

Exportable Python packages MUST contain a standard license file
---------------------------------------------------------------

A Data Lab copyright and license file MUST be include with every code
repository or exportable package.  License information should be in a file
called ``LICENSE``, if separate copyright information is required it should
be in a file called ``COPYRIGHT``.

The text of the default Data Lab license is:

    ......``TBD``


.. _style-guide-py-docstring-line-length:

Docstring and comment line length MUST be less than or equal to 79 columns
--------------------------------------------------------------------------

Limit all docstring and comment lines to a maximum of 79 characters.

This differs from the `PEP 8 recommendation of 72 characters <https://www.python.org/dev/peps/pep-0008/#maximum-line-length>`_ and the `numpydoc recommendation of 75 characters <https://numpydoc.readthedocs.io/en/latest/format.html#docstring-standard>`_ but maintains readability and compatibility with default terminal widths while providing the maximum space.


.. _style-guide-py-whitespace:

3. Whitespace
=============

Follow the `PEP 8 whitespace style guidelines <https://www.python.org/dev/peps/pep-0008/#whitespace-in-expressions-and-statements>`_, with the following adjustments.

.. _style-guide-py-minimal-parens:

The minimum number of parentheses needed for correctness and readability SHOULD be used
---------------------------------------------------------------------------------------

Yes:

.. code-block:: py

   a = b(self.config.nSigmaToGrow*sigma + 0.5)

Less readable:

.. code-block:: py

   a = b((self.config.nSigmaToGrow*sigma) + 0.5)

.. _style-guide-py-operator-whitespace:

Binary operators SHOULD be surrounded by a single space except for [``*``, ``/``, ``**``, ``//``, ``%``\ ]
----------------------------------------------------------------------------------------------------------

Always surround these binary operators with a single space on either side; this helps the user see where one token ends and another begins:

- assignment (``=``),
- augmented assignment (``+=``, ``-=``, etc.),
- comparisons (``==``, ``<``, ``>``, ``!=``, ``<>``, ``<=``, ``>=``, ``in``, ``not in``, ``is``, ``is not``),
- Booleans (``and``, ``or``, ``not``).

Use spaces around these arithmetic operators when they improve readability:

- addition (``+``),
- subtraction (``-``)
- multiplication (``*``),
- division (``/``),

Never surround these binary arithmetic operators with whitespace:

- exponentiation (``**``),
- floor division (``//``),
- modulus (``%``). Note that a single space **must always** surround ``%`` when used for string formatting.

For example:

.. code-block:: py

   i = i + 1
   submitted += 1
   x = x*2 - 1
   hypot2 = x*x + y*y
   c = (a + b)*(a - b)
   print('Hello %s' % 'world!')

This deviates from PEP 8, which `allows whitespace around these arithmetic operators if they appear alone <https://www.python.org/dev/peps/pep-0008/#other-recommendations>`__.
Error codes: E226 and E228.


.. _style-guide-py-comments:

4. Comments
===========

Source code comments should follow `PEP 8's recommendations <https://www.python.org/dev/peps/pep-0008/#comments>`__ with the following additional requirements.

.. _style-guide-py-comment-consistency:

Comments MUST always remain up-to-date with code changes
--------------------------------------------------------

Comment blocks should be updated when code is modified.  Code reviews should
include consistency checks of the comments.

.. _style-guide-py-block-comment-indentation:

Block comments SHOULD reference the code following them and SHOULD be indented to the same level
------------------------------------------------------------------------------------------------

Block comments generally apply to some (or all) code that follows them, and
are indented to the same level as that code.  Each line of a block comment
starts with a ``#`` and a single space (unless it is indented text inside the
comment).

Paragraphs inside a block comment are separated by a line containing a
single ``#``.

To-do / FIXME comments SHOULD include a Jira issue key
------------------------------------------------------

If the commented code is a workaround for a known issue, this rule makes it
easier to find and remove the workaround once the issue has been resolved.
If the commented code itself is the problem, this rule ensures the issue
will be reported on Jira, making it more likely to be fixed in a timely
manner.  For example,

.. code-block:: py

   # TODO: workaround for DL-6789

.. code-block:: py

   # TODO: DL-12345 is triggered by this line


.. _style-guide-py-docstrings:

5. Documentation Strings (docstrings)
=====================================

Use **Numpydoc** to format the content of all docstrings.
The page :doc:`numpydoc` authoritatively describes this format.
Its guidelines should be treated as an extension of this Python Style Guide.

.. seealso::

   The :doc:`..//restructuredtext/style`---and the :ref:`rst-formatting-guidelines` section in particular---provide guidelines on reStructuredText in general.


.. _style-guide-py-docstring-public-api:

Docstrings SHOULD be written for all public modules, functions, classes, and methods
------------------------------------------------------------------------------------

Write docstrings for all public modules, functions, classes, and methods.
See :doc:`numpydoc`.

Docstrings are not necessary for non-public methods, but you should have a comment that describes what the method does.
This comment should appear after the ``def`` line.

.. _style-guide-py-naming:

6. Naming Conventions
=====================

We follow `PEP 8ʼs naming conventions <https://www.python.org/dev/peps/pep-0008/#naming-conventions>`_, with exceptions listed here.

All Data Lab Python source code is consistent with :pep:`8` naming in the
following ways:

- class names are ``CamelCase`` with leading uppercase,
- method names are ``camelCase`` with leading lowercase,
- module variables used as module global constants are ``UPPERCASE_WITH_UNDERSCORES``,

Packages developed prior to the this document may not fully adhere to :pep:`8`.
As part of the code-review process and normal development, :pep:`8` standards
will be introduced with future releases.

Naming consistency within a package MUST be maintained.  Within these stated
constraints new packages SHOULD use :pep:`8` naming conventions.

Names MAY be decorated with leading and/or trailing underscores.

.. _style-guide-py-naming-attributes:
.. _style-guide-py-naming-functions:

- Class Attribute Names SHOULD be camelCase with leading lowercase (Error code: N803).
- Module methods (free functions) SHOULD be camelCase with leading lowercase (Error code: N802)
- Compound variable names SHOULD be camelCase with leading lowercase (Error code: N806).


.. _style-guide-py-naming-class-modules:

Modules which contain class definitions SHOULD be named after the class name
----------------------------------------------------------------------------

Modules which contain class definitions should be named after the class name
(one module per class).


.. _style-guide-py-2-2:

User-defined names SHOULD NOT shadow python built-in functions
--------------------------------------------------------------

Names which shadow a python built-in function may cause confusion for readers
of the code.  Creating a more specific identifier is suggested to avoid
collisions.  For example, in the case of *filter*, ``filter_name`` may be
appropriate; for *filter objects*, something like ``filter_obj`` might be
appropriate.  In other cases, a leading or trailing underscore on the 
identifier may be sufficient to avoid confusion.


.. _style-guide-py-naming-ambiguous:

Names l (lowercase: el), O (uppercase: oh), I (uppercase: eye) MUST be avoided
------------------------------------------------------------------------------

Never use these characters as single character variable names:

- ``l`` (lowercase letter el),
- ``O`` (uppercase letter oh), or
- ``I`` (uppercase letter eye).

In some fonts, these characters are indistinguishable from the numerals one and zero.
When tempted to use ``l``, use ``L`` instead.

.. note::

  This matches the `PEP 8 standard <https://www.python.org/dev/peps/pep-0008/#names-to-avoid>`_ but is repeated here for emphasis.

.. _style-guide-py-naming-metaclasses:

Always use ``cls`` for the first argument to metaclass instance methods
-----------------------------------------------------------------------

For regular classes ``self`` is used, but for class methods and hence also
for metaclass instance methods, ``cls`` should be used instead.

.. note::

    This is consistent with the naming conventions in PEP 8 as indicated
    explicitly by `upstream <https://mail.python.org/pipermail/python-dev/2018-January/151986.html>`_.


.. _style-guide-py-files:

7. Source Files & Modules
=========================

.. _style-guide-py-file-name:

A Python source file name SHOULD be camelCase-with-leading-lowercase and ending in '.py'
----------------------------------------------------------------------------------------

A module containing a single class should be a
``camelCase``-with-leading-lowercase transliteration of the class's name.

Test files MUST have the form ``test_{description}.py`` for compatibility with
Pytest.  The name of a test case should be descriptive without the need for
a trailing numeral to distinguish one test case from another.

.. TODO consider refactoring tests into their own section


.. _style-guide-py-file-header-comments:

File Header Comments
--------------------

Each Python source file MUST include comments or a docstring at the top
of the file summarizing the contents or purpose of the file.  If the file
contains a service or programmatic interface of some kind, a one-line 
description of each interface/service method contained in the file SHOULD be
included as a summary of the interface or service.  Docstring comments for
each service/interface method are REQUIRED to contain a description of the
method sufficient to auto-generate user documentation.  Docstrings should
describe the purpose of the method, each of the arguments
and provide an example usage. 


.. _style-guide-py-file-encoding:

ASCII Encoding MUST be used for new code
----------------------------------------

Always use ASCII for new Python code.

- **Do not** include a coding comment (as described in  :pep:`263`) for ASCII files.

- Existing code using Latin-1 encoding (a.k.a. ISO-8859-1) is acceptable so long as it has a proper coding comment. All other code must be converted to ASCII or Latin-1 except for 3rd party packages used "as is."


.. _style-guide-py-file-order:

Standard code order SHOULD be followed
--------------------------------------

Within a module, follow the order:

#. Shebang line, ``#! /usr/bin/env python`` (only for executable scripts)
#. Module-level comments (such as the `license statement <https://github.com/lsst/templates/tree/master/file_templates/stack_license_py>`__)
#. Module-level docstring
#. ``from __future__ import ....`` statement, if present
#. ``__authors__ = ['First Last <email>, ....]`` statement, if present
#. ``__version__ = [yyyymmdd]`` statement, if present
#. ``__all__ = [...]`` statement, if present (usually just __init__.py file)
#. Imports
#. Private module variables (names start with underscore)
#. Private module functions and classes (names start with underscore)
#. Public module variables
#. Public functions and classes


.. _style-guide-py-classes:

8. Classes
==========

.. seealso:: `Designing for Inheritance <https://www.python.org/dev/peps/pep-0008/#designing-for-inheritance>`__ in :pep:`8` describes naming conventions related to public and private class APIs.

.. _style-guide-py-super:

``super`` MAY be used to call parent class methods
--------------------------------------------------

If you are overriding a method from a parent class, use :py:func:`super()` to
call the parent class's method.  For example:

.. code-block:: py

    class B(object):
        def method(self, arg):
            self.foo = arg

    class C(B):
        def method(self, arg):
            super().method(arg)
            do_something()

    C().method(arg)

Using :py:func:`super()` ensures a consistent Method Resolution Order, and
prevents inherited methods from being called multiple times.  In Python 3,
:py:func:`super()` does not require naming the class that it is part of,
making its use simpler and removing a maintenance issue.

.. seealso:: :pep:`3135` discusses the use of super() and should be consulted when writing code supporting both Python 2 and 3.


super() and Multiple Inheritance
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In the presence of multiple inheritance (two or more parents, e.g.
``class C(A, B)``), the trickiest issue with the use of :py:func:`super()` is
that the class author generally doesn't know a priori which overridden method
will be called in what order.  In particular, this means that the calling
signature (arguments) for all versions of a method must be compatible.  As
a result, there are a few argument-related caveats about the use of
:py:func:`super()` in multiple inheritance hierarchies:

* Only pass :py:func:`super()` the exact arguments you received.
* When you use it on methods whose acceptable arguments can be altered on a subclass via addition of more optional arguments, always accept ``*args``, ``**kwargs``, and call :py:func:`super()` like ``super().currentmethod(arg1, arg2, ..., *args, **kwargs)``. If you don’t do this, document that addition of optional arguments in subclasses is forbidden.
* Do not use positional arguments in ``__init__`` or ``__new__``.  Instead, use keyword args in the declarations, always call them using keywords, and always pass all keywords on, e.g. ``super().__init__(**kwargs)``.

To use :py:func:`super()` with multiple inheritance, all base classes in
Python's Method Resolution Order need to use :py:func:`super()`; otherwise
the calling chain gets interrupted.  If your class may be used in multiple
inheritance, ensure that all relevant classes use :py:func:`super()`
including documenting requirements for subclasses.

For more details, see the :py:func:`super() documentation <super()>`, the `astropy coding guide <http://docs.astropy.org/en/stable/development/codeguide.html#super-vs-direct-calling>`__, and `this article from Raymond Hettinger <https://rhettinger.wordpress.com/2011/05/26/super-considered-super/>`__.


.. _style-guide-py-comparisons:

9. Comparisons
==============

.. _style-guide-py-comp-is:

``is`` and ``is not`` SHOULD only be used for determining if two variables point to same object
-----------------------------------------------------------------------------------------------

Use ``is`` or ``is not`` only for the case that you need to know that two variables point to the exact same object.

To test for equality in *value*, use ``==`` or ``!=`` instead.

.. _style-guide-py-comp-none:

``is`` and ``is not`` SHOULD be used when comparing to ``None``
---------------------------------------------------------------

There are two reasons:

1. ``is None`` works with NumPy arrays, whereas ``== None`` does not;
2. ``is None`` is idiomatic.

This is also consistent with :pep:`8`, which `states
<https://www.python.org/dev/peps/pep-0008/#programming-recommendations>`__:

   Comparisons to singletons like ``None`` should always be done with ``is`` or ``is not``, never the equality operators.

For sequences, (``str``, ``list``, ``tuple``), use the fact that empty
sequences are ``False``.

Yes:

.. code-block:: py

   if not seq:
       pass

   if seq:
       pass

No:

.. code-block:: py

   if len(seq):
       pass

   if not len(seq):
       pass


.. _style-guide-py-idioms:

10. Idiomatic Python
====================

Strive to write idiomatic Python.  Writing Python with accepted patterns makes
your code easier for others to understand and often prevents bugs.

`Fluent Python <http://shop.oreilly.com/product/0636920032519.do>`_ by Luciano
Ramalho is an excellent guide to writing idiomatic Python.

Idiomatic Python also reduces technical debt.
For more information see the online book `Supporting Python 3 <http://python3porting.com/toc.html>`_ by Lennart Regebro.

.. _style-guide-py-pitfalls-mutables:

A mutable object MUST NOT be used as a keyword argument default
---------------------------------------------------------------

Never use a mutable object as default value for a keyword argument in a
function or method.

When a mutable is used as a default keyword argument, the default
*can* change from one call to another leading to unexpected behavior.
This issue can be avoided by only using immutable types as defaults.

For example, rather than provide an empty list as a default:

.. code-block:: py

   def proclist(alist=[]):
       pass

this function should create a new list in its internal scope:

.. code-block:: py

   def proclist(alist=None):
       if alist is None:
           alist = []

.. _style-guide-py-context-managers:

Context managers (``with``) SHOULD be used for resource allocation
------------------------------------------------------------------

Use the ``with`` statement to simplify resource allocation.

For example, to be sure a file will be closed when you are done with it:

.. code-block:: py

   with open('/data/foo.dat', 'r') as f:
       for line in f:
           pass

.. _style-guide-py-dict-keys:

Avoid ``dict.keys()`` when iterating over keys or checking membership
---------------------------------------------------------------------

For iterating over keys, iterate over the dictionary itself, e.g.:

.. code-block:: py

   for x in mydict:
       pass

To test for inclusion use ``in``:

.. code-block:: py

    if key in myDict:
        pass

This is preferred over :meth:`~dict.keys`. Use :meth:`~dict.keys` when
storing the keys for later access:

.. code-block:: py

    k = list(mydict.keys())

where ``list`` ensures that a view or iterator is not being retained.

.. note::

    When writing code that supports both Python 2 and Python 3, note that under Python 2 the :meth:`~dict.keys` return a `list` type, where under Python 3 it returns a `dict_keys` type.  Both types allow iteration over the key set, however under Python 3 the `dict_keys` type will not allow you to take an index.


.. _style-guide-py-subprocess:

The ``subprocess`` module SHOULD be used to spawn processes
-----------------------------------------------------------

Use the :py:mod:`subprocess` module to spawn processes.

.. _style-guide-py-lambda:

``lambda`` SHOULD NOT be used
-----------------------------

Avoid the use of `lambda <https://docs.python.org/3/reference/expressions.html#lambda>`__.
You can almost always write clearer code by using a named function or using the :py:mod:`functools` module to wrap a function.

.. _style-guide-py-set:

The ``set`` type SHOULD be used for unordered collections
---------------------------------------------------------

Use the :py:class:`set` type for unordered collections of objects.


.. _style-guide-py-generators:

Iterators and generators SHOULD be used to iterate over large data sets efficiently
-----------------------------------------------------------------------------------

Use iterators, generators (classes that act like iterators) and generator expressions (expressions that act like iterators) to iterate over large data sets efficiently.

.. _style-guide-py-disabled-code:

``if False:`` and ``if True:`` SHOULD NOT be used
-------------------------------------------------

Code must not be placed inside ``if False:`` or ``if True:`` blocks, nor
left commented out.  Instead, debugging code and alternative implementations
must be placed inside a "named" ``if`` statement.
Such blocks should have a comment describing why they are disabled.
They may have a comment describing the conditions under which said code can
be removed (like the completion of a ticket or a particular date).
For example, for code that will likely be removed in the future, once testing
is completed:

.. code-block:: py

    # Delete old_thing() and the below "if" statement once all unittests
    # are finished (Jira-1234).
    use_old_method = False
    if use_old_method:
        old_thing()
    else:
        new_thing()

It is often beneficial to lift such debugging flags into the method's keyword
arguments to allow users to decide which branch to run. For example:

.. code-block:: py

    def foo(x, debug_plots=False):
        do_thing()
        if debug_plots:
            plot_thing()


.. _style-guide-py-exceptions:

Python exceptions SHOULD be raised and checked in Python code
-------------------------------------------------------------

When raising an exception in Python code, consideration should be given to
`defining a module-specific exception`_ for increased precision.  Such an
exception SHOULD inherit from an appropriate standard Python exception.
If a module-specific exception is not used, then the appropriate standard
Python exception SHOULD be raised.

.. _defining a module-specific exception: https://docs.python.org/3/tutorial/errors.html#user-defined-exceptions

When writing an ``except`` clause, the exception type caught SHOULD be, in
order of preference, a module-specific or standard Python exception,