3. Developer Guides

Contents:

• 3.1. Exception Handling in Data Lab Services
  • 3.1.1. Exceptions vs. return values
  • 3.1.2. Service Architecture Overview
  • 3.1.3. Server-side Code
    • 3.1.3.1. Service Return Codes
  • 3.1.4. Client-side Code
    • 3.1.4.1. Client-side exception handling
  • 3.1.5. Coding Style: Examples vs. Applications
• 3.2. Documenting Python APIs with Docstrings
  • 3.2.1. Basic Format of Docstrings
    • 3.2.1.1. Docstrings MUST be delimited by triple double quotes
    • 3.2.1.2. Docstrings SHOULD begin with """ and terminate with """ on its own line
    • 3.2.1.3. Docstrings of methods and functions SHOULD NOT be preceded or followed by a blank line
    • 3.2.1.4. Docstrings of classes SHOULD be followed, but not preceded, by a blank line
    • 3.2.1.5. Docstring content MUST be indented with the code’s scope
  • 3.2.2. ReStructuredText in Docstrings
    • 3.2.2.1. No space between headers and paragraphs
    • 3.2.2.2. Sections are restricted to the Numpydoc section set
    • 3.2.2.3. Simulate subsections with bold text
    • 3.2.2.4. Line Lengths
    • 3.2.2.5. Marking Up Parameter Names
  • 3.2.3. Numpydoc Sections in Docstrings
    • 3.2.3.1. Short Summary
    • 3.2.3.2. Deprecation Warning
    • 3.2.3.3. Extended Summary
    • 3.2.3.4. Parameters
      • 3.2.3.4.1. Describing Parameter Types
        • 3.2.3.4.1.1. Concrete types
        • 3.2.3.4.1.2. Choices
        • 3.2.3.4.1.3. Sequence types
        • 3.2.3.4.1.4. Dictionary types
        • 3.2.3.4.1.5. Array types
        • 3.2.3.4.1.6. Callable types
      • 3.2.3.4.2. Optional Parameters
      • 3.2.3.4.3. Shorthand
    • 3.2.3.5. Returns
      • 3.2.3.5.1. Basic example
      • 3.2.3.5.2. Dictionary return types
      • 3.2.3.5.3. Struct return types
      • 3.2.3.5.4. Naming return variables
    • 3.2.3.6. Yields
    • 3.2.3.7. Other Parameters
    • 3.2.3.8. Raises
    • 3.2.3.9. See Also
    • 3.2.3.10. Notes
    • 3.2.3.11. References
    • 3.2.3.12. Examples
  • 3.2.4. Documenting Modules
    • 3.2.4.1. Sections in Module Docstrings
    • 3.2.4.2. Placement of Module Docstrings
  • 3.2.5. Documenting Classes
    • 3.2.5.1. Sections in Class Docstrings
    • 3.2.5.2. Placement of Class Docstrings
    • 3.2.5.3. Examples of Class Docstrings
  • 3.2.6. Documenting Methods and Functions
    • 3.2.6.1. Sections in Method and Function Docstring Sections
    • 3.2.6.2. Placement of Module and Function Docstrings
    • 3.2.6.3. Dunder Methods
    • 3.2.6.4. Examples of Method and Function Docstrings
  • 3.2.7. Documenting Constants and Class Attributes
    • 3.2.7.1. Sections in Constant and Class Attribute Docstrings
    • 3.2.7.2. Placement of Constant and Class Attribute Docstrings
    • 3.2.7.3. Examples of Constant and Class Attribute Docstrings
      • 3.2.7.3.1. Minimal constant or attribute example
      • 3.2.7.3.2. Multi-section docstrings
      • 3.2.7.3.3. Attributes set in __init__ methods
  • 3.2.8. Documenting Class Properties
  • 3.2.9. Complete Example Module
  • 3.2.10. Acknowledgements
• 3.3. Python Performance Profiling
  • 3.3.1. Function-level profiling
  • 3.3.2. Line profiling
• 3.4. Data Lab Python Style Guide
  • 3.4.1. Introduction
    • 3.4.1.1. Stringency Levels
  • 3.4.2. 0. Python Version
    • 3.4.2.1. All Data Lab Python client-side code MUST work with Python 3
    • 3.4.2.2. Data Lab Python server-side code SHOULD work with Python 3
  • 3.4.3. 1. PEP 8 is the Baseline Coding Style
    • 3.4.3.1. Exceptions to PEP 8
    • 3.4.3.2. Code MAY be validated with flake8
      • 3.4.3.2.1. Flake8 installation
      • 3.4.3.2.2. Flake8 command line invocation
      • 3.4.3.2.3. Flake8 configuration files
    • 3.4.3.3. Lines that intentionally deviate from DM’s PEP 8 MUST include a noqa comment
    • 3.4.3.4. autopep8 MAY be used to fix PEP 8 compliance
  • 3.4.4. 2. Layout
    • 3.4.4.1. Service Configuration Information MUST be stored externally
    • 3.4.4.2. Client-side Configuration Information
    • 3.4.4.3. Packages SHOULD import only what is necessary
    • 3.4.4.4. Exportable Python packages MUST contain a standard license file
    • 3.4.4.5. Docstring and comment line length MUST be less than or equal to 79 columns
  • 3.4.5. 3. Whitespace
    • 3.4.5.1. The minimum number of parentheses needed for correctness and readability SHOULD be used
    • 3.4.5.2. Binary operators SHOULD be surrounded by a single space except for [*, /, **, //, %]
  • 3.4.6. 4. Comments
    • 3.4.6.1. Comments MUST always remain up-to-date with code changes
    • 3.4.6.2. Block comments SHOULD reference the code following them and SHOULD be indented to the same level
    • 3.4.6.3. To-do / FIXME comments SHOULD include a Jira issue key
  • 3.4.7. 5. Documentation Strings (docstrings)
    • 3.4.7.1. Docstrings SHOULD be written for all public modules, functions, classes, and methods
  • 3.4.8. 6. Naming Conventions
    • 3.4.8.1. Modules which contain class definitions SHOULD be named after the class name
    • 3.4.8.2. User-defined names SHOULD NOT shadow python built-in functions
    • 3.4.8.3. Names l (lowercase: el), O (uppercase: oh), I (uppercase: eye) MUST be avoided
    • 3.4.8.4. Always use cls for the first argument to metaclass instance methods
  • 3.4.9. 7. Source Files & Modules
    • 3.4.9.1. A Python source file name SHOULD be camelCase-with-leading-lowercase and ending in ‘.py’
    • 3.4.9.2. File Header Comments
    • 3.4.9.3. ASCII Encoding MUST be used for new code
    • 3.4.9.4. Standard code order SHOULD be followed
  • 3.4.10. 8. Classes
    • 3.4.10.1. super MAY be used to call parent class methods
      • 3.4.10.1.1. super() and Multiple Inheritance
  • 3.4.11. 9. Comparisons
    • 3.4.11.1. is and is not SHOULD only be used for determining if two variables point to same object
    • 3.4.11.2. is and is not SHOULD be used when comparing to None
  • 3.4.12. 10. Idiomatic Python
    • 3.4.12.1. A mutable object MUST NOT be used as a keyword argument default
    • 3.4.12.2. Context managers (with) SHOULD be used for resource allocation
    • 3.4.12.3. Avoid dict.keys() when iterating over keys or checking membership
    • 3.4.12.4. The subprocess module SHOULD be used to spawn processes
    • 3.4.12.5. lambda SHOULD NOT be used
    • 3.4.12.6. The set type SHOULD be used for unordered collections
    • 3.4.12.7. Iterators and generators SHOULD be used to iterate over large data sets efficiently
    • 3.4.12.8. if False: and if True: SHOULD NOT be used
    • 3.4.12.9. Python exceptions SHOULD be raised and checked in Python code
• 3.5. Python Unit Testing
  • 3.5.1. Introduction to unittest
  • 3.5.2. Supporting Pytest
  • 3.5.3. Testing Flask Applications
  • 3.5.4. Common Issues
    • 3.5.4.1. Testing global state
    • 3.5.4.2. Test Skipping and Expected Failures
  • 3.5.5. Enabling additional Pytest options: flake8
  • 3.5.6. Using a shared base class
• 3.6. ReStructuredText Sample
  • 3.6.1. Sectioning
  • 3.6.2. Directives
• 3.7. ReStructuredText Style Guide
  • 3.7.1. Sample
  • 3.7.2. Inline Text Styling
    • 3.7.2.1. Other semantic markup
  • 3.7.3. Lists
    • 3.7.3.1. Definition lists
  • 3.7.4. Sections
  • 3.7.5. Linking
    • 3.7.5.1. External links
    • 3.7.5.2. Internal links to labels
    • 3.7.5.3. Internal links to other pages
    • 3.7.5.4. Links to equations
    • 3.7.5.5. Links to code objects
      • 3.7.5.5.1. Links to Python objects
        • 3.7.5.5.1.1. Namespace resolution
      • 3.7.5.5.2. Customizing the link text
      • 3.7.5.5.3. Default domains
  • 3.7.6. Tables
  • 3.7.7. Images and Figures
    • 3.7.7.1. Plain images
    • 3.7.7.2. Figure directive
    • 3.7.7.3. Note on paths to image files
  • 3.7.8. Math
    • 3.7.8.1. Inline math
    • 3.7.8.2. Block math
      • 3.7.8.2.1. Referencing equations
      • 3.7.8.2.2. Multiple Equations
      • 3.7.8.2.3. Aligned Equations
  • 3.7.9. Source code
    • 3.7.9.1. Including source code examples from other files with literalinclude
    • 3.7.9.2. Lightweight syntax for Python code and sessions
  • 3.7.10. Command line prompts
  • 3.7.11. Footnotes
  • 3.7.12. Citations
  • 3.7.13. Comments in ReStructuredText
  • 3.7.14. RestructuredText Formatting Conventions
    • 3.7.14.1. Text wrapping
    • 3.7.14.2. Indentation
    • 3.7.14.3. Encoding and special characters
• 3.8. Page Title Example
  • 3.8.1. Section Heading
    • 3.8.1.1. Subsection Heading
      • 3.8.1.1.1. Sub-subsection Heading
        • 3.8.1.1.1.1. Sub-sub-subsection Heading