eBlu Python Guide

Contents

1. Overview
2. Style Guide
3. Deliverables

Style Guide

String Formatting

• We are permanently on Python 3+ and will continue to enhance upon the latest versions of python supported in modern clouds such as Microsoft Azure and Google Cloud Platform.
• Whenever migrating from Python 2+ we like to utilize modern python styling such as string formatting:

  output_string = "Output strings to console"
  
  # Python 2
  print "{0}".format("Output string")
  
  # Python 3+
  print(f'{output_string}')
  

Static Typing

• Static typing is very much encouraged whenever delivering python features. It prevents complexity and assumptions, while keeping a grasp on neat and D.R.Y programming methodologies

  from typing import List, Dict
  
  # Statically typed variable example
  def typed_variable_output(output: List[Dict]):
      print(output)
  
  # won't throw an error
  typed_variable_output([{"key": 123}])
  
  # will throw and error before it even starts the function logic
  typed_variable_output([2.0, 0, 1.3])
  

• This extends to classes and python objects. Libraries such as dataclasses or pydantic are great options to manage common data classes and have even further data structure within our python deliverables.

Deliverables

• A few things must be present before any and all python deliverables are fully launched into production
  
  

Error Handling

• All code is functional in theory, but since we are all humans here reading this style guide we should always embrace failure and plan for it!
• Error handling can extend all kinds of methodologies and a whole guide can be spent on this topic, but the internet is a much better place to research your exact use case. That being said, try/catch, static typing, and data class modeling are great places to start. This also extends into the next topic of logging...

Logging

• Example Modules
  • google-cloud-logging
  • ebs-logging
  • Logging, and especially structured logging is very important in python, cloud development, but most importantly, data features due to the very nature of the development context.
  • Structured logging is the best method to bullet-proof our code into a service mesh environment. Logs can be aggregated and developed off of, as well as use for model training for program enhancement.
  • Reporting can easily overlay with our internal LogStreams template, directly available within BigQuery.

Unit Testing

• Again, embracing failure, we need to plan for failure. This is why we test, and develop logically unit testing to bullet-proof our code.
• Python has numerous testing libraries such as:
  • pytest
  • unittest (builtin)
  • flake8
  • ... so on
• Usually keeping it simple with the unittest library is enough, but more advanced features and code bases need versatility. So pick the library that best suites your use case and roll with it!

(Optional) Integration Testing

• If your feature requires multiple pieces to function as a cohesive feature, more than likely you will want to incorporate integration testing to ensure all those pieces function together BEFORE you ship or stage the feature.