Cleaning the Codebase

Resources:

• https://pre-commit.com/

Whether it's work, a date, or a promise to a friend we think before we commit then why on our projects we commit recklessly and make our project a nightmare for someone new? In Python, we have pep-8 guidelines that most of us want to follow and may even keep in our mind but why compete with a computer in remembering? To make our development process easier, we can make use of the pre-commit hooks that will check our code for necessary formatting, pep8 guidelines enforcement, security checks, and much more.

Adding a code formator e.g. Black
Black formats our code to have a consistent style. Black formatting rules are a superset of Python. Let's see a very simple example.

#non black code
def myfunc(product_id, order_id):
    product = get_object(Product,   id = product_id, something = something, something_else = something_else, order_id   = order_id)

after black formats our code

# after using black
def myfunc(product_id, order_id):
    product = get_object(
        Product,
        id=product_id,
        something=something,
        something_else=something_else,
        order_id=order_id,
    )

Notice how it tackled whitespaces after the '=' operator and line length > 79. We will not be installing black individually, Instead, we will keep it as a part of our pre-commit pipeline. For that first, we need to install pre-commit. Let's modify our requiremets.txt file and install the requirements using pip install -r requirements.txt

pre-commit==3.3.3

Now, we need to add a configuration file in which we will specify all things we want in our pre-commit pipeline. For this, we need to create a .pre-commit-config.yaml file.

fastapi-blog/
├─.git/
├─.gitignore
├─.pre-commit-config.yaml  #new
├─setup.cfg                #new
├─backend/
└─env/

Put the following as the contents of this file and save. This is a bare minimum configuration.

repos:
-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v2.3.0
    hooks:
    -   id: check-yaml
    -   id: end-of-file-fixer
    -   id: trailing-whitespace
    
-   repo: https://github.com/psf/black
    rev: 22.10.0
    hooks:
    -   id: black

Run pre-commit install to set up git hooks. You should see this message "pre-commit installed at .git\hooks\pre-commit". Now, every time you try to make a git commit black will reformat your code.

Adding a code enforcement linter e.g. Flake8
A linter is a tool for static code analysis, used to flag programming errors, bugs, stylistic errors, and suspicious constructs. e.g. It will tell us that "useless import by admin", or "variable response is declared but never used". These small things can really help the developer as well as the reviewer. As there will be much less cognitive overload. Also, our codebase will be cleaner. To enable flake 8 add these lines in the .pre-commit-config.yaml file.

repos:
-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v2.3.0
    ##black etc here

-   repo: https://github.com/PyCQA/flake8
    rev: 4.0.1
    hooks:
    -   id: flake8
        args: [--max-line-length=88]
        #files: ^my_appname/|^test_suite_name/

However, most of the time this configuration of flake8 is not sufficient. Often, I want to exclude some files, change max_line_length, or ignore some errors. In that case, we can make a .flake8 file or setup.cfg file to give additional configurations. I suggest setup.cfg file because it is a shared config file. Basically, later if you want to add additional hooks similar to flake8 then you can append its configuration in setup.cfg. This is an e.g. setup.cfg file where our .pre-commit-config.yaml file is.

[flake8]
ignore = E501, W503, E203, E402, E712
max-line-length = 88
exclude = .git, backend/alembic/versions/* , backend/db/base.py

Every time you commit. Black and Flake8 will reformat and lint your files. To avoid pre-commit hooks use:: git commit -m "my message" --no-verify

Adding an import reorderer
To enable import reordering we can make use of python reorder imports, sort, etc. This is the additional yaml configuration.

-   repo: https://github.com/asottile/reorder_python_imports
    rev: v3.10.0
    hooks:
    -   id: reorder-python-imports

Now, every time we commit, these hooks will clean up many things in our codebase.
To run our pre-commit hooks on the entire codebase we can type pre-commit run --all-files. Caution: This might change several things in the codebase and it becomes difficult to review the commit. You might need to fix several lines of code and imports to make your git commit. Here is an example in my system: Until and unless I do not fix these unused imports flake8 won't allow me to commit my code.

Afterwards I fix all the problems, I do a gid add and then commit, This time allows the hooks allow me to commit.