The most boring part of setting up a new code project is typing the boilerplate: it’s easy to forget bits, so we just copy the last project’s folder and “file the serial numbers off” on the project name.
Copy-pasta-driven project setup is not great though, as it’s too easy to forget replacing values in obscure files, or misunderstand why project is set up that way in the first place, leading to nasty surprises down the line.
Related, I happen to have a personal rule, which is to answer every question asked for the third time in writing (as wiki entry, or on this blog), and link that document as answer, rather than re-explaining the same thing. The reasoning is that a question asked three times is evidence of missing documentation, hence valuable to write down.
I believe the same thing about code boilerplate: Once I’ve written the same code thrice, I should make it reusable directly. In this article, I show you cookiecutter project templates, and my personal python project template.
Cookiecutter is a tool (written in Python), used for project creation. It exposes Jinja2 file templates with a list of questions as JSON file. It’s a very nice introduction to the world of project templating, and can be used for all sorts of projects, not just Python ones.
To solve the repeat project-creation issue, I’ve started years ago to write down my own Python project template, using cookiecutter.
[tool.poetry]
name = "{{cookiecutter.package_name}}"
version = "0.1.0"
description = "{{cookiecutter.description}}"pyproject.toml sample from the template: note the {{cookiecutter.something}}, ready for replacement
As soon as I figured out a new trick, or a best practice I wish to reuse, I’d put it in the template. See my template on Github: python-template.
I eventually realized there are two major obstacles to project templates, which we should solve to get the full value of templating.
How do we know the template is right? How do we know the file generated is still valid in your language? Maybe it causes parse errors, maybe it fails your formatting rules. Basically, we want to know that we can “build” the project properly, after generating it from template.
This means defining what the template should be able to do once expanded, and running those actions on an expanded template, to validate it. Back to having a Continuous-Integration system that validates our changes!
I solved this by creating actual python tests for the template, built around custom template-expanding fixtures: expand the template in a new folder, run a few well-known commands, then assess the result is acceptable.
def tests_template_docs_ok(template: Template):
"""Checks we can run make docs on rendered code to get HTML"""
out_path = template.run_in_dev(["make", "docs"], template)
assert os.listdir(out_path + "/docs/build/html/"), "Docs not built"make docs does create a non-empty folder for HTML documentation
Another test is running make test and confirming test_results/results.xml
JUnit test result files created (for test result reporting in CI)
Some tests are more esoteric, like checking we can build a docker image off our new package, both the release-worthy image, and a development image, or that running the “build” commands leaves no file unstaged in git, because our gitignore should capture all relevant files.
The other challenge is sharing the reasoning for the template’s layout:
teaching, for instance, that a Makefile is the simplest way to interact with
the project. Justifying why a release-oriented Dockerfile is called
release.Dockerfile, not Dockerfile.release. Explaining why linters and
formatters are run as pre-commit hooks not standalone commands.
This is both a feature discovery problem (new users need to know what their
fancy templated project can do) and an architectural decision issue (justify
why use poetry when pip + virtualenv are sufficient).
I solved this by creating a file called DESIGN.md at the root of the
repository, which attempts to document major design decisions of the template.
Every big feature is an entry, along with an explanation why it was chosen, what alternatives may exist and why they were dismissed.
Again, this is both a log of decisions for the record, and an opportunity to teach new tricks to developers. Some of those decisions may come to be reversed over time, as I learn to do better myself. I look forward to that part.
After a few years tweaking it in the dark, I’ve finally decided to publish the template. You can find it in Github under python-template. I hope you try it out on a new project of yours, and that it either helps you get started, or teaches you (or me) something.