Tongfamily (Richtong.com Sync)

Well, it’s been about nine months since I’ve written a significant python program and I have to admit I’ve forgotten a ton of things and I’m reminded that I never did figure out the right way to package things, so during the last three weeks, I started with the simple stuff, but now I’ve ended up remembering quite a few things and here are some notes about that and yes I’m trying to stick with Ps in this post 🙂

Packaging

When you write your first Python programs, you probably, like me, scattered the files all over your root directory of your project. Then everything just works fine, if you create a util.py there then you can do an import util and it just works. And, if you want to run pytest, then just having a test_util.py also just works.

Well the problem occurs when you want to have your own nice Python PIP package and do a pip install pytong (which is now a real package) and have it installed everywhere. Then a few things break. The first is how do you actually import things.

After a bunch for reading, it turns out the trick is that you can leave everything the way that it is as long as you do four things:

1. Move all your source files to ./src/<package name>. the reason for this is that you are actually going to run your code like everyone else, instead of just being at the root, it is buried down and then you use the installation of a pip package to get to it
2. Move all your tests to ./tests and in fact most of the time your package starts as a simple task that gets to me more general, so your starting main.py can actually be a command line script that calls your package by adding at entry_point that points to console_scripts entry point to your setup.py
3. Now you create a setup.py at the the project route and then a ./src/<package name>/__init__.py which imports all the modules that are your source. This __init__.py file is very special, it basically says that this is a package and you should run this file first. It uses the classic double underscore (or dunder) notation that is all over Python.
4. Then run a pip install -e . which is the real magic command. This basically fakes the system and installs your source code like a pip package. The -e means editable but instead of downloading the package it looks directly into your source code on your machine.\
5. To get the proper type hints if you are using Python typing, you need to create a null file called py.typed in the package area and then in setup.py you need to add package_data = { 'yourpackage' : [ 'py.typed'] }

The reason for all this magic is that you no longer want to run your code directly, but to use to pip to fake out it being a package. Then you are running exactly like someone who is going to pip install your code.

Net, net, if you everything your code may become a package, you probably want to start with this directory layout right away.