Do you know what your full dependency graph looks like?

I built Emporium to get a better idea.

Emporium looks at libraries on the Python Packaging Index (PyPI), and analyses their dependencies.

For some libraries, that’s quite a small set - many have no dependencies at all, or a tiny handful at most.

For others, well, the dependency graph for Plone clocks in at around 1800 edges!

What’s the point?

Right now it’s exploratory.

Some things I’m interested in looking at:

• Building better dependency resolution systems. This is much easier if you know “the state of the world” all at once, rather than building it up one package at a time.
• The various patterns and conventions of people’s setup.py files
• Tracking how library dependencies change over time - when and what gets added and removed
• Making the full dependency tree more visible and accessible

What does it do?

To do this, Emporium fetches the sdist, extracts the setup.py file, and parses it for dependencies.

This is not fun or pleasant.

A common, straightforward setup.py looks like this and is straightforward to parse:

import setuptools

setuptools.setup(
    name="example-pkg-your-username",
    version="0.0.1",
    install_requires=[
        "foo==1.0",
        "bar>=1,<2",
    ],
    # etc.

But setup.py is a Python script, and thus can do anything.

Dealing with something like the following is a bit harder:

install_requires=[open("requirements.txt").readlines()],

Because the package metadata is a Python script, there’s only one guaranteed way to extract dependencies: run it and see what gets passed to setuptools.setup’s install_requires.

So

It’s an experimental side-thing I hack at from time to time.

It’s got a way to go before it’s particularly useful, but it’s certainly interesting already to see various dependency graphs and setup.pys

Check it out: https://app.emporium.cloud/