Things I've learned serving on the board of the Python Software Foundation

Suspending Tim Peters for 3 months on bogus claims of misbehavior was harmful for the whole Python community. He's the guy who wrote the Zen of Python. If nothing is changed I fear that the Python community will continue to suffer.

This is also another data point that confirms my dislike of Code of Conducts. At first glance they seem like a good idea, but in practice they are often used as a tool to oust people disliked by the ones with the power to enfore them. Afterwards they are used as shield from criticism.

There are a lot of strong assertions here, but no references for those who aren't already following along. Care to share any?

I don't like to nitpick word choices on HN comments, but since you used the word three times in this one: I believe you meant "equated" (to imply that two ideas are equal), and not "equivocated" (to show uncertainty, or to waffle).

I was surprised that there was no mention of the latest drama in "the kinds of things the board talks about" section. I would have assumed it was something that was a topic at the board retreat. But, maybe Simon was deliberately trying to avoid mentioning specifics to today and make this a more general post?

I would like to see an independent investigation into the recent events of the CoC WG, because it really appears to be indefensible.

Same thing happened to the Linux kernel via the same kind of self appointed CoC group. It's absolutely disgusting.

> It is sad that open source is all about foundations and positions now, not about software development.

Actually it isn't. There's plenty of new open source being developed. When a project gets big enough, some kind of formal support structure has to emerge. I can't think of a single time I looked at someone's profile on Stack Overflow and allowed credentials to override working, well-written code.

> I would not advise anyone to spend significant resources to donate free work to Python.

I'm not part of Python governance, but after over 25 years of writing Python code, it makes me sad that you feel that way. On the whole, the Python community has been awesome online and in person. It sounds like you had a really bad experience. I've had a few bad experiences with other open-source projects, and every time, it was really just a case of I had what I thought was a great idea, and the project owners thought differently. Looking back on it, I probably didn't have as great of an idea as I thought I had at the time.

"I would not advise anyone to spend significant resources to donate free work to Python."

The Python Software Foundation (PSF) is a 501(c)(3) which means it is for the benefit of the community as a whole. What better form of organization would you suggest?

This is already much better than most other open source organizations[1], which are 501(c)(6) and are explicitly for the benefit of their members only, which are usually comprised of the silicon valley behemoths.

> It works too quickly and seamlessly for a free service

In what sense? Linux distribution package managers work just as well, if not better.

Free-threading goes in because Facebook wants it. Microsoft wants more control, too.

The current PSF is doing nothing right. Up to 2014 it was a somewhat reasonable organization that marketed Python to academics, which is part of Python's success.

Now it rests on its laurels, gives money to administrators and friends, stifles free speech and locks down the infrastructure.

No innovation has ever come from the PSF. It takes the credit for the blood, sweat and tears of individual contributors.

Board members of the PSF are not paid. We get to claim expenses on travel to PyCon and to the annual board retreat, but we're asked to see if our employer can foot the bill for those first (several do).

Officers who are also PSF staff members are paid because they're staff members - officers who are not staff members aren't paid.

Two of the current developers in residence are paid as PSF staff. One is paid as a contractor which I believe is because they don't live in the USA.

The Fastly sponsorsip is indeed crucial to continuing to offer PyPI as a free resource. Having an infrastructure company sponsor that kind of thing makes a lot of sense, because their underlying costs are significantly less than if the PSF were to pay market prices for that bandwidth.

The only direct PSF elections are for the board - different work groups have different governance structures, none of which involve elections as far as I know.

Python core / steering council members are also elected but that's entirely separate from the PSF, see https://peps.python.org/pep-0013/

$1.8m for a multi-day conference that supports 3,000+ attendees is pretty standard for North America.

We made some significant changes to how grants work based on the feedback from DjangoCon Africa - details of those changes are here: https://pyfound.blogspot.com/2024/07/psf-board-update-on-imp...

It's not unusual for board members to also be officers and staff and it's generally understood that being on the board by itself does not denote a authority over staff, however, a staff member on the board would still be able to fulfill their staff responsibilities.

That said, it would be best to clarify if the board/officers are getting paid.

> My experiments suggest that about a quarter of this could be eliminated simply by enabling and then mandating the use of LZMA (XZ) compression for sdists and wheels (which currently must both use ordinary Gzip compression)

At this point, I wonder if jumping to zstd would be a much larger savings and require about the same amount of organizational effort, even if the technical work would be larger.

>To my understanding, there are currently three of these: Łukasz Langa - the individual who implemented my ban from the Python Discourse forum

Go on...

> I can't readily find these numbers in the report...

The disaggregation you couldn't readily find is on p. 25 of the 2023 PSF Annual Impact Report[1].

[1] https://s3.dualstack.us-east-2.amazonaws.com/pythondotorg-as...

I remember being pretty perturbed a few years ago learning PSF pulled in $5M a year and spent most of the profit on “outreach.” Which is a nice to have at the end of a budget but not as the primary item.

Meanwhile the packaging group been complaining for a decade+ that they are unpaid volunteers and don’t have the resources to maintain existing features in pip, i. e. why they have broken compatibility several times in the last few years.

Very unlikely board is getting paid.

5 million is a rounding error in terms of spend on python programming - Facebook alone between their website and ML likely does orders of magnitude more.

Often a thankless job - for whatever reason these small budget groups attract lots of criticism.

Nothing stops anyone from using or improving python

> My experiments suggest that about a quarter of this could be eliminated simply by enabling and then mandating the use of LZMA (XZ) compression for sdists and wheels (which currently must both use ordinary Gzip compression).

That won't happen as there is a slow effort behind the scenes to get Zstandard in the standard library, which is far better.

I am not a member of the Python community, and I know from here and elsewhere that there is a lot of drama (e.g. "the individual who implemented my ban from the Python Discourse forum"), so I know there is a lot of underlying context here I may not be aware of.

That said, as someone who does sit on a (non-software related) non-profit board, I got so frustrated by your comment because it included so many uninformed and flat out false assumptions, the same ones I have to roll my eyes over when I hear them directed at my non-profit.

First, it is rare (like extremely rare) for non-profit board members to get paid. For most non-profits, the expectation is that board members donate a considerable amount. The author of the post already commented that board members are not paid, making the underlying thesis of your post ("28 people on payroll") invalid.

The other thing that's frustrating about your post is the basic assumption that "number of paid people on staff = bloat", and it's something that drives me crazy. Sure, on one hand, there can be a lot of bloat (especially with larger nonprofits), and of course there are some nonprofits that are borderline frauds, only existing to funnel money to friends and family.

But getting good people to get a lot of shit done costs money. The vast, vast majority of people who work for non-profits could probably get a larger salary elsewhere - they're not doing it "for the money". And there are a lot of jobs where doing it with volunteers is basically impossible. Most volunteers don't want to do the boring yet critical grunt work of event planning, accounting, etc., and those tasks need to be done.

> The PSF had $4,356,000 revenue

In some ways, this makes me even more impressed with Zig. And possibly a signal other believe Zig will have an outsized impact.

Zig foundation raised ~$500k last year, which is 1/9th PSF.

Yet if I had to guess Zig is way smaller than 1/9th the ecosystem Python is today.

https://ziglang.org/news/2024-financials/

This is how most non-profits work. TBH, for something as widely-used as Python, everything from headcount to expenditures are extremely modest.

Just a nit - no one serving the estimated 150 gbits/s of 600PB/year would use AWS to serve it; a more realistic amount would be about 1 million per year.

> could be eliminated simply by enabling and then mandating

It seems like the PSF choosing an option that costs a little more while preserving the current UX of contributions on PyPI is a reasonable choice for a healthy ecosystem. Most of the ecosystem doesn't care about a few thousand dollars of funding, and most do care about the ecosystem of libraries on PyPI thriving. Given the state of Python packaging being what it is this seems like absolutely the right choice, to reduce complexity.

> PyCon US ($1,800,000)

They spend 1.8 million to mess up the sound recordings? I thought the cons was paid for by visitors and private sponsors.

> 600PB per year, this costs around $12 million USD..

This value is off by at least a factor of 10. I pay around $600/yr for a 1Gbit fiber line which transfers more than 3PB/year. PSF is paying $6,000/yr to transfer the same amount, i.e. 10x more. Where is the 10x inefficiency compared to AT&T retail coming from?

> But perhaps "the Python community" (with inclusivity as a core value) is meant to represent a much smaller group than Python developers in general.

I’m not really sure what you are trying to insinuate here. That pycon should be more racist and chauvinistic to increase its reach?

PyPI’s policies are here: https://policies.python.org/pypi.org/Acceptable-Use-Policy/

Outside of abuse, PyPI does not impose editorial standards on packages. That would take an incredible amount of additional work, and it’s not clear to me that it would be “better”. How much does it really matter if there’s a university student project on there with virtually no downloads?

“I think package maintainers should be required to push an update every other year or have their package get dropped.”

Sometimes libraries really are “finished” - if you go through your dependency stack you may find a surprising number of packages with no new releases in the past 12 months, because they didn’t need a release.

I tried that myself just now, here are some of the packages I found that haven't had a release in a few years:

    decorator               2022-01-07
    rfc3986                 2022-01-10
    aiosignal               2022-11-08
    colorama                2022-10-25
    h11                     2022-09-25
    jmespath                2022-06-17
    mdurl                   2022-08-14
    rsa                     2022-07-20
    mergedeep               2021-02-05
    dictdiffer              2021-07-22
    janus                   2021-12-17
    conda-content-trust     2021-05-12
    six                     2021-05-05
    uritemplate             2021-10-13
    pytest-clarity          2021-06-11
    ptyprocess              2020-12-28
    backcall                2020-06-09
    text-unidecode          2019-08-30
    PySocks                 2019-09-20
    sphinxcontrib-jsmath    2019-01-21
    pprintpp                2018-07-01
    homebrew-pypi-poet      2018-02-23
    pickleshare             2018-09-25
    webencodings            2017-04-05

Should there be an expectation of a package being particularly useful to be in a package repository?

You see the same in other places like npm or docker repositories and it is not a problem.

Manually checking things is very much out of scope for a service for open source like this. Limiting it by arbitrary metrics like code cleanliness would also just give a false sense of quality. One thing that'd make sense to me would just be asking for confirmation that the upload is not more suited to test pypi instead of the main one. Not sure whether the tools aren't already doing that or not.

The major problem that's being somewhat worked on now is typo squatting, names taken up by old packages, and other security considerations around pypi. Random packages being useless (or malware) doesn't fall under that in my mind as you just won't or shouldn't be downloading completely random things.

Admittedly there isn't as much man power dedicated to it as I think there should be, more so after I saw how much admin there is in PSF with the recent coc debacle.

I think you’re confusing two things: PyPI has maintainers end administrators, but that doesn’t mean that it’s a curated index. Like RubyGems, NPM, Cargo, etc., PyPI explicitly does not present a curated view of the packaging ecosystem. Doing so would require orders of magnitude more staffing than the index already has.

Python as a community prefers standards over implementations, which is why you could easily stand up your own curated alternative to PyPI if you wanted to. But think you’ll discover that the overwhelming majority of users don’t want their resolutions breaking just because a particular package hasn’t needed an update in the last 6 months.

That’s what I was trying to hint at with the word “ostensibly”.