One of the biggest problems with GitHub Actions is that, even with fully transitive locking at the action layer, you still can’t really guarantee hermetic execution: lots of actions do implicit version resolution on tools by default. For example, setup-python, etc. will select a reasonable version based on the constraints you give it, which may end up being the pre-installer runner’s Python, or a newly released patch version, etc.
Fully pinning action references themselves is a step in the right direction, but the ecosystem as a whole probably has expectations that are misaligned with reproducibility/hermeticity, and those expectations will be challenging to overcome.
Arguably, that's exactly the one action that will need to be hash-pinned, since all the consecutive actions will at least be verified against the lockfile.
I have trust issues using actions that are not from official and reputable sources like GitHub official and AWS actions. I don’t know why an ssh action is necessary, seems like a nice way to get your ssh credentials stolen and ssh isn’t that hard.
TBH this discussion and the need for a lockfile for your CI makes me dizzy, is there something I'm missing wrt GHA that makes it awesome enough to be worth these tradeoffs?
For reference, I come from a Gitlab CI background and all I want is to specify a container, and the CI system should clone my repo in it and run some tests; perhaps optionally allow me to write stuff in a text file that can be displayed on the pull request or the commit (although Gitlab CI doesn't do that AFAIK). Is there something I'm missing due to which GHA architecture is so complicated?
Yeah, GHA has a massive plugin ecosystem. A common pattern is using an “action” from a plugin as one CI step/job. So these are dependencies that get resolved at runtime when GHA puts together your workload.
Other CI platforms have plugins, but the “plugins” in GitHub really get used as the core primitive of the system, which is part of what makes it so simple & easy to use… for really basic workflows. You just hook up a couple actions like this and you’re good to go, no shell scripting required. (Though you can totally do that too.)
I mean at the end of the day, it’s a big part of the value proposition, even if I prefer a much more bare metal approach. GHA is really not great at massive CI workloads.
From what I see, this does not help with pinning the dependencies and it doesn’t verify the downloaded action has the same content as it used to have. In other words, this is a tiny patch on a big wound.
We use commit hashes to pin actions, have the version as a comment (e.g # v4) and renovate will keep both up to date in the PRs.
And there is a more or less recently added repository setting to require actions to be pinned to hashes.
I don’t want to throw process at the problem. I think GH should provide a better system not the developers locking down dependencies and adding extra processes and steps to update the CI via a PR workflow. Not like PRs became the development bottleneck anyways for a lot of development teams these days. I wonder how we functioned 15 years ago with trunk based YOLO development.
I also think that it wasn’t the best idea to base versioning on mutable branches and not introduce a registry in the middle. Think about it. The whole system is build on node anyways. But we pull “dependencies” via a weak git clone system.
You don't use actions pulling in unpinned dependencies outside of trusted distro package manager at runtime.
I believe this problem is probably overstated. Can you point us to such an action you are concerned with that has either transitive actions dependency or unlocked npm dependencies where maintainers aren't responsive to addressing PRs to illustrate?
Sure, or we come up with a proper solution via lockfiles so we don't have keep forking and maintaining, and make full dependency locks the default so everyone benefits.
This is a long solved problem in every other ecosystem. This particular implementation isn't great but it has the right idea.
I have been banging on that drum for like 2 years now, glad the community has figured a way around it. Still utterly ridiculous that this is not native.
They even closed the immutable action issue as a "wont fix" cause you know when it's too hard we all know the best way is to give up. Not like there wasany major security incident this year due to this /s
Fully pinning action references themselves is a step in the right direction, but the ecosystem as a whole probably has expectations that are misaligned with reproducibility/hermeticity, and those expectations will be challenging to overcome.
gjtorikian/gh-actions-lockfile@v1
Presumably since it has to run first it must run unpinned?
For reference, I come from a Gitlab CI background and all I want is to specify a container, and the CI system should clone my repo in it and run some tests; perhaps optionally allow me to write stuff in a text file that can be displayed on the pull request or the commit (although Gitlab CI doesn't do that AFAIK). Is there something I'm missing due to which GHA architecture is so complicated?
Other CI platforms have plugins, but the “plugins” in GitHub really get used as the core primitive of the system, which is part of what makes it so simple & easy to use… for really basic workflows. You just hook up a couple actions like this and you’re good to go, no shell scripting required. (Though you can totally do that too.)
I mean at the end of the day, it’s a big part of the value proposition, even if I prefer a much more bare metal approach. GHA is really not great at massive CI workloads.
We use commit hashes to pin actions, have the version as a comment (e.g # v4) and renovate will keep both up to date in the PRs.
And there is a more or less recently added repository setting to require actions to be pinned to hashes.
Pin by hash.
Verify that the actions themselves aren't pulling in unpinned dependencies from Actions, NPM, or elsewhere.
Have a CI job or bot create PRs for new versions. Verify those PRs before merging.
If any particular action becomes a recurring chore or risk, consider if you should keep depending on it.
If you do these things, the "we need a package manager" is moot and most if not all of the concerns in that blog post don't affect you.
I believe this problem is probably overstated. Can you point us to such an action you are concerned with that has either transitive actions dependency or unlocked npm dependencies where maintainers aren't responsive to addressing PRs to illustrate?
JavaScript actions are already bundled.
..and in the next update cycle, you will see all actions be pinned like this:
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6
Just pin your actions to shasum
This is a long solved problem in every other ecosystem. This particular implementation isn't great but it has the right idea.
Depending on the action you use, this is no small task. You might as well just switch to something else altogether.
They even closed the immutable action issue as a "wont fix" cause you know when it's too hard we all know the best way is to give up. Not like there wasany major security incident this year due to this /s