One thing I found systemd really confusing was its treatment of ExecStop in a service script. ExecStart is the command to run when systemd starts the service at system boot up (or when a user tells systemd to start the service). However, ExecStop is run when the starting command has finished running. You have to set RemainAfterExit=yes to have the desired function of running the stop command on system shutdown or on user stopping the service. ExecStop is basically the "on-cleanup" event rather than "to-shutdown-the-service" event.
It is important to keep in mind that systemd is tailored towards daemons. So if your service just runs a command that eventually exits, you need to explicitly tell systemd to treat it differently than a daemon.
Edit: As others noted, you’re probably looking for oneshot + RemainAfterExit.
It is a little asymmetric, because 'ExecStart' is actually normally 'Executable that is the service', not just script that starts the service, but I think that's a hangover from the self-daemonizing approach to init scripts.
Then ExecStop is basically on-cleanup, not to-stop. ExecStart really is to-start, not on-start. In the httpd server case, ExecStart runs the httpd command.
If that helps you understand it better, then sure.
All I am trying to say is that the name of the option makes sense as-is: it literally runs the provided command(s) on service stop (“execute on stop”).
Similarly, ExecStart literally means command(s) to execute on start.
If the command runs a blocking daemon/server (usually the case), then the server will be implicitly stopped by definition - because if you’re stopping the systemd service, you’re interrupting any commands that are still running/blocking.
I think part of the problem is with how one "naturally" or "habitually" thinks of a Service. From systemd's perspective/terminology, the Service is the thing that starts and stops. But whether because it is inherently more intuitive, or because of how daemons traditionally worked on *nix, the mind tends to think of the process which the Service starts as the thing which starts and stops. I'm not able to double check currently, but I also think that systemd isn't totally consistent with the mental model it bases the choice of keywords on when it comes to ExecStop, because with that mental model one would expect ExecStop to only run when the Service is stopped from systemd, but I'm fairly sure it runs in other cases when the process started with ExecStart exits, but the service isn't necessarily stopped. I could be wrong about that though.
This was ultimately what I needed to do when I wrote a systemd service that managed some firewall rules. It really was a footgun though, what with having essentially different meanings/purposes for ExecStop whether you’re doing a Type=forking, a Type=oneshot, or a Type=oneshot with RemainAfterExit=yes.
And relatedly, I honestly have no idea when I’d want to use ExecStartPre, or multiple ExecStarts, or ExecStartPost, and so on.
I would argue the semantics of ExecStop are always the same. It's the command that's executed to stop the service. On the other hand, what it means for a service to be "running" or "stopping" naturally depends on what type of service it is (i.e., is it a daemon or not?)
Yes, so whether the service is stopping as a result of the process exiting, or whether you requested the service to stop manually, it will run the ExecStop in either case.
That makes sense to me personally. What would be the more intuitive design in your mind?
Stopping as a result of the process exiting or requested the service to stop are two very different things. Systemd overloads the term ExecStop for different semantics, relying on different property settings. That's where the confusion comes from.
The name sounds like it means "this is how I want you to cause the service to stop" to me (and clearly to others as well). That would be symmetrical with ExecStart meaning "this is how I want you to cause the service to start". If it runs after the service stopped it should be called "ExecAfterStop" or something like that.
That is what ExecStop means. It specifies how you want to cause the service to stop. But the lifetime of the service isn't exactly the same thing as the lifetime of the process that got started in ExecStart.
Maybe think about it this way: ExecStart is what the system will run to transition the service from the "starting" state to the "started" state. ExecStop is what the system will run to transition the service from the "stopping" state to the "stopped" state.
For a service with RemainAfterExit=no (the default), you enter the stopping state right away once the processes that got started in ExecStart exit. That's useful when you are starting some long lived process as a service, and in that case there is usually no need for an ExecStop. But semantically, ExecStop has the same meaning either way -- it's what needs to be run, if anything, to transition the service from the stopping state to the stopped state.
I have now found the documentation for ExecStop (in systemd.service(5)), which hopefully improves my understanding.
It definitely seems to be both "cause to stop" and "after (unexpected) stop" in one. You can look at $MAINPID to see which case you have. This design apparently makes sense to you, but to me and several others in this thread a service that has already stopped isn't in need of being stopped and shouldn't execute commands intended for that. (There is a separate ExecStopPost for "after stopping, for any reason".)
Yes. ExecStart works the same for all the cases. ExecStop works differently though. While ExecStart is the event to kick off the command for the service, ExecStop is not. The asymmetric semantics are where the confusion comes from.
It's been enlightening to me to read through some of the distro-provided .service files to see what can be done, with services I'm more of less familiar with.
Love this. I use raw CLI commands until it hurts, and have recently embraced tools like lazygit/lazydocker to get visibility into otherwise opaque system/tree states, and it’s been a huge level-up.
I have several user and system level services I manage, but debugging them is tedious. Your opening line that lists common commands and their pain points really resonated with me.
I’m on NixOS, so editing immutable unit files directly won’t work, but the service discovery, visibility, and management will be really helpful. Nice work!
I am also a NixOS user and this is exactly what motivated me to work on this project!
I am planning on adding some "guides" in the documentation but in short:
You should check out `systemctl edit --runtime` for debugging units on NixOS. It makes debugging sooo much easier.
This looks very good, thanks for sharing! I maintain a similar project and working with the systemd/dbus APIs has been pretty painful; eager to try this and see what I can learn from it.
At long last, systemd-client: merry meet! Next step is such a TUI for non-Linux such as macOS, FreeBSD, Windows. For macOS I use LaunchControl.app but it isn't a TUI.
I haven't installed it yet so quick question: can it connect to remote host ? I often use systemctl --host <hostname> status foo.service (status, timers, logs etc. )
Dang. I have never heard of `systemctl --host`.
Sadly not. It is more or less a fancy wrapper around (the local) `systemctl`.
But there is also an appimage that should make it (hopefully) easy to run it on remote servers.
Either way, feel free to open an issue and I will have a look at it.
> Dang. I have never heard of `systemctl --host`. Sadly not. It is more or less a fancy wrapper around (the local) `systemctl`.
Sounds like you could easily support it by letting users pass in $REMOTE_HOST and when you use your `systemctl` wrapper, add `$CMD --host=$REMOTE_HOST`, after that everything should work as before.
So cool that with uv it becomes so easy to install such tools.
What's missing in the install routine is uv installing this tool ignoring the Python dependency. My box has 3.10 and isd won't work with it. Fixed with `-p 3.13` option. May be worth mention in the docs.
I can not install this stuff on remote servers and docker images. I would like multiple backbends to execute commands and gather informations (local, ssh, docker).
It should be installable locally, and run commands on remote machine via ssh! And via 'docker exec' commands.
> If you have nix installed, this AppImage will not work! Either use the nix or uv installation instructions!
Is this really true? I understand why it does not work on NixOS (I tried just out of curiosity and it seems like it is unable to exec the host systemctl for some reason) but I don't think there's any reason it wouldn't work on other OSes that merely have Nix installed.
Interestingly though, on Nix v2.24.11, I can't use the provided Nix command either:
$ nix run https://github.com/isd-project/isd
error:
… while fetching the input 'https://github.com/isd-project/isd'
error: Failed to open archive (Unrecognized archive format)
Even if that did work (you could adjust it into a Git URL to make it work) it would probably not be ideal since Nix has a native GitHub fetcher that is more efficient. I think this should be the actual Nix command:
nix run github:isd-project/isd
Anyway, this is cool. I actually wanted to make a similar thing using systemd's DBus API and Qt instead of a TUI and even started writing code for it, and if you wanted to I'm sure you'd find that the DBus API probably provides all of the functionality you would need (admittedly it is a lot easier to just call `systemd -H` than to implement your own SSH tunneling, though.) It kind of frustrates me that systemd and modern Linux in general is absolutely teeming with data and interfaces that could be exposed and make administering systems, especially desktop systems that were traditionally very inscrutable, much easier. e.g. in the past, how did you know what was going on when an xdg autostart app failed? Now with systemd running xdg autostart apps in some desktops, it would be really easy to provide a GUI that can show you the failed autostarts and even provide a GUI log viewer, and yet somehow, such a tool does not seem to exist, at least in the realm of things that are maintained and relatively feature-complete. Rather frustrating.
> Is this really true? I understand why it does not work on NixOS (I tried just out of curiosity and it seems like it is unable to exec the host systemctl for some reason) but I don't think there's any reason it wouldn't work on other OSes that merely have Nix installed.
Yeah, it get's complicated and I don't want to recommend it and explain the details. In short, I am creating the AppImage via nix. And the AppImage "mounts" (not overlays!) the AppImage's /nix/ directory. So calls from the TUI that would access /nix/ wouldn't go to the systems `/nix` directory, which leads to all kind of weird issues. For example, you could install your EDITOR via home-manager on Ubuntu. isd would start correctly because systemctl is "accessible" but if you open your EDITOR under `/nix` it wouldn't find it, which is super confusing as a user.
-> So it is just easier to say to use the nix installation method if you are already using it :D
And sorry for the wrong docs, it is fixed now.
I also agree with your frustration. Personally, I would really enjoy working on such a tool but it wouldn't be an easy task, and who would support the work? This TUI had a manageable scope but it was still quite a bit of work. So I don't see myself investing too much into "higher-level interfaces".
PS: I have no idea why your post is ill-received :/
> In short, I am creating the AppImage via nix. And the AppImage "mounts" (not overlays!) the AppImage's /nix/ directory. So calls from the TUI that would access /nix/ wouldn't go to the systems `/nix` directory, which leads to all kind of weird issues.
That makes a lot of sense. There's not really much trivial that I think you can do to deal with that.
> PS: I have no idea why your post is ill-received :/
It's not really a big deal, I was just surprised and wondered if anyone who was compelled to vote down might come back to explain why.
That may be a bit of nix backlash, don't sweat it.
I agree Linux could use better system APIs than "put file here" and "run these commands" which are much more error prone than making calls to properly documented interfaces.
Looks great, well done. It's a shame that it's needed at all. The vast majority of my interactions with systemd are trivial: (re)starting a service, looking at a log file to figure out what's wrong, and making sure a service starts on boot. I find it baffling that the ergonomics of systemd for those common tasks are so lacking. But the TUI seems to help, so thanks.
And sure, systemd it's more deterministic and includes the kitchen sink, unlike initd.
Thankfully these days I can automate most of such interactions out of existence, so I no longer feel the burning hatred that I once did. More like a smoldering ember.
I mean, I get why they don't want to "bloat" systemd with a complex/opinionated TUI, but I would've also really liked a more "upstreamed" interface.
Though, I guess making systemd more beginner-friendly/accessible is not really that important for their funding?
How's security handled? Not in terms of system permissions which Linux handles well, but in terms of guarantees that it can't be hijacked and remotely controlled by an external attacker.
Author here: I also find this an important thing to ask yourself when you are running applications/scripts that do anything with sudo and which is why I have written a fairly in-depth "Security" section on the isd documentation page:
As a suggestion, since your repo is open I think you can leverage sonarcloud without costs. It would make for another independent check that your code does what's intended and for instance won't ping a remote control server, either via dependency attacks or via malicious pull requests that could dodge reviews.
Where I work we also use defectdojo to catalogue and manage CVRs in our projects, but it's more involved to setup the testing pipeline and deploy the required services.
Keep in mind, systemd is still a default-allow access control list (ACL) and has a very long road ahead in form of Linux security, as long as CAP_SYS_ADMIN continuea to exist.
May make it easier to customize but it doesn't close the security loopholea like SELinux, GRSecurity, TOMOYA, or AppArmor does.
Honestly it's not all or nothing. It really depends on your threat model. If you assume all native code is both not malicious and incapable of being compromised to run untrusted code then the existing model works. At that point it's more important to focus your security efforts on how native code ends up running, and therefore the distro specific packaging ecosystem.
Now for many that threat model is not sufficient as they both run increasingly less trustworthy software, obtained by less trustworthy mechanisms such as npm or off a website, or simply want to protect against bugs that cause otherwise non malicious software from being compromised and resulting in security incidents. I'm in this latter camp but we can't ignore the fact that there are many who happily operate in the former. There also exist solutions such as we browsers with their sandboxes and VMs that somewhat fill the requirements for running untrusted software for these individuals.
Haha this is great. It’s funny that others also end up doing this enable, start, journalctl, start dance. Good stuff. I shall try it.
My only problem is that I wish there were a way to install it on my machine and have it connect to a remote systemctl but that probably is a lot of work to reliably work (port may not be open etc etc).
Maybe it is possible with `systemctl --host`...
Though, I would have to make quite a few changes to the UI design since it
would mean that there are "multiple modes". One for local and multiple remotes.
So... Check back in a month or so :D
- That the tab pane switching is the slow part is a bit weird. Is there a difference when you see all units and a subset of units? Maybe open an issue and I can provide more possible settings that could be changed.
- Jep, page up/down handling will be added. Thanks!
- Not quite sure when this happens. Maybe an example screen shot (before/after) could help?
I've opened an issue about the reordered attribute values (+ another bug I found). Tab pane switching stopped being slow (I suppose my laptop was simply overloaded, and I'm too used to things being fast even when it is).
It was great!
The extensive documentation and the vast amount of widgets were definitely the main reasons why I have chosen to do it in Python/Textual rather than other languages/libraries.
Though I don't agree with all design decisions, and there are some bugs, having such well-maintained documentation and clean repo makes it easy to dive into the project and understand what is going on and how other components are built.
Need to see this in action. The example videos overuse the fuzzy search for commands in my opinion. Makes it hard to concentrate what the actual action/information is when a huge part of the screen changes. Not to say I don’t like fuzzy searchable command prompts. But it can be clearer to either just type out a command or have a menu system. As I said I need to check this out in action for myself.
This looks super well done and polished. The rich docs are crazy detailed for something that was just released. You're really setting the bar for what other projects should do. Congrats!
I've put quite a lot of work into it and wanted the project to give a good first impression. This was important to me since I believe these "niche" TUI libraries need to immediately show what problems they are solving or how they save time, as users need a reason to _use_ an additional abstraction layer.
Thank you for shipping this. Need to test it but going to add this to the deps bundle for my deployment service (relies heavily on systemd). Looks like it would make day to day maintenance/debugging far less of a headache.
This would be extremely helpful. Although, I would love a feature where you can add different linux hosts and manage them all from a single place, using ssh
Most systemd functionality has built-in support for targeting remote hosts and local machines (containers, VMs) and mostly utilizes SSH under the hood to do so.
Check the -H and respective -M flags of many systemd CLI utilities.
Hey, mostly because the underlying TUI framework (textual) is _amazing_.
Not gonna lie, I am not a big fan of Python anymore because of the dependency hell you can run into when working with CUDA libraries but uv is a breath of fresh air and textual is just sooo easy.
Don't get me wrong, I am not saying that I would never consider rewriting it in Rust or Go, but the documentation and guides from Textual were great resources and the creator, Will, also seems like a really nice guy :D
Edit: To be explicit, I believe that Python has currently the most advanced and accessible TUI ecosystem. At least, that was my impression after checking a few examples on an afternoon.
1. My life improved a lot after I found that you can do "systemctl status $PID" and systemd will find what service (if any) is responsible for the process in question. This has been a life saver many, many times. But, more search would still be welcome, especially for cases when the system fails to boot, or fails to reach a particular target etc.
2. I think systemd didn't go far enough with unit files. The motivation was to escape the hell of Shell scripts, where each system was defined in its own unique way, and was failing in a dozen of unique ways. While, initially, it might have seemed that a simple INI-style format could manage to describe service requirements... I think, it's way overdue to realize that it doesn't. And sysadmins on the ground "fix" that by embedding more Shell into these configuration files, bringing us back to the many unique ways a service will fail. Perhaps, having a way to edit these unit files so that it doesn't expose the actual format may lead to improvement in the format (more structure, more types, templates).
This looks neat. I have to look up the very fiddly and unintuitive systemd commands all the time. service start? service.foo start? start foo.service? Oh right, sudo systemctl start service.foo
And the feedback is so bad. It should know everything in its own config dir and tell me how to do what I want to do. Was it enabled? I forget. How do I look at logs? Oh right journalctl. Also the layout of things with lots of symlinks and weird directories in places that annoy my 90's linux sysadmin brain. Why am I looking at /lib/systemd/system
I am annoyed by the redundant "systemd/system" directory name every time I have to go there. At this point, just promote it to /etc/systemd and build a better CLI.
As a very occasional linux sysadmin just trying to make things work, the "typing at a console" systemd interfaces are not fun to work with. Maybe nobody should be doing that. In an enterprise, sure that's different. I think interfaces should be human, and linux should still be fun.
> I have to look up the very fiddly and unintuitive systemd commands all the time. service start? service.foo start? start foo.service? Oh right, sudo systemctl start service.foo
I don't get this complaint. It's the same order as almost every other command-line utility that has subcommands: <command> <subcommand> <thing to operate on>. To me, that kind of consistency is very intuitive.
I feel the same way. The big part for me is that it tells us that owyn doesn't use tab completion if they're forgetting about the ".service" part. Sure, I don't remember either, I don't have to.
Anything in brackets is optional and might not appear or be available. By command I mean a category of commands. Such as 'pip install' vs 'pip uninstall', which are sub-programs inside the main program. But this can have layers such as 'uv pip install'. Often flags can be used in any order because you'll just loop over all the arguments but this is still the standard order.
Also the `.service` part is optional anyway for most commands (including the start, stop, restart etc. ones they use in their examples). Only commands that can operate on services and other non-service units require it.
When it comes to starting and stopping services I want the verb to go last. Way easier to press up, backspace backspace backspace o p to change service ssh start to service ssh top. This is a frequent pattern I follow as I start/stop/restart/reload. Having to go back at least one word adds keystrokes that aren’t necessary.
It's a reference to an XKCD comic https://xkcd.com/1172/ that pokes fun at users with strong attachments to some software behaviours and/or interfaces.
Converts the "start" in "foo start bar" to "stop", ie runs "foo stop bar". Append :p to do the substitution but print the command instead of running it.
Caret is a word designator in bash's history scheme. Here it acts as a way to mark the thing with the thing being replaced. There's a more detailed explanation in [1] and a quick read in [2]. Essentially bash let's you do manipulation and re-execution of commands from your history and one of the manipulations is via word substitution.
Because I switch computers (often, for work), and what's muscle memory on one then becomes "command not found" on others without the alias. Many of those computers I don't control and can't say "well, everyone should just run my aliases".
Because I have to share commands with other people who are troubleshooting their own systems, and copy/paste from history becomes useless if I have specific aliases.
Because someday I or someone will want to script these interactions, and aliases are not available in subprocesses.
I think the previous post mentions the `service` command, which has `service foo start` pattern.
I'm used to `service` command, so I have muscle memory which `systemd` breaks. The way `systemd` commands are laid out is better, it just messes with `service` command muscle memory.
I disagree that the way systemd is laid out is better.
With initV, issuing many command in sequence to the same service (enable, then start… stop then start… etc) is much quicker and easier as it’s “up-arrow, control-W, type new command. With the systemd option I either need to navigate one word to the left (or more depending on how the shell is configured wrt word separation) before I replace the command, or I need to delete both the command and the service name, and retype more.
It’s not a huge difference but small downgrades in ergonomics add up over time.
The systemd layout allows you to send the same command to multiple services, which is much more frequently useful than sending multiple commands to one service.
Many distros still provide the `service` command, and don't print any warnings when you use it. At best, you might get an informational message that the equivalent systemd command is being called. There is no recommendation to call the systemd command directly, or any sign of the `service` command being deprecated any time soon. As a result, a lot of people are probably still relying on their muscle memory for the `service` command.
I on the other hand DO get this complaint. It’s less about the “word order” and more remembering the words to type. As a software engineer I have to google stuff all the time. I haven’t touched C#’s PLINQ in five years, I doubt I could do it from memory. Likewise, I can count on one hand the number of times I will set up a systemd service in a given year.
Many (most?) of us are all over the place, expected to “wear many hats” and aren’t in a single IDE or language all day every day. Certainly if I were a “Linux admin” I would have systemd pretty committed to memory. Anyone else probably wouldn’t.
Sure, it's different from the old way, but I don't think "unintuitive" is the right word for that. systemd forced people to change their habits so that it could be more intuitive. Of course, people are going to disagree about whether it was worth it - it's the age-old question about breaking backwards compatibility for the sake of minor improvement. Personally, I got used to it pretty quickly and I like it more than the old commands now.
The way I remember this is that the old way didn't allow for applying the verb to multiple units. Now you can "restart" multiple units, i.e. `systemctl restart nginx webapp`, etc.
Plus, you could always write a quick shell script to swap the arguments if it detects this specific failure. Inability to remember or learn which commands expect which arguments is, at some point, not the responsibility of the software to fix. Unfortunately, this field does necessitate some amount of memorization and ability to problem-solve.
My biggest annoyance is "systemctl status" gives you just enough of the service's log to make the output take up most of the terminal each time you run it but never enough of the service's log to get a useful picture of what's actually happened with the service lately.
Not to mention unless the problem with the service completely prevented it from running (it advises some commands to run in that case) you're supposed to just always remember "journalctl -xeu $SERVICE" was the incantation, less you want to go look up the flags again or manually parse the entire "journalctl" output.
Overall I generally like systemd though. The syntax can just be a burden sometimes.
it's the same mentality that brought us the git design - the easiest, least typing options are rarely, if ever the thing you want to do.
Instead, these invocations give cryptic messages, throw errors, or sometimes, even break things.
The most common and helpful things are hidden deep behind multiple flags and command line arguments in manuals that read like dictionaries more than guides.
I'm always at a complete loss as to how such decisions are made. For instance, "git branch -vv" is the useful output you would like to see, every time, that should be "git branch". Why not make the current output, "git branch -qq"? Is a humane interface too much to ask for? Apparently...
I know people defend this stuff, but as a senior engineer in programming pits for 30 years, they're wrong. Needless mistakes and confusions are the norm. We can do better.
We need to stop conflating elitism with fucked up design.
> Why not make the current output, "git branch -qq"? Is a humane interface too much to ask for? Apparently...
Yes, it is too much.
You have the wrong mentality, and I hope this can help make your life easier. Programs are made so that the simplest option is the base option. This is because there is a high expectation that things will be scripted AND understanding that there is a wide breadth of user preference. There's an important rule
DON'T TRY TO MAKE A ONE SIZE FITS ALL PROGRAM
Customization is at the root of everything. We have aliases that solve most of the problems and small functions for everything else. You default to an non-noisy output, showing only __essentials__ and nothing more unless asked. Similarly, you do no filtering other than hidden files. This way, everyone can get what they want. Btw, this is why so many people are upset with default options on things like fdfind and ripgrep.
For your problem with git, there are 2 solutions you have.
# alias git branch using git
git config --global alias.branch 'branch -vv'
# write a simple function and add to .bashrc or .zshrc or .*rc
git() {
case "$1" in
branch)
shift
command git branch -vv "$@"
;;
*)
command git "$@"
;;
esac
}
> We need to stop conflating elitism with fucked up design.
The design isn't fucked up, it is that you don't understand the model. This is okay. You aren't going to learn it unless you read docs or books on linux. If you learn the normal way, by usage, then it is really confusing at first. But there is a method to the madness. Things will start making more sense if you understand the reason for the design choices. (In a sibling comment I wrote the abstraction to command patterns that makes the gp's confusion odd. Because systemd follows the standard)
Side note: if you try to design something that works for everyone or works for the average person, you end up designing something that is BAD for most people. This is because people's preference is not uniformly distributed, meaning the average person is not representative of any person in the distribution. This is because anything that is normally distributed has its density along the shell while a uniform distribution has a uniform density all throughout it.
If every person has the same point of confusion than they are not the problem, it's the thing they're confused by.
There's better ways to do things and calling people naive for suggesting the obvious is the problem.
And about your side note: no. For example, when people checkout a branch, they want to track the remote branch 99.9% of the time. It should be the default.
The default journalctl should show where things have failed, that's why people are invoking it.
Also there's plenty of counterexamples that do these things. "ping host" genuinely pings the host. "ssh host" genuinely ssh's into the host.
You don't need to specify say, the encryption algorithm, that you want a shell, use say, "--resolve=dns" to specify the hostname resolution... It has sensible defaults that do what most people intend.
Under the model you advocate for "ssh host" would simply open up a random socket connection and then you'd have to manually attach a virtual terminal to it and request the invocation of shell separately, stacking each piece on top of the other, before you log in.
This design could be defended in the same way: Some people do port mapping, tunneling, SOCKS proxies, there's too many use cases! How can we assume the user wants a shell? Answer: because they do.
Most things are reasonable like certbot, apt, tune2fs, mkfs, awk, cut, paste, grep, sort, so many reasonable things. Even emacs is reasonable.
But systemd and git are not and the users are not the problems. Choices were made to be hostile to usability and they continue to be defended by being hostile to usability. Things like lex and yacc are inherently complicated and there's nothing to do there. Other things are intentionally complicated. Those can be fixed.
> The default journalctl should show where things have failed
How? What do you filter for? Emergency? Critical? Error? Alert? (see `dmesg -l`). What's the default? Do you do since boot? Since a certain time? Since restart?
FWIW, I invoke it all the time for other reasons. I am legitimately checking the status. Is it online? Do I have it enabled? What's the PID? Where's the file (though I can run `sudo systemctl edit foo.service`)? What's the memory usage? When did it start? And so on. The tail of the log are useful but not the point of status.
If I'm looking to debug a service I look at the journal instead. I hope this helps
# Show 10 most recent logs (you don't know what service to debug)
journalctl -r -n 10
# Show recent logs for a known service (can tab complete)
journalctl -r -n 10 --unit foo.service
# Some useful flags
-S, -U
Since and until. Time filtering including keywords {now, yesterday, today, tomorrow}
-b
Since boot. You can even do since a specific boot! Or even `--boot=-1` for the last 2 boots
-p, --priority
You can use the numbers 0-7 representing {emerg,alert,crit,err,warning,notice,info,debug} respectively
-g, --grep
No comment needed
-f, --follow
This is the tool while debugging since it'll update.
-e, --pager-end
Start at the end of the pager
> that's why people are invoking it.
That's why __you're__ using it, but don't assume that your usecase is the general. Remember, linux has a large breadth in types of users. But their biggest market is still servers and embedded systems. There's far more of those than PC users.
> Choices were made to be hostile to usability and they continue to be defended by being hostile to usability.
Idk, when systemd became the main thing I hated it too. But mostly because it was different and I didn't know how to use it. But then I learned and you know what? I agreed. This took awhile though and I had to see the problems they are solving. Otherwise it looks really bloaty and confusing. Like why have things like nspawn? Why use systemd jobs instead of using cron? Why use systemd-homed instead of useradd?
Well a big part of it is security and flexibility.
I write systemd services now instead of cron jobs. With a cron job I can't make private tmps[0]. Cron won't run a job if the computer is off during the job time. Cron can't stagger services. Cron can't wait for other services first. Cron can't be given limited CPU, memory, or other resource limitations.
Nspawn exists to make things highly portable. Chroot on steroids is used for a reason. Being able to containerize things, placing specific capabilities and all that. This is all part of a systemd job anyways. It really makes it a lot easier to give a job the minimum privileges. So often nspawn is a better fit than things like docker.
Same goes for homed. You can do things like setting timezones unique to users. But there's so much more like how it can better handle encryption. And you can do really cool things like move your home directory from one machine to another. This might seem like a non-issue to most people but it isn't. That whole paradigm where your keyboard is just an interface to a machine (i.e. a terminal, and I don't mean the cli. There's a reason that's called a terminal "emulator"). This is a really useful thing when you work on servers.
Look, there's a reason the distros switched over. It's not collective madness.
I'm not assuming I'm the default. I've worked countless jobs where I'm the "linux guy". These are the most common points of failure for everyone.
You have yet again showed how you are one of the 0.1%.
Why should the defaults be to accommodate for you?
I've been using linux for over 30 years. I'm in that group photo on the Debian.org homepage, given Linux conference talks spanning decades...
I know I'm in a very small minority and don't expect everyone to be me.
I work a support shift at a cloud hosting company and constantly deal with our customers struggling with linux on the same pain points over and over and over again. We have a bunch of auto-replies to explain things because everyone trips over the same broken sidewalk and people insist it's not broken but instead, it's beautiful design. Alright, whatever. Keeps me employed. Cool.
This is the same problem that mathematics has. Elite mathematicians insist that they should give zero shits about trying to explain things and their convoluted impenetrable explanations are elegant - it's an ultra-exclusive nerd club and most people aren't invited regardless of their interest.
Making things more approachable is somehow, cognitively unavailable to them or against some deeply held constitution. Whatever it is, it's not happening.
Some people think iOS is successful and some people think Mobian should be. I try to understand the first group and not insist everyone be in the second.
I've heard this reasoning multiple times, but that doesn't make it right. Like you said, you ARE presuming something: That scripting and a (useless) base case are the most common usage.
How many people script Git? More importantly, how often do you change that script? Rarely, and rarely. That means it's FAR FAR less work for the scripter to look up the weird incantations to remove the "human" parts (like a quiet flag).
Conversely, the human is far more likely to write git commands impromptu, and I dare say, this is the FAR FAR more common use case.
That means Git (and a lot of these kinds of commands) optimize for the RARE choice that happens infrequently over the common case that happens often. That's horrible design.
TL;DR: Just because there's a consistent design philosophy behind obtuse linux commands does not make a good, helpful, useful, modern, or simple philosophy. If a library author wrote code this way, we'd all realize how horrible that design is.
> That scripting and a (useless) base case are the most common usage.
You're thinking as a user. I'm sorry, but the vast majority of linux systems are still servers and embedded systems.
Here's two versions of the Unix Philosophy
- Write programs that do one thing and do it well.
- Write programs to work together.
- Write programs to handle text streams, because that is a universal interface.
- Make it easy to write, test, and run programs.
- Interactive use instead of batch processing.
- Economy and elegance of design due to size constraints ("salvation through suffering").
- Self-supporting system: all Unix software is maintained under Unix.
Yes, scripting is a common use case (I use git in scripts! I use git in programs. You probably do too, even if you don't know it). Piping is a common use case. And I'm not sure what we're talking about that is "useless". I've yet to run into a command where the default is useless. Even though I have `alias grep='grep --color=always --no-messages --binary-files=without-match'` I still frequently run `\grep`.
But we're also talking about programs that are DECADES old. git will turn 20 this year. You're going to be breaking a lot of stuff if you change it now.
And look, you don't have to agree, that's fine. But I'm trying to help you get into a mindset so that this doesn't look like a bunch of gobbledegook. You call these weird incantations. I get it. But if you get this model and get why things are the way they are, then those incantations become a lot less weird and a lot easier to remember. Because frankly, you'll have to remember far less. And either way, it's not like you're going to get linux changed. If you don't use it, great, move on. But if you do, then I'm trying to help, because if you can't change it you got to figure out the way.
Man, we're talking about git. It has never subscribed to this philosophy (and most programs in *nix don't)
Git does about fifteen thousand different things with about fifteen million flags.
And it was never designed. It was thrown together as a kitchen sink of useful things with commands adding new and crazy incantations as new things were added.
"Just write this script to do the useful thing" is a weird hill to die on
Fully this. For all its foibles, Linux was built to never presume too much, and its users tend to be power users who will almost certainly have dotfiles to tune their systems to their needs. In the context of making choices that will necessarily be universal, I admire how thoughtfully most standard Linux packages have been designed to never interfere with the users’ intentions.
Making things customizable doesn't mean the defaults should be useless. zsh, tmux, emacs, vim and bash, out of the box, for instance, have both pretty nice defaults and are highly customizable.
I know it's hard to make things like this but let's do it anyway.
And to all the linux noobies[0], you'll be a hell of a lot more efficient if you learn the philosophy of the design early. It will make it so that you can learn a new command and instantly know how to use several options. It will dramatically reduce the number of things you have to learn. I also HIGHLY suggest learning a bit of bash scripting.
Live in the terminal as much as you can. It is harder at first, but you will get huge boosts in productivity quicker than you would know it (easily <2 weeks). It sucks doing things "the hard way" but sometimes that comes with extra lessons. The thing is, those extra lessons are the real value.
[0] no matter how many years you've been using it there's no shame in being a noobie
None of those problems are unsolvable. The basic idea is: your terminal and script can get two different outputs. Git can already do that with colours, so it can also use more verbose for basic commands.
The other part is - scripts can use longer options while the console commands should be optimised for typing.
This has nothing to do with understanding the model or model itself. Complex things can have good interfaces - those just require good interface design and git did not get much of that.
> Git can already do that with colours, so it can also use more verbose for basic commands.
You already answered your own question. There's plenty of terminals that don't support colors. Plenty that don't support 16 bit colors. Do a grep into a curl and you'll find some fun shenanigans.
> This has nothing to do with understanding the model or model itself.
You're right. This has to do with you thinking your usecase is the best or most common.
As stated before, there is no average user[0]. So the intent is that you have the command that can be built upon. Is everyone unhappy with the base command? Probably. But is is always easier to build up than to tear down.
[0] Again, linux servers and linux embedded devices significantly out number users. Each of them do. So even if there were an average user, guess what, it wouldn't be you.
> There's plenty of terminals that don't support colors.
Colour support is irrelevant to this case. They can check isatty() and give different output depending on that.
And verbose information is better in average case, if it's well formatted. Unless you're flooding the screen with useless data of course, but that's not what's being talked about here.
In case you don't know, you can use the `-n` argument to `systemctl status` to tweak the log output, e.g. `-n0` to disable the log output and `-n40` to get more than the default 10 lines.
That's a great option to tweak the behavior and I hadn't known about it (or if I ever had, I'd well forgotten). Thanks!
From the man page it ?looks like? if you want reverse or full then it's still off to the journalctl command and arguments but at least "-n9999" is better than "always 10 lines".
Glad you like it! I'd rather learn to deal/work with the rough spots than stick my head in the sand with an alias and be even more lost when I don't have it or am working with someone else though. At least for the super common tools like git, systemctl, grep, and the like.
> I'd rather learn to deal/work with the rough spots than stick my head in the sand with an alias
I highly encourage you to learn the commands. But it seems like you did.
I would not call something so stylistic as `git branch -vv` as "sticking your head in the sand". Your fallback is not going to really confuse you or put you at risk of not knowing what's going on. I can totally understand if we were talking about aliasing something much more complicated, but at that point it's a script...
> My biggest annoyance is "systemctl status" gives you just enough of the service's log to make the output take up most of the terminal each time you run it but never enough of the service's log to get a useful picture of what's actually happened with the service lately.
In my experience, most of the times a service fails, it fails on startup (misconfiguration, missing resources). So the status is OK. Also, you can always request more lines, if you think that's helpful.
This wouldn't be my problem with systemd. Not by a long shot.
> I am annoyed by the redundant "systemd/system" directory name
It's not redundant, you also have /etc/systemd/user (and /lib/systemd/user) where units that run in the user context (as opposed to system-wide context) are stored.
I tried, but found it very confusing around the user session. Apparently an environment variable need to point to a started dbus.
I can't get consistent results over local session vs ssh vs su vs sudo
I used to do this but frankly it's easier to run a system-level unit as whatever user you want and keep all the files in /etc instead of scattered around /home.
The user-level units are most useful when running an actual multi-user system. If you trust your users to not abuse them, anyway.
I find that at least systemd means that it's consistent across distros. I spent way more time looking up this kind of thing when every distro rolled their own init system.
It is a bit weird. On the one hand, I understand that it makes sense to have [command] [verb] [object] on a "logical" level and that viewing logs should be a separate command (`journalctl`), but it is definitely not ergonomic.
Especially if you frequently have to switch between start/stop/restart.
> As a very occasional linux sysadmin just trying to make things work, the "typing at a console" systemd interfaces are not fun to work with. Maybe nobody should be doing that. In an enterprise, sure that's different. I think interfaces should be human, and linux should still be fun.
This was precisely the case for me. I "enjoy" playing around with systemd and am super interested in better understanding it, but the feedback loop just felt sooo slow. So hopefully this TUI can make it "fun" again :)
When I'm really working with a service and get tired of typing sudo systemctl restart or status all the time, I'll just do a quick alias or two right then and there.
alias s=sudo systemctl status
alias r=sudo systemctl restart
And if I'm only working with one service, I'll throw the service name in there too.
Fiddly and unintuitive? A lack of exposure to something does not really allow room for valid criticism of the thing.
Just a tip, if you remember the service name, status will show the directory the unit file is in. That will hopefully get you over your issues with directories. Complaining about Linux directories seems weird though. Have you looked at... anything else at all?
It is a bit crazy to me how everyone says “dont use cron systemd is in now” but cron just does what it says on the tin with no problems. I have lines that work fine ran in script or on my crontab but when wrapped in a launchd command no longer work (log says things work until the db is to be updated which tells me launchd ran processes lack sufficient permissions perhaps to update my db but its not clear why this is the case or how I can elevate launchd sufficiently.
> cron just does what it says on the tin with no problems.
I can name a rather large problem with cron that systemd timers solve handily: long-running job duplication. When jobs take longer to run than the space between their triggering times, duplicates start piling up. I've had to rescue numerous systems from such states, which are difficult to detect until things have gotten quite bad. Sure, you can write a bunch of boilerplate to handle this yourself with cron, but with systemd timers it's all handled for you along with other niceties like capturing all output in journald and ensuring that the next run starts at soon as possible.
Launchd is not used in systemd - why would anything be wrapped in it? The config is simple - User for the user and Group for the group. You can print it the result of "id" if you're not sure what the result is.
> ...cron just does what it says on the tin with no problems.
Yeah, tell me about it.
We had a production-down support ticket filed for one of the things that I work on at $DAYJOB. The customer's VM's disk was full. Why? Because the 'timer unit' that was supposed to run logrotate every day had never been run and was never scheduled to run. (The VM had been up for a month at this point.) No other customers had ever reported this issue, and we'd not changed anything about that timer unit or what it is supposed to run in ages.
No amount of gyration or agitation with 'systemctl' and friends OR rebooting of the VM kicked the cron replacement into proper functioning. 'logrotate' was simply never being scheduled to run. This fucker was WEDGED, and the tools weren't helping us understand why.
We read through all the docs on all the various kinds of units and don't see what we're doing wrong. We do a BUNCH of digging, and find an -IIRC- open Github issue from years back where someone was running into this problem. More or less the last word on the issue was Poettering saying something like "Well... actually, now that we've said that that particular cascade of options isn't actually supposed to work, now I'm not so sure that they're NOT supposed to work.". And that was -apparently- that.
IMO, when your cron replacement can be easily configured in such a way as to never even try to run the thing that you scheduled it to run, you really need to go back and rethink how you've built your cron replacement.
Considering how many things of questionable veracity are said about systemd in general, and Lennart in particular, I would like a link to that issue before I take your characterization at face value.
Yeah, and you can omit the ".service" suffix, and you could use ".timer" suffix, too, if you have a timer for that service.
As much as I do not like systemd, I do not think its commands are an issue, and I use "systemctl status" and "journalctl" as well with some flags at times.
I think this isn't shoddy engineering; I think you disagree with a design tradeoff that was consciously made here.
If CLIs are designed around facilitating the easiest means of editing the most commonly-edited word, different programs will end up with semantically very different CLIs. In some programs, the "noun" (e.g. file path) will be the thing most frequently edited. In some others, like systemctl, it'll be the "verb", like start/stop. That means that different programs designed around this principle will be semantically extremely inconsistent with each other.
On the other hand, if consistency of "base_command sub_command --subcommand-arg sub_sub_command --option argument" is taken as the guiding principle, many different tools will act semantically similarly. This enables unfamiliar users (talking first-time users or people who have just opened the manpage for the first time) to intuit the right way to present commands to those tools, rather than expecting them to memorize (or re-look-up as needed) each one's specific semantics.
While there's merit to both, I think the second one--systemd's CLI approach--is better. Put another way: the user-centric and consistent (with other applications) design are sometimes at odds in this way. A tool that is hyper-focused on optimizing the most common tasks for its (power) users risks oddball status and hostility to new users. It's important to pick a point on the spectrum that appropriately balances those two concerns. Python ("There should be one-- and preferably only one --obvious way to do it") and the Apple HIG ecosystem both understand the truth of this: that it is sometimes necessary to trade away efficiency for familiarity/consistency. There's a reason Perl languished while Python grew.
Like, I get it. I've been a sysadmin/devops for decades, and the paper cuts add up. But it's easy to forget the paper cuts I'm not getting: modern tools are (generally, with exceptions) more consistent; there are fewer commands that are resistant to memorization and need to be looked up each time; fewer commands that lead to questions like "wait, is it 'cmd help subcommand', 'cmd --help subcommand', or 'cmd subcommand --help'? Am I going to have to Google how to get the help output of this command?"
Cause you can manipulate multiple services in a single command? Not a big deal when you're looking at one or two services but when you've got a 5 to 15 interlinked and interdependent services and in situations where you may not want to restart everything, it's nice to just have one CLI with tab completion to deal with that.
Ah yes, famously one can only be a sysadmin if they're unable to use a different cli verb order.
Come on now. Either present a real argument or accept the fact that tooling isn't forever going to be frozen to what you used in your 20's. Newer tooling uses newer best practices and the improved verb order is part of that.
I work in a company that creates (admittedly crappy) product for sysadmins (I have no say in design though, so don't blame that on me). I work in the infra / automation, so close to sysadmin, but not quite.
So, this isn't a foreign world to me... I still don't think that:
1. Starting a service is a big issue in general (most of the time this happens automatically anyways).
2. The command syntax of systemcl is defective in any way. Of all things that might be problematic with systemd, this is just not in the list at all. It's such a bizarre complaint to have... kind of like arguing whether "gray" or "grey" is the right way to spell it. Really, it's a whatever. There are much larger issues, even when it comes to comfort, not functionality of systemd tools. Like, for example, the autogenerated names of device units. I hate trying to recall the rules for writing these, and on top of that, having to escape characters just to type the unit's name. That's just a ridiculous design.
One thing I found systemd really confusing was its treatment of ExecStop in a service script. ExecStart is the command to run when systemd starts the service at system boot up (or when a user tells systemd to start the service). However, ExecStop is run when the starting command has finished running. You have to set RemainAfterExit=yes to have the desired function of running the stop command on system shutdown or on user stopping the service. ExecStop is basically the "on-cleanup" event rather than "to-shutdown-the-service" event.
It is important to keep in mind that systemd is tailored towards daemons. So if your service just runs a command that eventually exits, you need to explicitly tell systemd to treat it differently than a daemon.
Edit: As others noted, you’re probably looking for oneshot + RemainAfterExit.
- on start: start the server
- on stop: do nothing, because you are already terminating the server
But suppose you need to perform an additional task when the server is terminated. That is where you would add a ExecStop command or script.
All I am trying to say is that the name of the option makes sense as-is: it literally runs the provided command(s) on service stop (“execute on stop”).
Similarly, ExecStart literally means command(s) to execute on start.
If the command runs a blocking daemon/server (usually the case), then the server will be implicitly stopped by definition - because if you’re stopping the systemd service, you’re interrupting any commands that are still running/blocking.
And relatedly, I honestly have no idea when I’d want to use ExecStartPre, or multiple ExecStarts, or ExecStartPost, and so on.
That’s what is assumed. But in reality it runs after the started process stops.
That makes sense to me personally. What would be the more intuitive design in your mind?
Maybe think about it this way: ExecStart is what the system will run to transition the service from the "starting" state to the "started" state. ExecStop is what the system will run to transition the service from the "stopping" state to the "stopped" state.
For a service with RemainAfterExit=no (the default), you enter the stopping state right away once the processes that got started in ExecStart exit. That's useful when you are starting some long lived process as a service, and in that case there is usually no need for an ExecStop. But semantically, ExecStop has the same meaning either way -- it's what needs to be run, if anything, to transition the service from the stopping state to the stopped state.
It definitely seems to be both "cause to stop" and "after (unexpected) stop" in one. You can look at $MAINPID to see which case you have. This design apparently makes sense to you, but to me and several others in this thread a service that has already stopped isn't in need of being stopped and shouldn't execute commands intended for that. (There is a separate ExecStopPost for "after stopping, for any reason".)
I have several user and system level services I manage, but debugging them is tedious. Your opening line that lists common commands and their pain points really resonated with me.
I’m on NixOS, so editing immutable unit files directly won’t work, but the service discovery, visibility, and management will be really helpful. Nice work!
I am planning on adding some "guides" in the documentation but in short: You should check out `systemctl edit --runtime` for debugging units on NixOS. It makes debugging sooo much easier.
https://github.com/rgwood/systemctl-tui
Just one thing: I had to do
instead of and uvx wouldn't work at all, version: uv 0.5.21. That said, uv is way more quick than pip(x) so I just switched.Hey, that's me ! (And I love systemd !)
I haven't installed it yet so quick question: can it connect to remote host ? I often use systemctl --host <hostname> status foo.service (status, timers, logs etc. )
Either way, feel free to open an issue and I will have a look at it.
Sounds like you could easily support it by letting users pass in $REMOTE_HOST and when you use your `systemctl` wrapper, add `$CMD --host=$REMOTE_HOST`, after that everything should work as before.
https://github.com/isd-project/isd/issues/6
What's missing in the install routine is uv installing this tool ignoring the Python dependency. My box has 3.10 and isd won't work with it. Fixed with `-p 3.13` option. May be worth mention in the docs.
It should be installable locally, and run commands on remote machine via ssh! And via 'docker exec' commands.
Is this really true? I understand why it does not work on NixOS (I tried just out of curiosity and it seems like it is unable to exec the host systemctl for some reason) but I don't think there's any reason it wouldn't work on other OSes that merely have Nix installed.
Interestingly though, on Nix v2.24.11, I can't use the provided Nix command either:
Even if that did work (you could adjust it into a Git URL to make it work) it would probably not be ideal since Nix has a native GitHub fetcher that is more efficient. I think this should be the actual Nix command: Anyway, this is cool. I actually wanted to make a similar thing using systemd's DBus API and Qt instead of a TUI and even started writing code for it, and if you wanted to I'm sure you'd find that the DBus API probably provides all of the functionality you would need (admittedly it is a lot easier to just call `systemd -H` than to implement your own SSH tunneling, though.) It kind of frustrates me that systemd and modern Linux in general is absolutely teeming with data and interfaces that could be exposed and make administering systems, especially desktop systems that were traditionally very inscrutable, much easier. e.g. in the past, how did you know what was going on when an xdg autostart app failed? Now with systemd running xdg autostart apps in some desktops, it would be really easy to provide a GUI that can show you the failed autostarts and even provide a GUI log viewer, and yet somehow, such a tool does not seem to exist, at least in the realm of things that are maintained and relatively feature-complete. Rather frustrating.Yeah, it get's complicated and I don't want to recommend it and explain the details. In short, I am creating the AppImage via nix. And the AppImage "mounts" (not overlays!) the AppImage's /nix/ directory. So calls from the TUI that would access /nix/ wouldn't go to the systems `/nix` directory, which leads to all kind of weird issues. For example, you could install your EDITOR via home-manager on Ubuntu. isd would start correctly because systemctl is "accessible" but if you open your EDITOR under `/nix` it wouldn't find it, which is super confusing as a user. -> So it is just easier to say to use the nix installation method if you are already using it :D
And sorry for the wrong docs, it is fixed now.
I also agree with your frustration. Personally, I would really enjoy working on such a tool but it wouldn't be an easy task, and who would support the work? This TUI had a manageable scope but it was still quite a bit of work. So I don't see myself investing too much into "higher-level interfaces".
PS: I have no idea why your post is ill-received :/
That makes a lot of sense. There's not really much trivial that I think you can do to deal with that.
> PS: I have no idea why your post is ill-received :/
It's not really a big deal, I was just surprised and wondered if anyone who was compelled to vote down might come back to explain why.
I agree Linux could use better system APIs than "put file here" and "run these commands" which are much more error prone than making calls to properly documented interfaces.
And sure, systemd it's more deterministic and includes the kitchen sink, unlike initd.
Thankfully these days I can automate most of such interactions out of existence, so I no longer feel the burning hatred that I once did. More like a smoldering ember.
https://isd-project.github.io/isd/security/
Let me know if anything is missing!
Where I work we also use defectdojo to catalogue and manage CVRs in our projects, but it's more involved to setup the testing pipeline and deploy the required services.
If you add a sane cli with tab completion support, it’ll come full circle.
May make it easier to customize but it doesn't close the security loopholea like SELinux, GRSecurity, TOMOYA, or AppArmor does.
https://www.cyberciti.biz/tips/selinux-vs-apparmor-vs-grsecu...
Now for many that threat model is not sufficient as they both run increasingly less trustworthy software, obtained by less trustworthy mechanisms such as npm or off a website, or simply want to protect against bugs that cause otherwise non malicious software from being compromised and resulting in security incidents. I'm in this latter camp but we can't ignore the fact that there are many who happily operate in the former. There also exist solutions such as we browsers with their sandboxes and VMs that somewhat fill the requirements for running untrusted software for these individuals.
Which is a fanciful way of saying that I don't understand the relevance of your comment at all to the topic at hand, which is an interactive frontend.
My only problem is that I wish there were a way to install it on my machine and have it connect to a remote systemctl but that probably is a lot of work to reliably work (port may not be open etc etc).
Some feedback :
- it is.. relatively slow ? especially when focusing on different panes (tab/shift+tab). on my machine it takes at least half a second to react
- the unit list is missing page-up/page-down handling
- in some unit attributes, the ordering of some values frequently changes (for example, on unstarted services, in `TriggeredBy`)
- it could be interesting to integrate the output of `systemd-analyse security`
Nice work !
- Jep, page up/down handling will be added. Thanks!
- Not quite sure when this happens. Maybe an example screen shot (before/after) could help?
- Yeah, that is definitely on the roadmap.
Though I don't agree with all design decisions, and there are some bugs, having such well-maintained documentation and clean repo makes it easy to dive into the project and understand what is going on and how other components are built.
Thanks for the library!
Can it connect to remote hosts like you can with systemctl --host?
I've put quite a lot of work into it and wanted the project to give a good first impression. This was important to me since I believe these "niche" TUI libraries need to immediately show what problems they are solving or how they save time, as users need a reason to _use_ an additional abstraction layer.
Though I am by far no security specialist. Please let me know where I can improve the section!
Check the -H and respective -M flags of many systemd CLI utilities.
> /etc/systemd/system/<unit>.service.d
I've used PartOf to enforce "dependency". Lots of other ways to enforce order and dependency once you have the override in place.
Please open an issue on GitHub if you encounter any silly alpha issues.
foo restart vs restart foo
Not gonna lie, I am not a big fan of Python anymore because of the dependency hell you can run into when working with CUDA libraries but uv is a breath of fresh air and textual is just sooo easy.
Don't get me wrong, I am not saying that I would never consider rewriting it in Rust or Go, but the documentation and guides from Textual were great resources and the creator, Will, also seems like a really nice guy :D
Edit: To be explicit, I believe that Python has currently the most advanced and accessible TUI ecosystem. At least, that was my impression after checking a few examples on an afternoon.
1. My life improved a lot after I found that you can do "systemctl status $PID" and systemd will find what service (if any) is responsible for the process in question. This has been a life saver many, many times. But, more search would still be welcome, especially for cases when the system fails to boot, or fails to reach a particular target etc.
2. I think systemd didn't go far enough with unit files. The motivation was to escape the hell of Shell scripts, where each system was defined in its own unique way, and was failing in a dozen of unique ways. While, initially, it might have seemed that a simple INI-style format could manage to describe service requirements... I think, it's way overdue to realize that it doesn't. And sysadmins on the ground "fix" that by embedding more Shell into these configuration files, bringing us back to the many unique ways a service will fail. Perhaps, having a way to edit these unit files so that it doesn't expose the actual format may lead to improvement in the format (more structure, more types, templates).
And the feedback is so bad. It should know everything in its own config dir and tell me how to do what I want to do. Was it enabled? I forget. How do I look at logs? Oh right journalctl. Also the layout of things with lots of symlinks and weird directories in places that annoy my 90's linux sysadmin brain. Why am I looking at /lib/systemd/system
I am annoyed by the redundant "systemd/system" directory name every time I have to go there. At this point, just promote it to /etc/systemd and build a better CLI.
As a very occasional linux sysadmin just trying to make things work, the "typing at a console" systemd interfaces are not fun to work with. Maybe nobody should be doing that. In an enterprise, sure that's different. I think interfaces should be human, and linux should still be fun.
I don't get this complaint. It's the same order as almost every other command-line utility that has subcommands: <command> <subcommand> <thing to operate on>. To me, that kind of consistency is very intuitive.
I'll add the abstraction for anyone confused
Anything in brackets is optional and might not appear or be available. By command I mean a category of commands. Such as 'pip install' vs 'pip uninstall', which are sub-programs inside the main program. But this can have layers such as 'uv pip install'. Often flags can be used in any order because you'll just loop over all the arguments but this is still the standard order.There's also the two actor pattern
Also, it looks like you are one of today's lucky 10,000 ! https://xkcd.com/1053/
systemctl status myapp mydb
Hence the verb-last CLI rulez!
Something like:
systemctl --stop myapp mydb
systemctl myapp mydb --stop
^art^op
Converts the "start" in "foo start bar" to "stop", ie runs "foo stop bar". Append :p to do the substitution but print the command instead of running it.
[1] https://www.gnu.org/software/bash/manual/html_node/Word-Desi...
[2] https://www.johndcook.com/blog/2022/10/21/shell-replace-word...
(Kidding aside, quick reminder that on Mac you have to enable Settings>Profiles>Keyboard>Use Option as Meta key, or else Alt doesn't work)
Because I have to share commands with other people who are troubleshooting their own systems, and copy/paste from history becomes useless if I have specific aliases.
Because someday I or someone will want to script these interactions, and aliases are not available in subprocesses.
I'm used to `service` command, so I have muscle memory which `systemd` breaks. The way `systemd` commands are laid out is better, it just messes with `service` command muscle memory.
With initV, issuing many command in sequence to the same service (enable, then start… stop then start… etc) is much quicker and easier as it’s “up-arrow, control-W, type new command. With the systemd option I either need to navigate one word to the left (or more depending on how the shell is configured wrt word separation) before I replace the command, or I need to delete both the command and the service name, and retype more.
It’s not a huge difference but small downgrades in ergonomics add up over time.
A backward step in intuitive CLI usage.
We can be such a fuddy-duddy gray-beards.
Too bad it doesn't have a '--status 2' flag to show the status after two seconds too.
Many (most?) of us are all over the place, expected to “wear many hats” and aren’t in a single IDE or language all day every day. Certainly if I were a “Linux admin” I would have systemd pretty committed to memory. Anyone else probably wouldn’t.
* /etc/init.d/my-service stop
and Ubuntu’s:
* service my-service stop
both lurk in my brain.
And yet my fingers still want to type it the other way.
Ah well, it's all been said before.
It took me ages to unlearn that pattern for using systemctl, even though as you say: it's far more consistent
sudo service my-service stop
Not to mention unless the problem with the service completely prevented it from running (it advises some commands to run in that case) you're supposed to just always remember "journalctl -xeu $SERVICE" was the incantation, less you want to go look up the flags again or manually parse the entire "journalctl" output.
Overall I generally like systemd though. The syntax can just be a burden sometimes.
Instead, these invocations give cryptic messages, throw errors, or sometimes, even break things.
The most common and helpful things are hidden deep behind multiple flags and command line arguments in manuals that read like dictionaries more than guides.
I'm always at a complete loss as to how such decisions are made. For instance, "git branch -vv" is the useful output you would like to see, every time, that should be "git branch". Why not make the current output, "git branch -qq"? Is a humane interface too much to ask for? Apparently...
I know people defend this stuff, but as a senior engineer in programming pits for 30 years, they're wrong. Needless mistakes and confusions are the norm. We can do better.
We need to stop conflating elitism with fucked up design.
You have the wrong mentality, and I hope this can help make your life easier. Programs are made so that the simplest option is the base option. This is because there is a high expectation that things will be scripted AND understanding that there is a wide breadth of user preference. There's an important rule
Customization is at the root of everything. We have aliases that solve most of the problems and small functions for everything else. You default to an non-noisy output, showing only __essentials__ and nothing more unless asked. Similarly, you do no filtering other than hidden files. This way, everyone can get what they want. Btw, this is why so many people are upset with default options on things like fdfind and ripgrep.For your problem with git, there are 2 solutions you have.
The design isn't fucked up, it is that you don't understand the model. This is okay. You aren't going to learn it unless you read docs or books on linux. If you learn the normal way, by usage, then it is really confusing at first. But there is a method to the madness. Things will start making more sense if you understand the reason for the design choices. (In a sibling comment I wrote the abstraction to command patterns that makes the gp's confusion odd. Because systemd follows the standard)Side note: if you try to design something that works for everyone or works for the average person, you end up designing something that is BAD for most people. This is because people's preference is not uniformly distributed, meaning the average person is not representative of any person in the distribution. This is because anything that is normally distributed has its density along the shell while a uniform distribution has a uniform density all throughout it.
If every person has the same point of confusion than they are not the problem, it's the thing they're confused by.
There's better ways to do things and calling people naive for suggesting the obvious is the problem.
And about your side note: no. For example, when people checkout a branch, they want to track the remote branch 99.9% of the time. It should be the default.
The default journalctl should show where things have failed, that's why people are invoking it.
Also there's plenty of counterexamples that do these things. "ping host" genuinely pings the host. "ssh host" genuinely ssh's into the host.
You don't need to specify say, the encryption algorithm, that you want a shell, use say, "--resolve=dns" to specify the hostname resolution... It has sensible defaults that do what most people intend.
Under the model you advocate for "ssh host" would simply open up a random socket connection and then you'd have to manually attach a virtual terminal to it and request the invocation of shell separately, stacking each piece on top of the other, before you log in.
This design could be defended in the same way: Some people do port mapping, tunneling, SOCKS proxies, there's too many use cases! How can we assume the user wants a shell? Answer: because they do.
Most things are reasonable like certbot, apt, tune2fs, mkfs, awk, cut, paste, grep, sort, so many reasonable things. Even emacs is reasonable.
But systemd and git are not and the users are not the problems. Choices were made to be hostile to usability and they continue to be defended by being hostile to usability. Things like lex and yacc are inherently complicated and there's nothing to do there. Other things are intentionally complicated. Those can be fixed.
FWIW, I invoke it all the time for other reasons. I am legitimately checking the status. Is it online? Do I have it enabled? What's the PID? Where's the file (though I can run `sudo systemctl edit foo.service`)? What's the memory usage? When did it start? And so on. The tail of the log are useful but not the point of status.
If I'm looking to debug a service I look at the journal instead. I hope this helps
That's why __you're__ using it, but don't assume that your usecase is the general. Remember, linux has a large breadth in types of users. But their biggest market is still servers and embedded systems. There's far more of those than PC users. Idk, when systemd became the main thing I hated it too. But mostly because it was different and I didn't know how to use it. But then I learned and you know what? I agreed. This took awhile though and I had to see the problems they are solving. Otherwise it looks really bloaty and confusing. Like why have things like nspawn? Why use systemd jobs instead of using cron? Why use systemd-homed instead of useradd?Well a big part of it is security and flexibility.
I write systemd services now instead of cron jobs. With a cron job I can't make private tmps[0]. Cron won't run a job if the computer is off during the job time. Cron can't stagger services. Cron can't wait for other services first. Cron can't be given limited CPU, memory, or other resource limitations.
Nspawn exists to make things highly portable. Chroot on steroids is used for a reason. Being able to containerize things, placing specific capabilities and all that. This is all part of a systemd job anyways. It really makes it a lot easier to give a job the minimum privileges. So often nspawn is a better fit than things like docker.
Same goes for homed. You can do things like setting timezones unique to users. But there's so much more like how it can better handle encryption. And you can do really cool things like move your home directory from one machine to another. This might seem like a non-issue to most people but it isn't. That whole paradigm where your keyboard is just an interface to a machine (i.e. a terminal, and I don't mean the cli. There's a reason that's called a terminal "emulator"). This is a really useful thing when you work on servers.
Look, there's a reason the distros switched over. It's not collective madness.
[0] https://www.redhat.com/en/blog/new-red-hat-enterprise-linux-...
https://systemd.io/
You have yet again showed how you are one of the 0.1%.
Why should the defaults be to accommodate for you?
I've been using linux for over 30 years. I'm in that group photo on the Debian.org homepage, given Linux conference talks spanning decades...
I know I'm in a very small minority and don't expect everyone to be me.
I work a support shift at a cloud hosting company and constantly deal with our customers struggling with linux on the same pain points over and over and over again. We have a bunch of auto-replies to explain things because everyone trips over the same broken sidewalk and people insist it's not broken but instead, it's beautiful design. Alright, whatever. Keeps me employed. Cool.
This is the same problem that mathematics has. Elite mathematicians insist that they should give zero shits about trying to explain things and their convoluted impenetrable explanations are elegant - it's an ultra-exclusive nerd club and most people aren't invited regardless of their interest.
Making things more approachable is somehow, cognitively unavailable to them or against some deeply held constitution. Whatever it is, it's not happening.
Some people think iOS is successful and some people think Mobian should be. I try to understand the first group and not insist everyone be in the second.
How many people script Git? More importantly, how often do you change that script? Rarely, and rarely. That means it's FAR FAR less work for the scripter to look up the weird incantations to remove the "human" parts (like a quiet flag).
Conversely, the human is far more likely to write git commands impromptu, and I dare say, this is the FAR FAR more common use case.
That means Git (and a lot of these kinds of commands) optimize for the RARE choice that happens infrequently over the common case that happens often. That's horrible design.
TL;DR: Just because there's a consistent design philosophy behind obtuse linux commands does not make a good, helpful, useful, modern, or simple philosophy. If a library author wrote code this way, we'd all realize how horrible that design is.
Here's two versions of the Unix Philosophy
Yes, scripting is a common use case (I use git in scripts! I use git in programs. You probably do too, even if you don't know it). Piping is a common use case. And I'm not sure what we're talking about that is "useless". I've yet to run into a command where the default is useless. Even though I have `alias grep='grep --color=always --no-messages --binary-files=without-match'` I still frequently run `\grep`.But we're also talking about programs that are DECADES old. git will turn 20 this year. You're going to be breaking a lot of stuff if you change it now.
And look, you don't have to agree, that's fine. But I'm trying to help you get into a mindset so that this doesn't look like a bunch of gobbledegook. You call these weird incantations. I get it. But if you get this model and get why things are the way they are, then those incantations become a lot less weird and a lot easier to remember. Because frankly, you'll have to remember far less. And either way, it's not like you're going to get linux changed. If you don't use it, great, move on. But if you do, then I'm trying to help, because if you can't change it you got to figure out the way.
https://en.wikipedia.org/wiki/Unix_philosophy
Man, we're talking about git. It has never subscribed to this philosophy (and most programs in *nix don't)
Git does about fifteen thousand different things with about fifteen million flags.
And it was never designed. It was thrown together as a kitchen sink of useful things with commands adding new and crazy incantations as new things were added.
"Just write this script to do the useful thing" is a weird hill to die on
I know it's hard to make things like this but let's do it anyway.
Take a look at the manual
and bookmark "Bash Pitfalls" Live in the terminal as much as you can. It is harder at first, but you will get huge boosts in productivity quicker than you would know it (easily <2 weeks). It sucks doing things "the hard way" but sometimes that comes with extra lessons. The thing is, those extra lessons are the real value.[0] no matter how many years you've been using it there's no shame in being a noobie
The other part is - scripts can use longer options while the console commands should be optimised for typing.
This has nothing to do with understanding the model or model itself. Complex things can have good interfaces - those just require good interface design and git did not get much of that.
As stated before, there is no average user[0]. So the intent is that you have the command that can be built upon. Is everyone unhappy with the base command? Probably. But is is always easier to build up than to tear down.
[0] Again, linux servers and linux embedded devices significantly out number users. Each of them do. So even if there were an average user, guess what, it wouldn't be you.
Colour support is irrelevant to this case. They can check isatty() and give different output depending on that.
And verbose information is better in average case, if it's well formatted. Unless you're flooding the screen with useless data of course, but that's not what's being talked about here.
From the man page it ?looks like? if you want reverse or full then it's still off to the journalctl command and arguments but at least "-n9999" is better than "always 10 lines".
And IIRC you can change the default behavior. In the worst case, just alias it if it is bothering you that much.
I would not call something so stylistic as `git branch -vv` as "sticking your head in the sand". Your fallback is not going to really confuse you or put you at risk of not knowing what's going on. I can totally understand if we were talking about aliasing something much more complicated, but at that point it's a script...
How about
systemctl status foo | tail
This wouldn't be my problem with systemd. Not by a long shot.
It's not redundant, you also have /etc/systemd/user (and /lib/systemd/user) where units that run in the user context (as opposed to system-wide context) are stored.
If you don't need elevated permissions this is ideal.
All you have to do is enable linger using loginctl if you want your service to auto start as the user on boot unattended.
Your user services live in ~/.config/systemd/user
The user-level units are most useful when running an actual multi-user system. If you trust your users to not abuse them, anyway.
It is a bit weird. On the one hand, I understand that it makes sense to have [command] [verb] [object] on a "logical" level and that viewing logs should be a separate command (`journalctl`), but it is definitely not ergonomic. Especially if you frequently have to switch between start/stop/restart.
> As a very occasional linux sysadmin just trying to make things work, the "typing at a console" systemd interfaces are not fun to work with. Maybe nobody should be doing that. In an enterprise, sure that's different. I think interfaces should be human, and linux should still be fun.
This was precisely the case for me. I "enjoy" playing around with systemd and am super interested in better understanding it, but the feedback loop just felt sooo slow. So hopefully this TUI can make it "fun" again :)
alias s=sudo systemctl status alias r=sudo systemctl restart
And if I'm only working with one service, I'll throw the service name in there too.
Just a tip, if you remember the service name, status will show the directory the unit file is in. That will hopefully get you over your issues with directories. Complaining about Linux directories seems weird though. Have you looked at... anything else at all?
Give me files for my logs, give me a single place for my service definitions, be simple.
After years of systemd, I have never really "got it", while it took a weekend for runit.
It's not always our choice, corporate world is something else, I know...
I can name a rather large problem with cron that systemd timers solve handily: long-running job duplication. When jobs take longer to run than the space between their triggering times, duplicates start piling up. I've had to rescue numerous systems from such states, which are difficult to detect until things have gotten quite bad. Sure, you can write a bunch of boilerplate to handle this yourself with cron, but with systemd timers it's all handled for you along with other niceties like capturing all output in journald and ensuring that the next run starts at soon as possible.
Yeah, tell me about it.
We had a production-down support ticket filed for one of the things that I work on at $DAYJOB. The customer's VM's disk was full. Why? Because the 'timer unit' that was supposed to run logrotate every day had never been run and was never scheduled to run. (The VM had been up for a month at this point.) No other customers had ever reported this issue, and we'd not changed anything about that timer unit or what it is supposed to run in ages.
No amount of gyration or agitation with 'systemctl' and friends OR rebooting of the VM kicked the cron replacement into proper functioning. 'logrotate' was simply never being scheduled to run. This fucker was WEDGED, and the tools weren't helping us understand why.
We read through all the docs on all the various kinds of units and don't see what we're doing wrong. We do a BUNCH of digging, and find an -IIRC- open Github issue from years back where someone was running into this problem. More or less the last word on the issue was Poettering saying something like "Well... actually, now that we've said that that particular cascade of options isn't actually supposed to work, now I'm not so sure that they're NOT supposed to work.". And that was -apparently- that.
IMO, when your cron replacement can be easily configured in such a way as to never even try to run the thing that you scheduled it to run, you really need to go back and rethink how you've built your cron replacement.
Was not aware that vixie cron was first released 1987(https://github.com/vixie/cron/blob/master/Documentation/Chan... 1.md), and still has fresh commits.
As much as I do not like systemd, I do not think its commands are an issue, and I use "systemctl status" and "journalctl" as well with some flags at times.
Bring back the verb-last CLI.
Don't be suggesting `^op^art` over up-CtrlW This is worse than pacman.If CLIs are designed around facilitating the easiest means of editing the most commonly-edited word, different programs will end up with semantically very different CLIs. In some programs, the "noun" (e.g. file path) will be the thing most frequently edited. In some others, like systemctl, it'll be the "verb", like start/stop. That means that different programs designed around this principle will be semantically extremely inconsistent with each other.
On the other hand, if consistency of "base_command sub_command --subcommand-arg sub_sub_command --option argument" is taken as the guiding principle, many different tools will act semantically similarly. This enables unfamiliar users (talking first-time users or people who have just opened the manpage for the first time) to intuit the right way to present commands to those tools, rather than expecting them to memorize (or re-look-up as needed) each one's specific semantics.
While there's merit to both, I think the second one--systemd's CLI approach--is better. Put another way: the user-centric and consistent (with other applications) design are sometimes at odds in this way. A tool that is hyper-focused on optimizing the most common tasks for its (power) users risks oddball status and hostility to new users. It's important to pick a point on the spectrum that appropriately balances those two concerns. Python ("There should be one-- and preferably only one --obvious way to do it") and the Apple HIG ecosystem both understand the truth of this: that it is sometimes necessary to trade away efficiency for familiarity/consistency. There's a reason Perl languished while Python grew.
Like, I get it. I've been a sysadmin/devops for decades, and the paper cuts add up. But it's easy to forget the paper cuts I'm not getting: modern tools are (generally, with exceptions) more consistent; there are fewer commands that are resistant to memorization and need to be looked up each time; fewer commands that lead to questions like "wait, is it 'cmd help subcommand', 'cmd --help subcommand', or 'cmd subcommand --help'? Am I going to have to Google how to get the help output of this command?"
apt-get/yum/dnf install foo git add file
And the normal CLI utilities: rm/touch/cat/ls/mv/cp etc are all verbs, and they act on nouns that come last.
There are so many ways to recall the command you typed from history... I don't understand your complains at all.
Come on now. Either present a real argument or accept the fact that tooling isn't forever going to be frozen to what you used in your 20's. Newer tooling uses newer best practices and the improved verb order is part of that.
So, this isn't a foreign world to me... I still don't think that:
1. Starting a service is a big issue in general (most of the time this happens automatically anyways).
2. The command syntax of systemcl is defective in any way. Of all things that might be problematic with systemd, this is just not in the list at all. It's such a bizarre complaint to have... kind of like arguing whether "gray" or "grey" is the right way to spell it. Really, it's a whatever. There are much larger issues, even when it comes to comfort, not functionality of systemd tools. Like, for example, the autogenerated names of device units. I hate trying to recall the rules for writing these, and on top of that, having to escape characters just to type the unit's name. That's just a ridiculous design.