Push Notifications for Decentralized Services

(unifiedpush.org)

146 points | by notmine1337 4 days ago

14 comments

  • orblivion 20 hours ago
    I was surprised how easy it was to integrate. I'm on GrapheneOS/FDroid. I installed the ntfy app and connected it to my ntfy server. I wanted to set it up for Tusky (Mastodon app). I was confused about the lack of instructions. What do I set on Tusky? What do I set on all my Mastodon servers? It turned out there were no instructions. I was already configured for Tusky. Tusky found the ntfy app, got my ntfy server's URL from it, and passed it on to the Mastodon servers.

    Now, I actually think this is a little too easy. What if I want to keep my ntfy server private for certain at-home uses? But still, very impressive. Element does it right, it at least asks you when you open the app (or it's in a setting or something). But I think UnifiedPush or ntfy instructions should make these steps more clear (or I failed to find it), even if just to say "it's actually that easy" to save our time looking.

    BTW I'm working on porting ntfy to Sandstorm. Already works, I just have a handful of loose ends.

    • ryukafalz 18 hours ago
      > BTW I'm working on porting ntfy to Sandstorm.

      Oh this would be really handy, I'd love to run it on my Sandstorm server!

      • orblivion 18 hours ago
        Awesome.

        One major caveat is that because of Sandstorm's draconian security (I mean this in a good way), a handful of ntfy's features won't work because they use special headers. Mastodon works so I'm assuming they use the json API. One of my tasks is to go through major services and test them.

        Another one is protected topics. "Users" use auth headers (there's a GET string option, but the Android app source code indicates that it opts for headers). Sandstorm does give you the benefit of randomized API URLs that you can revoke, which protects you from random actors, but any service (Mastodon, Matrix, etc) you connect to will need to see that URL so they may in principle snoop on easy-to-guess topics.

        If you're curious you can see my tangled mess of notes: https://github.com/orblivion/ntfy/blob/sandstorm/.sandstorm/... Don't get too invested in understanding it if it's confusing, it's a fucking mess.

        • EGreg 15 hours ago
          What would you say the main points are for sandstorm?

          Is it about hosting apps on different domains and using a powerbox between them? Or is it something more? Why does sandstorm use capnproto for instance?

          In our framework for instance, instead of capabilities, we implemented "invites", which people can send to each other. If an invite grants you readLevel access to a document, you'll be able to see it even before you log in and create a user account. But if you want to get the full access to write etc, you'll have to create a user account and accept the invite as that user.

          https://github.com/Qbix/Streams/blob/main/classes/Streams/St...

          • ferfumarma 11 hours ago
            I'm your example it sounds like invites are a form of capability based security, which is what sandstorm uses.

            I'm not an expert, but my understanding is that sandstorm takes all of the system calls from your apps, and uses capn proto to serialize those calls to the powerbox. The powerbox then checks to see if that operation is allowed for that user, and if so it executes the call, and serializes it back to the app (which then displays it to the user, for example).

            Sandstorm uses CapnP because it's technically sound, battle tested, and efficient.

            You could just make http requests back and forth (that's kinda if the default for web apps) but there is latency overhead and way more surface area with http.

          • ocdtrekkie 11 hours ago
            Well, first and foremost, Sandstorm uses Cap'n Proto because Cap'n Proto was built, first and foremost, to enable Sandstorm. :) Though of course, now it also powers Cloudflare Workers and a bunch of other things.

            My favorite explainer on the object capability model is... the first half of this exceptionally long page: http://habitatchronicles.com/2017/05/what-are-capabilities/ It's worth the read, but read it when you have time to sit down.

            So access to apps or documents by users, as well as those apps and documents being able to reach out to other resources, is all generally wrapped in capabilities.

            I think as a self-hosting platform, the biggest advantages of Sandstorm is that we force developers to provide a no-config-file, no-sysadmin-required experience, so Sandstorm is probably the only self-hosting platform currently intended to be used by people who are not tech people. This naturally flows into some other things: If you don't have a sysadmin running it, it needs to be really secure (capabilities) and very good at self-managing it's own performance.

            So Sandstorm apps do not consume any resources unless you are accessing them. Everything is spun up and closed down on demand. And the platform's security model not only means vulnerabilities in apps mostly aren't exploitable and apps don't need to be patched, but Sandstorm strives to safely allow non-technical users to install and run potentially even malicious apps without risk.

  • nicoco 17 hours ago
    It's worth noting that there are self-hostable XMPP-based implementations for this: https://unifiedpush.org/users/distributors/conversations/

    What is old is new again (Google used XMPP for Android Push Notifications before they switched to something else).

  • pvo50555 14 hours ago
    > Regarding iOS, they have made some progress in adopting webpush, but it does not seem possible to push for mobile applications with it yet.

    This is a key point right at the very end. It has always been much easier to implement push notifications to Android devices, even when only Web Push is implemented. Google has been the main force behind the adoption of PWAs as Native apps on devices, even by immediately showing a "Install this app?" popup when navigating to a PWA-enabled website/web app.

    iOS permits "Installing to home screen" but there is no popup shown natively, nor any indication from the OS that the website you're visiting is a PWA. The user has to go through several steps to add the app to his device.

    Once it is added, it's possible for the device to receive web push notifications and show them as native push notifications (IIRC), even with the app is closed, however the reliability is inconsistent. Sometimes you get them, sometimes you don't. At least that was my experience still about 6 months ago when I tried implementing web push with PWA for iOS.

    I gave up trying to go the pure PWA route and instead wrapped my web app with PWABuilder into a native app, and went through the whole process of submitting to the App Store, getting APN, registering the web app in Firebase, then sending the push notifications to the native app via Firebase Messaging. Works great then, but it's a lot more work than keeping everything simply as a web app and using web push.

    • wruza 13 hours ago
      https://news.ycombinator.com/item?id=38430581

      Web push is basically useless because Google hasn't prioritized fixing the issues causing Android to delay the notifications by 10 minutes or more. They are basically only useful for "re-engagement" spam, nothing time critical.

  • jraph 19 hours ago
    If this encourages more open source applications to use standards and an open source library for push notification,that's quite cool. Extra bonus if it allows users not to rely on Google server.
  • ahmedfromtunis 18 hours ago
    Push notifications are a powerful concept, but I feel like their potential has been stifled by Google's and Apple's near-total control over them on their respective OSes.

    And while I'm not fun of heavy-handed government intervention, I hope the EU compels both companies to adopt a more open approach; provided that security remains uncompromised.

  • mjl- 18 hours ago
    If I understand the notifications flow correctly: a device (typically mobile phone) will get a "registration" with a "push url" from its platform/provider (eg android/google or iphone/apple), which it hands to an application server (eg IMAP server), which then uses that to send notifications to the push url, "authenticating" with vapid. And due to vapid, no special account/preregistration is required for the application server to talk to the provider, hence decentralized (good!). The platform/provider behind the push url makes the notification appear at the device. It is quite a complicated way to get a message from server to device...

    The post says it is about Android. But I'm online searches turns up documents about VAPID at apple/ios. Will it work there too? What about other platforms (eg non-google android-likes)?

    • cyberax 17 hours ago
      Android is very simple. All their notifications are internally supported by a TCP connection to a Firebase endpoint. So when you need to send a notification, you ping the central (and Google-controlled) Firebase endpoint with a device ID. Firebase then pushes a message over the TCP connection that is permanently open between the phone and the Firebase.

      On the phone, the Play Services are exempted from the battery optimizations, so the socket connection stays up. And once a notification is received, Play Services unmarshall it, and route it to the correct application ("activity"), possibly waking it up.

      From the notifier's side, you need to have the credentials for Firebase interaction and the device's token (that Google Play Services make available to the application during the notification setup).

      iOS works in an exactly similar fashion, except that the endpoint is controlled by Apple. To send it, you need to set up an "APN".

  • pull_my_finger 12 hours ago
    So would I be correct to say the distributor is a a 3rd party "satellite" app, that your app pulls from and the notification server pushes to? So end users would need to install this distributor app in addition to your own app?
  • alberth 10 hours ago
    OT: anyone have experience with pushover.net, if so - how is it?
  • stefanha 11 hours ago
    In this direct to FCM model, does the application running on the untrusted user's device need to embed any sensitive credentials?

    What prevents someone else from impersonating my app or other users on my app?

  • kragen 11 hours ago
    This is great news and very timely!
  • EGreg 15 hours ago
    Yeah but how does it work with iOS? I thought Apple has APNS and that's it!
  • dboreham 19 hours ago
    Quick note that this is for Android and gets rid of the centralized gateway talking to the centralized Google service. Leaving only the centralized Google service. There's no decentralized way to get a push notification onto a Google/Android phone. Except perhaps if you hack something up using EAS.
    • jacoblambda 18 hours ago
      ? I think I'm misunderstanding your comment. This replaces the entire push notification infrastructure.

      The "distributor" receives notifications onto the device and sends them to the apps via a standardized API.

      The apps themselves then post the notifications to the user via the android on-device notification API.

      The "gateway" receives the notifications from the services over a standardized API and sends them to the distributor.

      How the gateways actually distribute the notifications is actually completely up to the gateways. The main gateway (ntfy) uses websocket, another (nextpush) piggybacks off of nextcloud, and the new kid on the block (sunup) uses the Mozilla Push Service behind the scenes.

      Nothing in this is restricted or locked in to a centralised platform.

    • jraph 19 hours ago
      Is this true even with some deep integration of UnifiedPush with something like microG?
    • orblivion 19 hours ago
      > There's no decentralized way to get a push notification onto a Google/Android phone

      What is a "push notification" exactly?

      • freeone3000 18 hours ago
        A notification delivered “by” a phone application when that application is not necessarily running.

        They’re “notifications” in common speech but there’s also stuff like “now playing” and live activities and rich content that are also “notifications” but do require the app to run.

        • orblivion 18 hours ago
          That's not really what I mean. The person I was replying to was saying that I can't get push notifications without Google's centralized stuff. ntfy/UnifiedPush appears to do just that, so I'm wondering what is missing here.
          • YarickR2 17 hours ago
            They are doing that by polling private servers, which is sometimes less efficient power-wise, and sometimes notifications are being delivered slower than in case of using Google-provided mechanism (FCM). We've been using ntfy for alerting, using our private server and running ntfy client in polling mode, and I do not see significantly increased battery usage by the ntfy client; Android 15.
            • orblivion 15 hours ago
              So ntfy Android app polls a ntfy service. (Which is more efficient than each app polling separately) What does Google's Android library do if not also poll? Does it run a server on the phone? Does it get special signals from cell towers that most apps don't have access to?
              • freeone3000 13 hours ago
                It also polls, it just does it as part of the OS. Power saving by the OS can often interfere with apps like nfty that want a long-running background process, especially if they haven’t been opened in a while.
                • orblivion 12 hours ago
                  Okay, thank you.

                  From what you're saying, it doesn't sound like the OS-based polling would be more power efficient, as someone in this chain suggested, just more reliable. Though, I think you can just turn off battery optimization for the ntfy app, in which case I'd expect them to be on par.

                  If that's right, it seems like ntfy with battery optimization turned off is effectively a "real" push notification system.

  • colesantiago 18 hours ago
    This looks very over engineered and complicated, we don't need more push notifications.

    Just use or improve upon email or RSS (recommended).

    It's anonymous, fast, privacy friendly (no device registrations or whatever), open and already decentralised so no need for anything else.

    • jacoblambda 18 hours ago
      The purpose of this standard is to replace Google's FCM push notifications so that you can receive push notifications on a mobile device for a bunch of services while only needing to maintain a single low-overhead connection.

      This exists to improve battery life and decrease data usage for android users who don't use google services or prefer to host their own. And generally it exists to provide a standardised, independent, privacy preserving option for users and app developers.

      Neither RSS nor email solve this problem and actually this specific service would be the exact type of service you'd use to notify the user's device that the apps that use RSS or email have new content to fetch.

      • toast0 17 hours ago
        > The purpose of this standard is to replace Google's FCM push notifications so that you can receive push notifications on a mobile device for a bunch of services while only needing to maintain a single low-overhead connection.

        Isn't the purpose of Google's push service that you can receive notifications for a bunch of services with only a single connection?

        I don't quite understand how adding a different service helps, although having an alternative when Google's push is unavailable could be useful, and if there's some way to share that alternate service among many apps, that would reduce overhead of having two notification channels where each app with two channels has their own alternate channel?

        • rcMgD2BwE72F 3 hours ago
          >I don't quite understand how adding a different service help

          I simply don't like having two monopolies (Google and Apple) handling every single push notification for all mobile users everywhere on Earth – except in China, maybe.

          There's so much you privacy risks and they can easily infer many things from the meta data alone.

          I thought we were in the post-Snowden phase already.

        • jacoblambda 17 hours ago
          Yes that is the goal of google's push service.

          The benefit of this service is mainly for people who can't or don't want to use google's push service. Namely for users of de-googled devices and for users of android devices that google blocks from using play services (ex: users of Huawei devices).

          But also it's worth noting that unified push provides an optional google FCM fallback so that apps can use google's push services (provided they are available on the device) when a unifiedpush distributor isn't available or configured. So in this sense it means supporting a single push notification standard instead of multiple and the underlying system can switch between which provider to use.

          And of course for users of completely self hosted services this means you get all the benefits of a proper push aggregator like google's FCM without having to configure firebase with your services. This is particularly appealing if you self host locally and don't already use cloud services.

      • colesantiago 17 hours ago
        I receive content from E-mail all the time and I am also notified about it and it doesn't drain my battery.
        • jacoblambda 17 hours ago
          But what is your email client using for notifications?

          If it's using intermittent/timed fetch, that runs into the latency problem (taking longer to receive notifications). Push notifications exist to solve that problem.

          If it's using IMAP's IDLE, MAPI's push, or JMAP's push, that's just another long-running HTTP or websocket connection.

          And this protocol (along with the proprietary push notif aggregation services it is aiming to replace) exists to deduplicate those many long-running connections into a single connection out to some hardwired server that manages the many services/connections for it.

          • colesantiago 16 hours ago
            > But what is your email client using for notifications?

            Local notifications?

            I don't see the problem here with email, it seems to work well that it is great for my use case.

            When you're offline with push notifications only the latest notification gets sent, not the previous notifications. This seems by design for notifications.

            At least with E-mail I can see a history of emails that was sent to me.

            I just see push notifications as a flawed concept that doesn't make sense for decentralised services.

            • jraph 15 hours ago
              I don't know what you mean by local notifications, but if you get a notification when a mail arrives, it's most probably because your mail client uses Google or Apple's push service if you use a regular mobile device, or if you installed and configured microG.

              And here's your issue: your app that has nothing to do with Google relies on a centralised Google service, and depends on a proprietary library that speaks to some proprietary code running on your device.

              Alternatively, your app isn't doing this, but then it risks being unreliable and stop notifying you if the system kills your app because it's been inactive for some time to save battery, or to reclaim memory, and/or it uses more battery than it could.

              This post is not about a way to have more notifications: it is an answer to these issues.

              • colesantiago 14 hours ago
                > ... but if you get a notification when a mail arrives, it's most probably because your mail client uses Google or Apple's push service if you use a regular mobile device, or if you installed and configured microG.

                Except it doesn't?

                The Mail app on my iPhone doesn't use push notifications, fetch works well and I still get local notifications. No external push service at all.

                > Alternatively, your app isn't doing this, but then it risks being unreliable and stop notifying you if the system kills your app because it's been inactive for some time to save battery, or to reclaim memory, and/or it uses more battery than it could.

                You're saying a bunch of "issues" that never actually happened with me or anyone really, there is actually no issues here.

                E-Mail is fine.

                • crtasm 13 hours ago
                  How frequently is the app checking for new emails?
                • jraph 13 hours ago
                  > Except it doesn't?

                  See how I introduced your case later with "Alternatively"? You saw it since you quoted me.

                  Fine, your email app works well in fetch mode. It took you three roundtrips to mention your actual setup despite having been asked about it quite early and despite it being a critical element in this discussion. Omitting to mention that doing this choice, your notifications are delayed.

                  Fetch mode is totally a valid choice, email doesn't need to be real time for every one. It will actually save battery life from what I can read: you don't need constant polling. Of course, push notifications are for real time stuff. What costs battery life or requires a push notification service on current widespread mobile devices is timely notifications. I can't help you if you fail to see how this can be useful, even if you don't personally need this or are against the idea.

                  Despite a thread that started well (yeah, notifications via emails, a decentralized, open standard, why not? It's terrible for many reasons including deliverability and the fact that you need to be notified of new mails so it's a bit of a chicken and egg issue, but sure, let's discuss), you don't seem actually curious or interested in the topic at hand, preferring to stick to your original point of view, to ignore the answers from all of us and to ultimately show contempt and spite to people discussing with you here. By the way, frankly, loose the scare quotes, they are ridiculous. Of course Apple Mail's fetch mode will be reliable on iOS but that doesn't make the topic trivially solved and yes, the problem exists despite what you are stating.

                  You are calling something over engineered without stopping to wonder why the complexity, assuming it's useless.

                  • colesantiago 2 hours ago
                    I mean, my point with E-mail is to say it already IS a solved problem.

                    Many services and apps use E-Mail so there is no need to setup additional infrastructure unlike push notifications. That is the complexity.

                    Send an email notifying me to [email protected] is more intuitive than setting up a push service.

                    Push notifications suffer from deliverability more frequently (regularly expiring tokens) than E-mail so much so, that when your device is offline and it gets back online, you only get the last notification. This is by design.

                    Having apps use fetch doesn't mean it's delayed, you can set it to a shorter duration but 5 minutes is the default.

                    But, it is over engineered as this is complexity that we do not need and would have a higher maintenance and setup than just using E-mail.

            • jacoblambda 15 hours ago
              I think you are thinking about something different. So "Push Notifications" can refer to two different things.

              The user oriented term refers to on-device notifications that pop up at the top of the screen. Those are technically just normal notifications but people like to call them push notifications for various reasons.

              Push notifications in the technical sense however refer to a system for remote services to notify a client that something happened rather than the client manually polling the server for new updates. It's the difference between the client asking "do you have anything new" and the server telling the client "hey pay attention you have something new". This is what we are talking about when we say push notifications.

              Email clients that automatically get new emails in the background all either do so using fetch polling or push notifications. The traditional method is fetch polling at a regular interval but nowadays most email clients support push notifications. We are discussing a tool that these clients can use behind the scenes to provide these push notifications for the email client in a battery and data efficient manner.

            • ianburrell 15 hours ago
              Push notifications work with any app. It is nicer to group notifications by app, and have app specific behavior. Emails add an extra step to get to the content in the app.

              Gmail does push notifications. Otherwise, have to wait for the client to poll the server.

              • colesantiago 15 hours ago
                > Push notifications work with any app...

                What if I don't want to use an app?

                Email works with every email client that has been ever made.

                It is already universal.

    • crtasm 13 hours ago
      I would like all my push notifications to not rely on Google so this seems good to me.

      Signal sending me an email to tell me I have a new message would be a bad system.

  • methuselah_in 18 hours ago
    There is no too much talk started happening around this. Google will soon I guess going to cut off this like Apple in the be name of battery saving.