Immutable in this context refers to an OS that can’t be changed while running. Steam deck does something like that. Basically the all of the OS system files are read only, so that the user or some malware can’t Bork the system. The only parts that are writable are the users profile directory and the logs.
You can still receive updates and install apps. It’s just that that’s handled a bit differently than with a standard OS.
E.g. it could be that the OS provider only issues complete updates, and then you either have to reboot. This is the case with steam os on the steam deck. The System portion of the OS is mounted read only during use.
Another prominent example is Android. Sure system apps can be upgraded individually – by storing the new version in a restricted part of the ‘user’ partition – but otherwise the system files are strictly read only until a new ‘image’ is ‘flashed’ to it by the update system or a power user with debugging tools. In the past, a common use of root capabilities was to remount the system partition as read/write and then change files on it directly. It’s more complex now.
That’s also why system apps can be rolled back to the stock version, and can sometimes be disabled, but can’t be directly uninstalled like user apps. Only the updated version on the user partition (if there is one) can be removed.
You can update the whole base image. Vanilla OS and SteamOS have an A/B partition that holds the currently-in-use image and can also hold a to-be-used image.
Updating works by adding the to-be-used image, setting a configuration option that tells the system to boot that one, and on the next boot it’ll check if the new one is bootable, then either boot it and mark it as working, or boot into the old one and display an error about how out wasn’t able to boot into the new one.
There’s smart things going on like maybe hard linking files that didn’t change between the two images and therefore saving space and copying time.
The result is that you never have a broken system, but you can still frequently update the base image.
I’ve read several topics trying to explain it and this single comment does a way better job, thank you XD
If you don’t mind me asking a follow-up, why are non-immutable OS’s in Linux more popular? Or in other words, is there a definite downside to an immutable OS that people should be wary of? I was planning to install Fedora 40 soon, but now I think I may opt for the Atomic one (with the KDE env) instead.
Regular Linux distros have 30+ years of history. It’s what most of us are used to. Immutable/atomic/transactional OSes are relatively recent hence the relatively low adoption rate.
Also, atomic OSes are, by nature, much harder to tinker with. After all, the goal is to provide the exact same image for all users. As a power user, it’s a bit frustrating. As a new user, having a virtually unborkable system is excellent.
If you plan on installing an atomic variant of Fedora, may I suggest uBlue Aurora instead of Fedora Kinoite? It is based on Silverblue/Kinoite but includes by default, among other QOL improvements, the restricted-licence codecs that must be manually installed in official Fedora products.
Well, currently I use Tumbleweed with just couple of tweaks, but I can’t live without things like Yakuake, fish, yt-dlp and bunch of other console commands that are not present in most dostros’ defaults. How does atomic distribution handle this? I believe flatpak only has gui applications…
// I just diacovered Yakuake is there, but I can’t imagine how does this specific program integrate with system?
You can layer basically any RPM onto the base system with rpm-ostree, but it’s slow and inefficient, or you can install anything from any distro by spinning a container with Distrobox and exporting the command to your main system.
The universeal blue family of operating systems also comes with Homebrew, the Linux port of the popular Mac package manager. The idea being that flatpak is for GUI apps and homebrew for the cli
Oh yeah thanks I forgot about brew. TBH the only uBlue machine I’m currently playing with is destined to be my dad’s new computer, so he’s not expected to get anywhere near the command line :D
I am all open for suggestions! I will add a bit of context; I am proficient with Linux command line, good enough to troubleshoot if problems pop up. But I currently do not feel the desire to tinker a lot with the system itself, I just want to do daily driving, play games, and do some basic coding for fun. What value do those restricted licence codecs bring to the system?
If the immutability in OS is well designed, then there shouldn’t be really an downsides or loss in comfort. That is, unless you’re a linux expert and like to tinker under the hood.
The general idea is, the core of the OS if read-only, and everything else that needs to be modified is mounted writeable.
Ideally, protecting the core of the OS from writes, should for example prevent malware from installing a modified kernel or boot loader. Or maybe preventing the user from accidentally borking something so that their system becomes unbootable. How much of an advantage that is practice is dependent on use case.
In the case of Steam OS on the steam deck, it’s perfect, since boot issues on the steam deck could potentially be tricky to fix as opposed to a standard PC.
Another advantage of immutable could theoretically be wear and tear of certain storage devices. e.g. Think of a raspberry PI and SDcards. If you could have most of the important stuff of the OS as read only on the SD card, and everything else on a usb disk or even an NFS mount, then the SD card should last much longer since no writes are happening on it.
As far as true security benefit is concerned… I can’t really say. It depends on how updates and eventual writes are actually handled to the immutable part of the OS. Obviously at some point, changes do happen. Like during a system update.
In the case of Steam OS, The system portion is wiped and replaced the new version. Chimera OS, did something similar (I don’t know if they still use the same method). They had a read-only BTRFS partition, where they would then provide a new snapshot during an update, which would be downloaded and applied at the next reboot.
This approach would hinder automated crypto malware for example (at least for system files).
Thank you, then it looks like I’ll be giving the atomic one a try! I can always overwrite and install normal Fedora KDE if the atomic version is giving me issues after all :)
I know of a good specific example of a downside. I play SS14 on Steam Deck and musical instruments in it require midi support. This means I need to properly install FluidSynth, but I can’t. I can get the flat pack, but the game won’t recognize that it’s there because it’s looking in the wrong place.
This is why I’m going to install bazzite on my deck.
Yes, pretty much always has too. /system is normally mounted read-only and all use data is in /data, so when you want to factory reset, you just format /data and reboot and voilà, brand new. Later versions also have a /vendor and a bunch of other partitions but the idea remains the same, as a user you only get to change /data (and /sdcard back before they had any meaningful internal storage, now it’s an overlay mount to /data/media/0).
I believe the original idea was mainly focused around making software developer’s lives easier, by (as you say) making versions of linux os’s with known, static, unchanging, immutable, standard system configurations, which could not be altered by the user.
Again, I think the main idea here was basically focused around making either new distros, or sort of locked down versions of distros such that they could be deployed in I guess an enterprise or some kind of group environment, so as to greatly streamline the process of software development.
You say ‘can receive updates’, and I am not sure if you mean higher level app updates, or lower level system updates.
When people attempted to evangelize this immutable concept to me a few years back, they would say the whole point is that lower level updates on which many other things depend do not happen, that you would have to do basically a completely new install of a new immutable version when one comes out.
Kind of like ‘more stable than stable.’
But… judging by other comments here, it would appear that this concept of immutable has uh… changed, lol.
Also, for what its worth, the Steam Deck OS is basically just a standardized version of Arch, upon which sits the whole nifty Steam Deck UI and Controller mappings, with the system set to read only by default.
You can change this and then do whatever the hell you want, if you so please.
I think its just one line in the terminal after you have set up your user as a system admin.
Immutable distros were originally very focused on servers, and more recently distros for workstations has stated gaining more interest as the concept has matured.
With the advent of cloud computing “immutable infrastructure” started becoming more and more popular. This concept started out as someone sitting down and grabbing a normal Linux distribution, and installing all the necessary bits for the server purpose they needed. Then baking that into an image. Now you could launch new copies of that machine whenever you felt like it, and they would behave exactly the same. If any of them started doing something wonky, you just destroyed it and launched a new copy. This was very useful for software developers and operations people who could now more easily reason about how things behaved. And be sure that the difference in behaviour wasn’t because someone forgot to enable a setting, install a tool, or skipped a step in the setup.
On the software development side, you also simultaneously saw more and more developers make use of functional programming methods, and al’ng with those immutable data structures. Fundamentally, instead of adding an item to a list, you make a new list with all the old and the new items in it. You never change the data after it’s creation. Each “change” is a new copy, with the difference already built in.
Then containers started becoming popular. Which allowed software developers to build a container image on their local computer, and then ship that image to a server, where the image behaved exactly as it did on their local machine. This also meant that the actual OS became less and less important, as everything needed by the container was already bundled in the container. The containers also worked as “immutable”, since everything you would install or change within the containers would be immediately lost when the container was destroyed, and recreating it would be exactly as when the image was built.
The advent of containerised workloads, gave rise to a lot of different Linux distributions. Since the containers pretty much only needed the Linux kernel from the OS, it was pretty easy to make a container-centric operating system. And in turn lock down everything else, even completely omitting having a package manager. Stuff like CoreOS, Flatcar, Rancher OS, and many others were immutable linuxes that only catered to containers. I don’t know the exact mechanism for all of these, but at least the original CoreOS and Flatcar make the actual system read only, and on top of that had two man partitions, one of the partitions would be the current system, and the other would be where updates were downloaded. Once an update was downloaded and ready, you just rebooted the machine, and it would be running off the updated partition. Which also meant easy rollback if you got a broken update. You could just boot off the other unupdated partition.
Containers were however rather ill suited for desktop applications, as there were no good way to provide a GUI. You could serve up a Web page, but native GUI apps were tricky.
That’s where Flatpak, Snaps and all that came, which essentially brings the container mentality to normal desktop apps. This brought immutability to individual apps, as they brought their own dependencies. And therefore didn’t have to rely on the correct versions of dependencies being available on the machine.
The logical next step was of course to add immutability to workstation distributions. This is where the popularity of Fedora Silverblue, NixOS, and many others really started taking shape.
I believe Fedora Silverblue uses ostree to make the system “immutable”. Of course you can still make changes to your system, but the system is built to be completely aware of the state before and the state after, this is what’s called “atomic”. There’s no such thing as a partially installed package. There is only the state before installing something, and the state when the thing is fully installed. You can roll back to any of the previous states, to recover from a broken update or misconfiguration. This also makes trying out new things with no risk. Trying out a new desktop environment, and it broke your system? Just roll back. Accidentally uninstalled a critical package? Just roll back. What to try out a new display manager? Just apply the config and roll back if you don’t like it.
SteamOS also does the thing with multiple partitions, and even allows you to turn off the immutability. Other distributions aren’t as lenient. There’s no way to turn off the immutability in NixOS or Fedora Silverblue.
Depending on the use case there’s usually a temporary system that’s there only to take the update from the user partition and apply it to the system partition. So even if you bork the update it’ll still boot into that environment and install the system again. Valve does provide bootable images to put on a USB stick if you do break it pretty bad. It’s just a PC, it doesn’t do much to stop you from wiping the disk. The route Android took is A/B devices, when you’re using A you update B and then reboot into B, then the next update you’ll be updating the A partition and reboot into it. Plus if the next one fails to boot for some reason you can revert to the old version as if nothing happened, and retry the update from scratch. Except Samsung, because I don’t know I guess they want to turn the updating into a whole experience of anticipation or whatever crap reason they have for it.
I’m no expert on this but I’m pretty sure the /etc directory is writeable too for config files, which sadly still allows a user or malware to still bork the system if they get superuser privilege
I find it hard to imagine a system that is not borkable by a superuser. Maybe it’s helpful to think of immutable setups as harder to bork by accident during routine maintenance (e.g. through faulty updates) and more resilient to bad code (through containerization).
good point, that’s fair. The reason I think it bears mentioning is that editing configs under /etc/ is totally something we might expect a user to do. So you could follow a tutorial online that is wrong or outdated and with enough bad luck, tada, you bricked your “immutable” system. Or, less dramatic and more likely, something doesn’t work as intended anymore and you don’t know how to restore to the original config from when you installed.
And I agree that a lot of documentation and how-to-guides don’t account for immutable setups (yet?), which can get novice users especially in a lot of trouble.
Personally, I prefer a declarative system (NixOS) that solves this problem rather cleanly and gives me most benefits of so-called immutable distros as well.
What if a new OS comes with, say, a GPU driver that is not compatible anymore? Is there not even an override for this possible? I have a PC with the GTX 970 which in computer terms is basically just after the meteorite crash ending the jurassic era. And i have been considering to use Bazzite OS for this machine.
Immutable in this context refers to an OS that can’t be changed while running. Steam deck does something like that. Basically the all of the OS system files are read only, so that the user or some malware can’t Bork the system. The only parts that are writable are the users profile directory and the logs.
You can still receive updates and install apps. It’s just that that’s handled a bit differently than with a standard OS.
E.g. it could be that the OS provider only issues complete updates, and then you either have to reboot. This is the case with steam os on the steam deck. The System portion of the OS is mounted read only during use.
Another prominent example is Android. Sure system apps can be upgraded individually – by storing the new version in a restricted part of the ‘user’ partition – but otherwise the system files are strictly read only until a new ‘image’ is ‘flashed’ to it by the update system or a power user with debugging tools. In the past, a common use of root capabilities was to remount the system partition as read/write and then change files on it directly. It’s more complex now.
That’s also why system apps can be rolled back to the stock version, and can sometimes be disabled, but can’t be directly uninstalled like user apps. Only the updated version on the user partition (if there is one) can be removed.
Aren’t apps better compareable to something like flatpack and this is the reason why they are updateable during runtime?
Sure they are, but system apps are still installed in the immutable space initially, which is the important thing, that updates to it can’t go there.
I don’t know how desktop immutable systems deal with that.
You can update the whole base image. Vanilla OS and SteamOS have an A/B partition that holds the currently-in-use image and can also hold a to-be-used image.
Updating works by adding the to-be-used image, setting a configuration option that tells the system to boot that one, and on the next boot it’ll check if the new one is bootable, then either boot it and mark it as working, or boot into the old one and display an error about how out wasn’t able to boot into the new one.
There’s smart things going on like maybe hard linking files that didn’t change between the two images and therefore saving space and copying time.
The result is that you never have a broken system, but you can still frequently update the base image.
I’ve read several topics trying to explain it and this single comment does a way better job, thank you XD
If you don’t mind me asking a follow-up, why are non-immutable OS’s in Linux more popular? Or in other words, is there a definite downside to an immutable OS that people should be wary of? I was planning to install Fedora 40 soon, but now I think I may opt for the Atomic one (with the KDE env) instead.
Regular Linux distros have 30+ years of history. It’s what most of us are used to. Immutable/atomic/transactional OSes are relatively recent hence the relatively low adoption rate.
Also, atomic OSes are, by nature, much harder to tinker with. After all, the goal is to provide the exact same image for all users. As a power user, it’s a bit frustrating. As a new user, having a virtually unborkable system is excellent.
If you plan on installing an atomic variant of Fedora, may I suggest uBlue Aurora instead of Fedora Kinoite? It is based on Silverblue/Kinoite but includes by default, among other QOL improvements, the restricted-licence codecs that must be manually installed in official Fedora products.
Well, currently I use Tumbleweed with just couple of tweaks, but I can’t live without things like Yakuake, fish, yt-dlp and bunch of other console commands that are not present in most dostros’ defaults. How does atomic distribution handle this? I believe flatpak only has gui applications…
// I just diacovered Yakuake is there, but I can’t imagine how does this specific program integrate with system?
You can layer basically any RPM onto the base system with
rpm-ostree, but it’s slow and inefficient, or you can install anything from any distro by spinning a container with Distrobox and exporting the command to your main system.The universeal blue family of operating systems also comes with Homebrew, the Linux port of the popular Mac package manager. The idea being that flatpak is for GUI apps and homebrew for the cli
Oh yeah thanks I forgot about brew. TBH the only uBlue machine I’m currently playing with is destined to be my dad’s new computer, so he’s not expected to get anywhere near the command line :D
Pretty sure I installed protontricks from a flatpak too, and that one is a console app. But it depends on some flatpak being available for the app.
Like the other reply said though, you can use other means to install apps in ways that don’t require altering system files.
I am all open for suggestions! I will add a bit of context; I am proficient with Linux command line, good enough to troubleshoot if problems pop up. But I currently do not feel the desire to tinker a lot with the system itself, I just want to do daily driving, play games, and do some basic coding for fun. What value do those restricted licence codecs bring to the system?
Hardware acceleration mostly.
If the immutability in OS is well designed, then there shouldn’t be really an downsides or loss in comfort. That is, unless you’re a linux expert and like to tinker under the hood.
The general idea is, the core of the OS if read-only, and everything else that needs to be modified is mounted writeable. Ideally, protecting the core of the OS from writes, should for example prevent malware from installing a modified kernel or boot loader. Or maybe preventing the user from accidentally borking something so that their system becomes unbootable. How much of an advantage that is practice is dependent on use case. In the case of Steam OS on the steam deck, it’s perfect, since boot issues on the steam deck could potentially be tricky to fix as opposed to a standard PC.
Another advantage of immutable could theoretically be wear and tear of certain storage devices. e.g. Think of a raspberry PI and SDcards. If you could have most of the important stuff of the OS as read only on the SD card, and everything else on a usb disk or even an NFS mount, then the SD card should last much longer since no writes are happening on it.
As far as true security benefit is concerned… I can’t really say. It depends on how updates and eventual writes are actually handled to the immutable part of the OS. Obviously at some point, changes do happen. Like during a system update. In the case of Steam OS, The system portion is wiped and replaced the new version. Chimera OS, did something similar (I don’t know if they still use the same method). They had a read-only BTRFS partition, where they would then provide a new snapshot during an update, which would be downloaded and applied at the next reboot. This approach would hinder automated crypto malware for example (at least for system files).
Thank you, then it looks like I’ll be giving the atomic one a try! I can always overwrite and install normal Fedora KDE if the atomic version is giving me issues after all :)
I know of a good specific example of a downside. I play SS14 on Steam Deck and musical instruments in it require midi support. This means I need to properly install FluidSynth, but I can’t. I can get the flat pack, but the game won’t recognize that it’s there because it’s looking in the wrong place.
This is why I’m going to install bazzite on my deck.
Is Android immutable?
Yes, pretty much always has too.
/systemis normally mounted read-only and all use data is in/data, so when you want to factory reset, you just format/dataand reboot and voilà, brand new. Later versions also have a/vendorand a bunch of other partitions but the idea remains the same, as a user you only get to change/data(and/sdcardback before they had any meaningful internal storage, now it’s an overlay mount to/data/media/0).I believe the original idea was mainly focused around making software developer’s lives easier, by (as you say) making versions of linux os’s with known, static, unchanging, immutable, standard system configurations, which could not be altered by the user.
Again, I think the main idea here was basically focused around making either new distros, or sort of locked down versions of distros such that they could be deployed in I guess an enterprise or some kind of group environment, so as to greatly streamline the process of software development.
You say ‘can receive updates’, and I am not sure if you mean higher level app updates, or lower level system updates.
When people attempted to evangelize this immutable concept to me a few years back, they would say the whole point is that lower level updates on which many other things depend do not happen, that you would have to do basically a completely new install of a new immutable version when one comes out.
Kind of like ‘more stable than stable.’
But… judging by other comments here, it would appear that this concept of immutable has uh… changed, lol.
Also, for what its worth, the Steam Deck OS is basically just a standardized version of Arch, upon which sits the whole nifty Steam Deck UI and Controller mappings, with the system set to read only by default.
You can change this and then do whatever the hell you want, if you so please.
I think its just one line in the terminal after you have set up your user as a system admin.
Immutable distros were originally very focused on servers, and more recently distros for workstations has stated gaining more interest as the concept has matured.
With the advent of cloud computing “immutable infrastructure” started becoming more and more popular. This concept started out as someone sitting down and grabbing a normal Linux distribution, and installing all the necessary bits for the server purpose they needed. Then baking that into an image. Now you could launch new copies of that machine whenever you felt like it, and they would behave exactly the same. If any of them started doing something wonky, you just destroyed it and launched a new copy. This was very useful for software developers and operations people who could now more easily reason about how things behaved. And be sure that the difference in behaviour wasn’t because someone forgot to enable a setting, install a tool, or skipped a step in the setup.
On the software development side, you also simultaneously saw more and more developers make use of functional programming methods, and al’ng with those immutable data structures. Fundamentally, instead of adding an item to a list, you make a new list with all the old and the new items in it. You never change the data after it’s creation. Each “change” is a new copy, with the difference already built in.
Then containers started becoming popular. Which allowed software developers to build a container image on their local computer, and then ship that image to a server, where the image behaved exactly as it did on their local machine. This also meant that the actual OS became less and less important, as everything needed by the container was already bundled in the container. The containers also worked as “immutable”, since everything you would install or change within the containers would be immediately lost when the container was destroyed, and recreating it would be exactly as when the image was built.
The advent of containerised workloads, gave rise to a lot of different Linux distributions. Since the containers pretty much only needed the Linux kernel from the OS, it was pretty easy to make a container-centric operating system. And in turn lock down everything else, even completely omitting having a package manager. Stuff like CoreOS, Flatcar, Rancher OS, and many others were immutable linuxes that only catered to containers. I don’t know the exact mechanism for all of these, but at least the original CoreOS and Flatcar make the actual system read only, and on top of that had two man partitions, one of the partitions would be the current system, and the other would be where updates were downloaded. Once an update was downloaded and ready, you just rebooted the machine, and it would be running off the updated partition. Which also meant easy rollback if you got a broken update. You could just boot off the other unupdated partition.
Containers were however rather ill suited for desktop applications, as there were no good way to provide a GUI. You could serve up a Web page, but native GUI apps were tricky.
That’s where Flatpak, Snaps and all that came, which essentially brings the container mentality to normal desktop apps. This brought immutability to individual apps, as they brought their own dependencies. And therefore didn’t have to rely on the correct versions of dependencies being available on the machine.
The logical next step was of course to add immutability to workstation distributions. This is where the popularity of Fedora Silverblue, NixOS, and many others really started taking shape.
I believe Fedora Silverblue uses ostree to make the system “immutable”. Of course you can still make changes to your system, but the system is built to be completely aware of the state before and the state after, this is what’s called “atomic”. There’s no such thing as a partially installed package. There is only the state before installing something, and the state when the thing is fully installed. You can roll back to any of the previous states, to recover from a broken update or misconfiguration. This also makes trying out new things with no risk. Trying out a new desktop environment, and it broke your system? Just roll back. Accidentally uninstalled a critical package? Just roll back. What to try out a new display manager? Just apply the config and roll back if you don’t like it.
SteamOS also does the thing with multiple partitions, and even allows you to turn off the immutability. Other distributions aren’t as lenient. There’s no way to turn off the immutability in NixOS or Fedora Silverblue.
Best post in this whole thread, imo.
Thank you 😊
Sounds pretty secure except for at the update stage, but you said that’s handled differently so maybe that’s more secure too.
Depending on the use case there’s usually a temporary system that’s there only to take the update from the user partition and apply it to the system partition. So even if you bork the update it’ll still boot into that environment and install the system again. Valve does provide bootable images to put on a USB stick if you do break it pretty bad. It’s just a PC, it doesn’t do much to stop you from wiping the disk. The route Android took is A/B devices, when you’re using A you update B and then reboot into B, then the next update you’ll be updating the A partition and reboot into it. Plus if the next one fails to boot for some reason you can revert to the old version as if nothing happened, and retry the update from scratch. Except Samsung, because I don’t know I guess they want to turn the updating into a whole experience of anticipation or whatever crap reason they have for it.
IIRC, Samsung recently announced they’re moving to A/B partitioning as well.
I’m no expert on this but I’m pretty sure the /etc directory is writeable too for config files, which sadly still allows a user or malware to still bork the system if they get superuser privilege
I find it hard to imagine a system that is not borkable by a superuser. Maybe it’s helpful to think of immutable setups as harder to bork by accident during routine maintenance (e.g. through faulty updates) and more resilient to bad code (through containerization).
good point, that’s fair. The reason I think it bears mentioning is that editing configs under /etc/ is totally something we might expect a user to do. So you could follow a tutorial online that is wrong or outdated and with enough bad luck, tada, you bricked your “immutable” system. Or, less dramatic and more likely, something doesn’t work as intended anymore and you don’t know how to restore to the original config from when you installed.
You’re right that “immutable” is a bit of a misnomer in that regard, and it’s been argued that “atomic” is a more fitting term.
And I agree that a lot of documentation and how-to-guides don’t account for immutable setups (yet?), which can get novice users especially in a lot of trouble.
Personally, I prefer a declarative system (NixOS) that solves this problem rather cleanly and gives me most benefits of so-called immutable distros as well.
What if a new OS comes with, say, a GPU driver that is not compatible anymore? Is there not even an override for this possible? I have a PC with the GTX 970 which in computer terms is basically just after the meteorite crash ending the jurassic era. And i have been considering to use Bazzite OS for this machine.
Sounds proprietary.