This post can be considered a follow-up to my previous post on file management. It goes into more details towards describing what I have in mind as far as filesystem organization and everyday file and software management is concerned. If you are interested in what happens the first time software is run or where temporary files should be stored, please read on !
First, let’s put a quick reminder of some things which were said last time and are of particular interest right now. I suggest a standard directory structure which follows the following pattern :
- /Applications : This folder contains sandboxed and independent applications, stored in OSX-like “bundles” (folders acting like files for non-knowledgeable entities) that group together code, data, and configuration, and sorted in well thought-out categories.
- /Storage : This is where storage media (HDD partitions, ramdisks, external storage…) are mounted as a default. These can be accessed through both device name and unique UUID-like identifiers through some symlink magic.
- /System : This is where one finds system files, which are bundled with the OS and necessary for the everyday operation of both said OS and third-party applications.
- /Users : This is where one finds users’ private folders, plus a “Global” folder for easy inter-user communication. Unlike on other OSs, the pollution of this directory is minimal : Desktop and Trash folders, maybe one hidden folder for per-user system service configuration, that should be about it. The rest is user-created files.
From this point, let’s talk in more details about applications. As said above, they should be sandboxed from the outside world, in sense that they should live in an isolated process environment that doesn’t allow them to interact with other applications, raw hardware peripherals, or user files without asking for permission first through an OS-standard mechanism. This idea stems from the observation that while legacy OSs were mainly designed to prevent users from doing harm to each other and to the OS install, the biggest evil on modern PCs is malicious and buggy software, not users. Hiding this fact behind the outdated and fuzzy notion of “admin privileges” will not help, and what’s urgently needed is a limitation of the privileges which applications can get without a clear request to the user.
In the specific case of file management, sandboxing resolves around trapping a program in a set of private files and folders and forbidding it to take a look at the filesystem beyond that without explicit user permission (through privilege elevation, passing of file location as a command line parameter, or standard file open/save dialogs). The easiest way to achieve this result in practice is to physically put all of a program’s file in a single application folder. This is part of what makes the concept of “application bundles” so attractive.
The content of such a bundle could be…
- A “Code” folder, that contains read-only executable binaries and libraries
- A “Data” folder, that contains readable and writable data files
- a “Config” folder, that contains one “Global” subfolder for system-wide configuration and one subfolder per user who has saved configuration data for the application.
- A “System” folder, that contains metadata about the application (icon, main executable location, file associations, requested privileges…)
Now, some long-time UNIX users may be alarmed by this idea, and emit the following objections about this concept :
- It makes it harder to separate programs from their configuration for efficient backup purposes
- It requires the configuration folder hierarchy of all applications to be updated when users are added, deleted, or changed
- It makes the configuration data of all users accessible by the application, which is a potential security issue
Let’s answer these concerns right now :
- Assuming that data and configuration are perfectly transferable from one version of a program’s code to another is a lucky guess. It is true that it can be the case if the developer follows good programming practices, but it cannot be guaranteed by the OS. Besides, software regularly disappears from the internet, and it can never be assumed that a binary that was installed on the OS yesterday will still be on the web tomorrow. For these reasons, program code, data, and configuration should preferably be backed up together.
- In the context of home and mobile computing, which is what I mainly have in mind, external HDDs have grown big enough in the past few years that such a full backup can indeed be performed.
- Storing most application-specific data in a single folder makes software removal more straightforward : it can be as simple as having the user delete the application’s folder, while a system service monitors the contents of the /Applications folder and makes sure to clean up associated OS management structures (privileges, file associations…) in such an event.
- Applications only need to create configuration data for a new user on the first time that said user opens them, not to maintain a full database of users. Also, in the context of home and mobile computing, users are deleted and renamed extremely rarely, so that a full database regeneration may not be such a big deal.
- The last concern (users accessing each other’s private configuration data through bugs in the application) is an illustration of the security paradigm shift represented by my approach to security. Here, we don’t care so much about the possibility that very knowledgeable users may peek into each other’s data. If a piece of private user data is so secret, it should be encrypted anyway in order to prevent offline access. What we care most about is the possibility for a rogue or buggy application to smash the configuration data of other applications. By moving configuration data in per-application isolated storage regions, we ensure that this cannot happen, save for full-scale file management security breakage situations.
Now that this is said, let’s examine how these application bundles work from the point of view of everyday user experience : “So, I have just downloaded or extracted from a backup an application bundle, what should I do with it now ?”.
Installation should be performed in the most straightforward way, by double-clicking the bundle. In best-case scenario, this would be the only action that the user has to perform, before the application is automatically set up and copied to the /Applications folder. No drag and drop, no endless clicking on a “next” button, no root password to type. In more details, here is what would happen :
- OS services notice that no management structure is associated to this application, and call the software installer to take care of the installation.
- If the application bundle is a compressed file, it is first silently unpacked to a temporary location, either on a ramdisk if there is enough space in RAM* or on the system drive otherwise.
- If the application requests privileges above a user-adjustable “danger” threshold, an installation step is to request the user to confirm the cession of those extra privileges. The action of each OS-defined privileges is clearly documented, and adding extra privileges to them is in itself a privileged action.
- If the application is associated to some file types, the OS check if some of these file types are already associated to another application. If so, the next installation step is to select for which file types the new application should become the default handler. If not, the application becomes the default handler for everything.
- If the application wants to be run on every OS boot, the next installation step is to let the user choose whether he wants it to happen, default choice being “no”.
- If the installer finds pre-existing configuration data in the application bundle, it asks how the data should be migrated to the new computer, defaults being to migrate OS-wide settings and to migrate user settings to users of the same name.
- Once all these installation steps are performed, the application installer copies the application bundle to the /Applications folder, and adjusts system and application configuration to match the user’s choices. At the end of these operations, a message is transmitted to the user in some way, stating that the application has been copied to /Applications and is ready for use.
* I actually think that in many case, a ramdisk is the best place to store a temporary file on a modern PC : it offers both high performance and guaranteed instantaneous deletion on OS reboot even when the shutdown did not go well.
It is, in my opinion, not the job of the OS to manage per-application post installation tasks, such as EULA validation. These can lead to an inhomogeneous application installation experience at best, and to the OS being accused of wrongdoing that it did not cause at worst. As such, applications should manage those themselves on first start. Also, I am against the existence of software interdependencies, such as shared libraries that are not part of the core OS, so I do not plan to do any step on the OS side to help software that puts itself in such situations. Though I might change my mind on this one later, given some relevant use cases.
Anyway, our software is now installed, and we may use it the way we want. The next step in its lifecycle is updates, a necessary evil of modern computing. Now, I am sure that many of you have experienced the joy of software-specific updaters such as the ones of VLC, Adobe Flash and Acrobat, or Oracle Java, so let’s get this out of the door : I do not want to let this happen to my OS. Application-specific updaters are unneeded effort duplication, security issues, and most of all a major source of user exasperation. Instead, the OS updater should also update individual applications, as can be observed on the repository-based software management systems that are flourishing on many OSs today.
A difference, though, is that I am neither Red Hat nor Apple, and don’t have the money, time, and megalomania it takes to manage a huge software repository containing almost every software available on my OS. Instead, I want to follow a decentralized scheme in which each application downloads updates from a unique set of repositories, so that developers may distribute their updates themselves (while still keeping the possibility to switch back to a repository system later if more favourable circumstances arise).
Being decentralized does not mean being outdated though, and I would like the updater to support all the bells and whistles that one can expect from a modern program of this kind : simultaneous download and installation of independent packages, delta updates with a full package fallback, ability to update system services on the fly without an OS restart and to install the rest without a program restart, regulated network bandwidth usage, etc…
As for software deletion, as said before, it should be as simple for the user as deleting the soft from the /Applications folder. OS services should watch in some way over file management operations in this folder to monitor file deletions (to update associated management structures), but also other possible user actions such as an attempt to copy a file in the folder or to create stuff in it.
And that’s it ! Hope you found this interesting, and in any case thank you for reading !