Building software designs that last, rather than decay

Software interfaces rarely age well. Very often, products that looked fine enough in version 1 needed many major UI and API redesigns as they gained features and lost bugs, until they ended up unable to adapt further and either disappeared or underwent complete rewrites.

In the realm of OSdeving, this issue is particularly salient, as operating system software is extremely long-lived and has a lot of users and applications relying on its various interfaces, which means that changing any one of them is very difficult.

In this post, I will be discussing some ways OS interfaces can be made more future-proof.

Setting expectations

First of all, this is about designing interfaces, not writing code. Many excellent resources have already been written on ways to make code more maintainable, from McConnell’s Code Complete to Hunt and Thomas’ The Pragmatic Programmer, and the subject is vast enough to warrant such hundred-pages-long reviews. Here, I am interested in the ways code communicates with human beings (end users, sysadmins, developers alike) because…

  • Fixing broken interface designs is several orders of magnitude more work than fixing broken implementations
  • Especially so in operating systems
  • Coding tips are everywhere, while design discussions are much harder to come by
  • The design problem looks deceptively simple : you think you’ve cracked it, until you haven’t

I also won’t be trying to make a distinction between end user interfaces and APIs. From the point of view of operating systems, both belong to the user interface, the only difference is that they cater to different kinds of users.

This post is about trying to shine some light on the mystery of ancient software interface designs like malloc(), UNIX pipes, and “OK” buttons, that have remained useful across decades without any major modification. To find some common patterns that seem to underpin such sustainable software interface design work.

This post is about questioning how we can build software interfaces that will survive the next computer form factor evolutions without throwing decades of API and user interface research out of the window.

Brainstorming sustainable software

At this point, you may be asking yourself “Hey, isn’t that a little too ambitious?”. Sure it is, and I certainly won’t be covering the full subject in one post. I only want to raise your attention on it and make both of us think about it.

I’m publishing my brainstorming results on this matter here, but feel free to add your own, propose extra points, debate existing ones, etc. This blog has comments for a reason.

Anyway, here’s what I’ve come up so far:

  • Sustainable software solves simple problems – Complex problems require complex solutions, whose designs have more failure points. If your problem can’t be described in a couple of sentences, break it down.
  • It is unmistakably useful – And it will remain so for a predictably long time. Sustainable software design doesn’t follow temporary fads, it is built to address permanent and important issues.
  • Its mode of operation is clear – If you can’t explain to your target user base how to use the software interface in a couple of sentences, it is still too complex.
  • It is usable – The interface should not only be easy to understand, but also easy to use. Any interface which requires careful thought and frequent documentation lookup to be used will end up wrapped into a better one.
  • It is as abstract as reasonably possible – This is the one us programmers are most often aware of. Your interface should put as little constraints on implementation details as possible, except (and that part is critically important) where following this rule would contradict the previous ones. Don’t unleash a Java file manipulation API on humanity.

An example of these rules in action

So far, what I’ve been writing must remind you a lot of the early UNIX philosophy as professed by e.g. Eric S. Raymond in The Art of UNIX Programming. And indeed, although UNIX developers subsequently proved quite bad at following their own tenets in many respects (X11, systemd and gigantic kernels anyone?), I think there is the lot of wisdom to be found in the early design of that platform, before merry hackers without much taste or knowledge of software design took over.

But an example which I find more interesting, in this area, is that of the Microsoft Windows APIs. These started out with absolutely terrible designs, and were then gradually fixed by multiple iterations of incompatible APIs. Of course, I wouldn’t recommend that path to any serious OS designer, as throwing away software designs and starting over like that is prohibitely expensive and poses serious backwards compatibility challenges. But I have to salute the ability that Microsoft engineers have shown here to admit their errors and make amends.

For an example of such design fixes, I will invoke the original Windows API’s CreateFile() method and one of its replacements in the more modern .Net framework, File.Create().

If we consider the original CreateFile()…

  • It solved a very complicated problem. It would create and/or open files and many other kinds of I/O interfaces, for many different kinds of purposes, in various ways, set up many properties of the resulting system resource, and also included access to a wide range of Windows features that are only weakly related to file access, such as user impersonation by server processes.
  • Solving this problem was mostly useless. There was no real, compelling need to do all of these tasks at once. Developers would have been equally satisfied, and most likely more so, by a set of functions that would perform all of these tasks separately, as is common in the API of UNIX systems.
  • Its mode of operation is hopelessly unclear. Just look at the length of the MSDN documentation page for it, then contemplate how much of that documentation is still a summary, and how following some links is still necessary to understand the various values of the function’s parameters, and what they do. Acquiring full understanding of the interface provided by this function would be a matter of hours, or weeks, depending on the developer’s expertise level.
  • It is barely usable. Unless you have superhuman brain abilities, you will need to constantly refer to the MSDN documentation any time you use this function, and think deeply about what you’re doing with each and every one of its parameters. And even then, with so much complexity to deal with, you’ll probably get it wrong anyway, especially by the time you will begin to try testing your program and handling errors properly.
  • It is a very weak abstraction. Its specification poses many constraints on the way the underlying filesystem implementation must work. By being part of the function’s specification, feature flags like FILE_FLAG_POSIX_SEMANTICS and FILE_FLAG_OPEN_REPARSE_POINT intertwin the concerns of what looks like a file management API with that of many other Windows subsystems.

Contrast this with the single-string-argument File.Create() function of the more modern .Net API.

  • It solves a simple problem: to create or overwrite a file at a given position in the filesystem, and open it for writing.
  • It is useful. This is a very common software development task.
  • It has a clear design. Anyone who understands filesystem paths is pretty much ready to use it.
  • It is usable. The name and parameter list is easy to remember, as they follow many API conventions. Using it is fast and painless. It’s hard to make mistakes with it.
  • It is abstract. The only dependency that this function sets on implementations is that they must by able to efficiently designate files by (unicode) strings. Pretty much every filesystem out there is capable of that.

What File.Create()’s implementation likely does is simply to wrap CreateFile into a more usable interface, a fate which I’ve previously predicted for every OS interface that fails at following basic usability rules. After all, Microsoft still need to support all the legacy software which uses that API anyway. However, the wrapper turns out to be, interestingly enough, a lot more versatile and future-proof than its parent function, in spite doing functionally less.

  • CreateFile was so complicated that Microsoft’s own attempts at porting it to mobile platforms only went halfway. Contrast with File.Create(), which has compatible implementations on every popular OS out there, some of which weren’t written by Microsoft.
  • CreateFile’s specification was constantly revised as Windows evolved, and many incompatible variants like the horribly named CreateFile2 were created to handle new file creation edge cases. Whereas File.Create()’s specification hasn’t changed since it was introduced in the .Net framework.
  • While File.Create()’s basic specification may seem overly simplistic, it is overloaded with variants featuring several optional parameters, which allow one to access the most useful advanced functionality of its CreateFile ancestor — without the cruft.


You may argue that this is only anecdotal evidence. I’m only quoting one example. I’m using a lot of gut feeling and common sense arguments to prove my point. I may underestimate or overestimate the impact of some of the desirable software design properties that I’m invoking. And guess what, that’s true.

There’s a lot of fascinating research to be done in this area, in my opinion, but I’m not going to do all of it. Here, I only wanted to raise an issue which I feel is important in the realm of OS design, share my thoughts on it so far, and provide one detailed example of how I think they are reasonable (rather than a collection of barely explored ones).

If you want to do more research on this subject, please do it. And share it with me, too, if you feel so inclined. I welcome more examples, discussions of the elements of sustainable designs proposed above, and propositions of adding or removing some. I also, and especially, welcome extra bibliographical or web sources to cultivate myself more on the work of others on this subject, if you know of any.

For now thought, if I have convinced you that designing good OS interfaces that stand the test of time is crucial, and that I have found at least a couple of relevant means to this end, I’ll consider myself satisfied.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s