The power of the doc side 3: Increasing make efficiency

So, I am finally pretty satisfied with those little makefiles. As announced previously, they now feature automated file detection and automated dependency tracking, which bring both extra comfort and a sizeable boost in performance. I also separated the arch-specific part of the makefiles from the arch-independent one since although not requiring immediate action, that mess lying around irritated me quite a bit.

Automated file detection

This part has been addressed using little scripts based on the UNIX find command, such as the following bit from the kernel makefile:

KNL_ASM_SRC:= $(shell find kernel/ -path '*/opt/*' -prune -o -name '*.s' -print)

For those not familiar with either find or the source tree’s organization, this script autodetects ASM source files, excluding optional components (debug code, tests…) which are taken care of elsewhere, and stores the result in a variable that can be used later in the makefile.

Also, notice the use of immediate evaluation in variable assignments (“:=” rather than “=”), recently generalized in my makefiles, which avoids the find command to be executed all over again every time this variable is referenced, which would be a blatant waste of resources.

Dependency tracking

As it happens, the GCC preprocessor does feature a way to generate, as a side-effect of file compilation, make rules that specify the headers which a given file depends on. It also happens that make will silently merge multiple dependency lists for a single file without spitting errors or ignoring parts of the dependency graph. There is a lot of literature on the web as for how these two capabilities can be combined to achieve automated dependency tracking in makefile, but the most elegant solution which I have found so far is this one.

  • Each time a C source file is compiled, the -MD compilation flag is used, telling the GCC preprocessor to generate a .d file alongside every object file containing the appropriate dependency rules for this file. The -MP compilation flag is also used, tweaking the contents of the generated file to avoids build errors when headers are removed from the source tree.
  • An include statement, prefixed with a dash to avoid errors when these files have not been generated yet, is appended at the end of the makefiles, so that these rules are taken into account in further project builds.

Considering that source files are systematically rebuilt when modified, and that additional header includes can only be added by modifying a source file, it becomes apparent that the only way this can go wrong would be a situation in which someone would delete the automatically generated dependency files without deleting the associated object files. Considering how unlikely such an outcome is (and given that in such a case, a simple make clean and rebuild would be enough to solve the problem), I do not think that addressing this situation is worth the cost of an explicit make rule for the generation of dependency files, which is much more difficult to do in an elegant way.

And that’s all for today. Next step is for me to write the updated makefile documentation. Thank you for reading !

One thought on “The power of the doc side 3: Increasing make efficiency

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s