Skip to content

Generic Makefiles with GCC and GNU Make

November 8, 2014

Introduction

This article looks at the dependency generation feature of the GCC compiler and its use with GNU Make to create generic makefiles which do not require maintenance as a project grows. Sample makefiles and C++ projects using them are provided on Bitbucket. The projects were created and tested using TDM GCC and MSYS on Windows, but should also work on Linux. You will need to have some familiarity of the C and C++ separate compilation model in order to fully understand the article – I have a few articles on this beginning here.

Make 101

Here, I’m going to give a brief introduction to the GNU Make utility. You are strongly advised to also read the excellent documentation for GNU Make.

Make is a program for building things from separate components. The things built are often executable programs, but they don’t have to be. For example, at a training company I used to work for we used Make to construct our courseware documentation from typesetting instructions in troff files, images, program code and various other sources.

Make is controlled by a makefile, which is a text file (usually named “makefile”) containing a number of rules which specify how to build the product(s) we are interested in. Each rule looks something like this:

to-build-this : from-these
    do-this

where “to-build-this” is the name of the component we are trying to build (such as an executable, a library, or an object code file) and “from-these” is a list of files from which the component will be built (in a C or C++ project these would be source and header files); the component we are trying to build is said to be dependent on these files- if any of them change the component needs to be re-built. Handling these dependencies is the basic task that Make performs for us. The component name and the dependency list are separated by a colon.

The third element of a Make rule is the command that will be used to make the component. This command must be indented from the other elements using a TAB character (using spaces will result in strange error messages from Make) and will usually be something like a compiler or linker command line. You can actually specify any number of commands – all must be indented using tabs. The command is invoked if any dependent file has a modification date later than that of the component being built, or if the component does not exist.

Here’s a simple example of a rule:

myprog.exe : main.cpp utils.cpp utils.h
    g++ –g main.cpp utils.cpp –o myprog.exe

which says that the executable called myprog.exe is dependent on the source files main.cpp and utils.cpp, and the header file utils.h – if any of these files change, then make should rebuild the executable using the supplied g++ command line.With that rule in place in a makefile, we can build myprog.exe simply by typing make at the command line prompt.

GCC Dependency Generation

Unfortunately, although manually editing the makefile is do-able for small projects, as the project increases in size this becomes harder and harder, as source files include headers, which include other headers, and so on. It would be nice to have a utility which looked up the #include directives that cause the dependencies for us, and in fact there is such a tool – the GNU GCC compiler collection.

Given the example files above we can run GCC on the .cpp files like this:

$ g++ -MM *.cpp
main.o: main.cpp utils.h
utils.o: utils.cpp utils.h

The use of the –MM option tells GCC to read the input source files and extract the dependency information from the #include directives. The dependency information is then written to standard output. Notice there is no command specified, only dependency information. When make sees lines like this, it then knows what a product is dependent on, but doesn’t know how to build the product from the dependencies.

A Generic Makefile

Now that we have a means of automatically producing dependency information, we need to write a generic makefile that can use it. The makefile I present here is for the simple project on Bitbucket where all the source (two .cpp files and two .h files – take a look at the project to fully understand what follows) and compiler-generated files are contained in the same directory. The makefile in the project Mercurial tree  is also pretty simple (though slightly more complicated than this tutorial example), and is quite heavily commented – here it is without the comments, and slightly simplified (but still functional):

PRODUCT := rollem.exe

CXX := g++
LINKER := g++
INCDIRS := -I.
CXXFLAGS := -std=c++11 -Wall -Wextra

SRCFILES := $(wildcard *.cpp) 
OBJFILES := $(patsubst %.cpp,%.o,$(SRCFILES))
DEPFILES := $(patsubst %.cpp,%.d,$(SRCFILES))

$(PRODUCT): $(OBJFILES)
    $(LINKER) $^ -o $@

%.o: %.cpp
    $(CXX) $(CXXFLAGS) $(INCDIRS) -c $< -o $@

%.d: %.cpp
    $(CXX) $(INCDIRS) -MM $< > $@

-include $(DEPFILES)

I’ll now go over the makefile’s contents in some detail. The first part sets up a bunch of Make variables, which traditionally are spelled in upper-case. Here:

PRODUCT := rollem.exe

we set up a variable PRODUCT which contains the name of the final executable the makefile is going to produce. The next section sets up some compiler options:

CXX := g++
LINKER := g++
INCDIRS := -I.
CXXFLAGS := -std=c++11 -Wall -Wextra

The CXX and LINKER variables say which C++ compiler and linker to use – in this simple example  we will use the g++ driver program to perform both tasks. The INCDIRS variable will be used to say where the  compiler should look for #included header files (the current directory, in this case), and CXXFLAGS specifies some standard C++ compiler options.

The next section is a bit more interesting – it uses some of Make’s built-in functions to build lists of files we are interested in. Here:

SRCFILES := $(wildcard *.cpp)

we use the wildcard function (a bit like the ls or dir commands) to generate a list of all files which end with the .cpp extension (extensions are called suffixes in Make-speak) and assigns the list to the SRCFILES variable. The $( … ) syntax is used in Make to both call functions and to fetch the contents of a variable. So, we now have a list of our C++ source files. Next:

OBJFILES := $(patsubst %.cpp,%.o,$(SRCFILES))

we use the patsubst (it stands for “pattern substitution” – a bit like the sed command) function to generate a list of object files from the list of source files we just created by replacing all .cpp suffixes with .o.

The last list of files we need is one containing the names of files that will contain the dependency information we are going to create using the –MM option of GCC, as described in the previous section. One such file will be generated for each .cpp file.

The creation of these file lists, and of the other variables, will always be performed whenever Make is run on this makefile. The next section of the file contains the rules that will be used to build the various components – these will only run if directly invoked, or when dependent files have changed. The first rule:

$(PRODUCT): $(OBJFILES)
    $(LINKER) $^ -o $@

This rule (because it is the first rule in the makefile, it is the default rule, invoked when you simply type make at the command line prompt) says that the product we are trying to produce is dependent on the files in the list OBJFILES. It also says that to build the product Make needs to invoke the LINKER (i.e. g++) command. The special symbols $^ and $@ are special Make variables – the first evaluates to the list of dependencies, and the second to the name of the product. So, what this rule expands to when variable contents are used is:

rollem.exe: main.o dice.o
    g++ main.o dice.o -o rollem.exe

and will be invoked if either of main.o or dice.o is newer than rollem.exe, or if rollem.exe does not exist.

The next two rules use wild-carding to specify how to build the object and dependency files. This rule:

%.o: %.cpp
    $(CXX) $(CXXFLAGS) $(INCDIRS) -c $< -o $@

says to Make – “A file with a .o suffix is dependent on a file with the same name, but with a .cpp suffix, and here’s how you build one from the other”. This rule will be applied following the first rule if either main.o or dice.o needs to be re-built. With variable expansion, the command to do the re-build will be (for main.o):

g++ -std=c++11 -Wall -Wextra -I. -c main.cpp -o main.o

The last rule says how to use the –MM option of GCC to generate the .d dependency files. Here’s the command it produces (for main.d):

g++ -I. -MM main.cpp > main.d

Here we use shell indirection to write the output the –MM option to a file. This generates a .d dependency files which looks like this:

main.o: main.cpp dice.h report.h

which says that main.o is dependent on main.cpp, dice.h and report.h. A similar .d file is generated for the dice.cpp source file.

The last bit of the makefile is probably hardest to understand. It looks like this:

-include $(DEPFILES)

which should pull in the .d files, much like #include works in a C or C++ program. However, this isn’t exactly what happens. If the .d files already exist, then they are simply included in the makefile, and Make reads the dependency information from them. However, if they don’t exist (as they won’t the first time Make is run using this makefile) Make will look to see if there is a rule to create them. There is (the last rule) so Make runs the command associated with that rule to create the .d files and then re-reads the whole makefile. This time the .d files do exist (because we just created them), so they are read by the simple include mechanism.

Running Make

Phew! That’s quite a lot to take in (and it took me some time to get the makefile right when I first started working on this article). Let’s see what happens when we run the makefile for the very first time:

$ make
g++ -I. -MM main.cpp > main.d
g++ -I. -MM dice.cpp > dice.d
g++ -std=c++11 -Wall -Wextra -I. -c dice.cpp -o dice.o
g++ -std=c++11 -Wall -Wextra -I. -c main.cpp -o main.o
g++ dice.o main.o -o rollem.exe

You can see that the dependency information files are generated, then the .cpp files are compiled to .o files, and finally the .o files are linked to create the executable. Notice that none of the names of the .d, or .cpp files are explicitly mentioned in the makefile.

Now, let’s run Make again:

$ make
make: `rollem.exe' is up to date.

Make sees that nothing has changed, so there is nothing to do. Now let’s touch one of the .cpp files (touch changes the modification date on the file) and run Make again:

$ touch dice.cpp
$ make
g++ -I. -MM dice.cpp > dice.d
g++ -std=c++11 -Wall -Wextra -I. -c dice.cpp -o dice.o
g++ dice.o main.o -o rollem.exe

Make has to re-build both the dice.d dependency file, the dice.o file and the executable. However, it does not rebuild main.d or main.o.

Now, let’s touch one of the headers:

$ touch dice.h
$ make
g++ -std=c++11 -Wall -Wextra -I. -c dice.cpp -o dice.o
g++ -std=c++11 -Wall -Wextra -I. -c main.cpp -o main.o
g++ dice.o main.o -o rollem.exe

Both of the .cpp files need to be re-compiled as both contain #includes referencing the dice.h header file. However, the dependency files are not re-built, and this is a problem, which I address below.

Lastly, let’s create a new .cpp file:

$ touch foobar.cpp
$ make
g++ -I. -MM foobar.cpp > foobar.d
g++ -std=c++11 -Wall -Wextra -I. -c foobar.cpp -o foobar.o
g++ dice.o foobar.o main.o  -o rollem.exe

The new .cpp is recognised, dependency information is generated, and the file is correctly compiled and linked, all without the makefile needing to be changed in any way.

Forcing Dependency Generation

The rules the makefile contains so far will force regeneration of the .d files if the corresponding .cpp file is changed. For simple projects, this will work 90% of the time, but we would really like the .d file for a given .cpp file to be re-built if any of the dependencies of the specific .cpp file change – for example, if we add a #include directive to a header file, we have created a new dependency on the newly-included header, and we would like the dependencies to be updated to reflect that. In other words, instead of:

main.o: main.cpp dice.h report.h

we need to produce;

main.d main.o: main.cpp dice.h report.h

and the same for the dice.* files.

Neither Make nor GCC provide a simple, built-in way of doing this, so we need to resort to one of the other command line tools, the sed stream editor. I don’t propose to give a sed tutorial here, but what we need to do is to change the rule for generating .d files to:

%.d: %.cpp
    $(CXX) $(INCDIRS) -MM $< | sed 's/^/$@ /' > $@

which uses sed to edit the dependency to begin with whatever %.d actually is when the makefile is run, which will be the name of the .d file we are generating. With this rule in place, whenever we add a dependency to existing dependencies, the .d file files will be automatically updated with the new dependency information when we run Make.

Conclusion

This article has presented a simple generic makefile which can be used to build multi-file C++ projects, with dependency information being provided by the GCC compiler rather than being hard-coded (possibly incorrectly) in the makefile by the developer. In a future article I’ll look at how this makefile can be expanded to work with multi-directory projects and with debug and release builds of the product(s) being developed.

Advertisements
Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: