I am trying to instruct GNU Make 3.81 to not stop if a command fails (so I prefix the command with -
) but I also want to check the exit status on the next command and print a more informative message. However my Makefile below fails:
$ cat Makefile
all:
-/bin/false
([ $$? -eq 0 ] && echo "success!") || echo "failure!"
$
$ make
/bin/false
make: [all] Error 1 (ignored)
([ $? -eq 0 ] && echo "success!") || echo "failure!"
success!
Why does the Makefile above echo «success!» instead of «failure!» ?
update:
Following and expanding on the accepted answer, below is how it should be written:
failure:
@-/bin/false && ([ $$? -eq 0 ] && echo "success!") || echo "failure!"
success:
@-/bin/true && ([ $$? -eq 0 ] && echo "success!") || echo "failure!"
asked Sep 4, 2014 at 15:13
3
Each update command in a Makefile
rule is executed in a separate shell. So $?
does not contain the exit status of the previous failed command, it contains whatever the default value is for $?
in a new shell. That’s why your [ $? -eq 0 ]
test always succeeds.
pevik
1,33315 silver badges27 bronze badges
answered Sep 4, 2014 at 18:16
Kyle JonesKyle Jones
14.5k3 gold badges40 silver badges51 bronze badges
You don’t need the test of $?
since &&
works if $?
is zero and ||
is proceed in case of a non-zero return value.
And you don’t need the minus since at the return value to make is taken from the last proceed program call of the line. So this works fine
failure:
@/bin/false && echo "success!" || echo "failure!"
success:
@/bin/true && echo "success!" || echo "failure!"
The opposite happens: If you wanna do your own message and want to break the make process anyway with a non-zero value, you need to write something like this:
failure:
@/bin/false && echo "success!" || { echo "failure!"; exit 1; }
Jeff Schaller♦
65.2k34 gold badges106 silver badges240 bronze badges
answered Feb 10, 2015 at 14:24
AndreasAndreas
1911 silver badge2 bronze badges
From the GNU make documentation:
When errors are to be ignored, because of either a ‘-’ or the ‘-i’ flag, make treats an error return just like success, except that it prints out a message that tells you the status code the shell exited with, and says that the error has been ignored.
To utilize make
‘s exit status in a case like this, execute make
from a script:
#!/bin/bash
make
([ $? -eq 0 ] && echo "success!") || echo "failure!"
And have your Makefile contain:
all:
/bin/false
answered Sep 4, 2014 at 18:04
Timothy MartinTimothy Martin
8,2971 gold badge34 silver badges40 bronze badges
Here is how I ended up doing it:
.PHONY: test-cleanup
test-cleanup:
$(eval CLEANUP = echo "Some cleanup procedure")
@/bin/true && touch _testok || $(CLEANUP) ;
if [ -f "_testok" ]; then
rm -f _testok ;
echo "What a success!" ;
else
echo "Failure :-(" ;
exit 1 ;
fi
Which prints:
What a success!
If you change /bin/true
to /bin/false
then you get the following output:
Some cleanup procedure
Failure :-(
make: *** [Makefile:4: test-cleanup] Error 1
answered Sep 8, 2022 at 15:13
Chapter 12. Debugging Makefiles
Debugging makefiles is somewhat of a black art. Unfortunately, there is no such thing as a makefile debugger to examine how a particular rule is being evaluated or a variable expanded. Instead, most debugging is performed with simple print statements and by inspection of the makefile. GNU make
provides some help with various built-in functions and command-line options.
One of the best ways to debug a makefile is to add debugging hooks and use defensive programming techniques that you can fall back on when things go awry. I’ll present a few basic debugging techniques and defensive coding practices I’ve found most helpful.
Debugging Features of make
The warning
function is very useful for debugging wayward makefiles. Because the warning
function expands to the empty string, it can be placed anywhere in a makefile: at the top-level, in target or prerequisite lists, and in command scripts. This allows you to print the value of variables wherever it is most convenient to inspect them. For example:
$(warning A top-level warning) FOO := $(warning Right-hand side of a simple variable)bar BAZ = $(warning Right-hand side of a recursive variable)boo $(warning A target)target: $(warning In a prerequisite list)makefile $(BAZ) $(warning In a command script) ls $(BAZ):
yields the output:
$ make makefile:1: A top-level warning makefile:2: Right-hand side of a simple variable makefile:5: A target makefile:5: In a prerequisite list makefile:5: Right-hand side of a recursive variable makefile:8: Right-hand side of a recursive variable makefile:6: In a command script ls makefile
Notice that the evaluation of the warning
function follows the normal make
algorithm for immediate and deferred evaluation. Although the assignment to BAZ
contains a warning
, the message does not print until BAZ
is evaluated in the prerequisites list.
The ability to inject a warning
call anywhere makes it an essential debugging tool.
Command-Line Options
There are three command-line options I find most useful for debugging: --just-print
(-n
), --print-data-base
(-p
), and --warn-undefined-variables
.
—just-print
The first test I perform on a new makefile target is to invoke make
with the --just-print
(-n
) option. This causes make
to read the makefile and print every command it would normally execute to update the target but without executing them. As a convenience, GNU make
will also echo commands marked with the silent modifier (@
).
The option is supposed to suppress all command execution. While this may be true in one sense, practically speaking, you must take care. While make
will not execute command scripts, it will evaluate shell
function calls that occur within an immediate context. For instance:
REQUIRED_DIRS = ... _MKDIRS := $(shell for d in $(REQUIRED_DIRS); do [[ -d $$d ]] || mkdir -p $$d; done) $(objects) : $(sources)
As we’ve seen before, the purpose of the _MKDIRS
simple variable is to trigger the creation of essential directories. When this is executed with --just-print
, the shell command will be executed as usual when the makefile is read. Then make
will echo (without executing) each compilation command required to update the $(objects)
file list.
—print-data-base
The --print-data-base
(-p
) option is another one you’ll use often. It executes the makefile, displaying the GNU copyright followed by the commands as they are run by make
, then it will dump its internal database. The data is collected into groups of values: variables, directories, implicit rules, pattern-specific variables, files (explicit rules), and the vpath search path:
# GNU Make 3.80
# Copyright (C) 2002 Free Software Foundation, Inc.
# This is free software; see the source for copying conditions.
# There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
# PARTICULAR PURPOSE.
normal command execution occurs here
# Make data base, printed on Thu Apr 29 20:58:13 2004
# Variables
...
# Directories
...
# Implicit Rules
...
# Pattern-specific variable values
...
# Files
...
# VPATH Search Paths
Let’s examine these sections in more detail.
The variables section lists each variable along with a descriptive comment:
# automatic <D = $(patsubst %/,%,$(dir $<)) # environment EMACS_DIR = C:/usr/emacs-21.3.50.7 # default CWEAVE = cweave # makefile (from `../mp3_player/makefile', line 35) CPPFLAGS = $(addprefix -I ,$(include_dirs)) # makefile (from `../ch07-separate-binaries/makefile', line 44) RM := rm -f # makefile (from `../mp3_player/makefile', line 14) define make-library libraries += $1 sources += $2 $1: $(call source-to-object,$2) $(AR) $(ARFLAGS) $$@ $$^ endef
Automatic variables are not printed, but convenience variables derived from them like $(<D)
are. The comment indicates the type of the variable as returned by the origin
function (see the section Less Important Miscellaneous Functions in Chapter 4). If the variable is defined in a file, the filename and line number of the definition is given. Simple and recursive variables are distinguished by the assignment operator. The value of a simple variable will be displayed as the evaluated form of the righthand side.
The next section, labeled Directories, is more useful to make
developers than to make
users. It lists the directories being examined by make
, including SCCS and RCS subdirectories that might exist, but usually do not. For each directory, make
displays implementation details, such as the device number, inode, and statistics on file pattern matches.
The Implicit Rules section follows. This contains all the built-in and user-defined pattern rules in make
’s database. Again, for those rules defined in a file, a comment indicates the file and line number:
%.c %.h: %.y # commands to execute (from `../mp3_player/makefile', line 73): $(YACC.y) --defines $< $(MV) y.tab.c $*.c $(MV) y.tab.h $*.h %: %.c # commands to execute (built-in): $(LINK.c) $^ $(LOADLIBES) $(LDLIBS) -o $@ %.o: %.c # commands to execute (built-in): $(COMPILE.c) $(OUTPUT_OPTION) $<
Examining this section is a great way to become familiar with the variety and structure of make
’s built-in rules. Of course, not all built-in rules are implemented as pattern rules. If you don’t find the rule you’re looking for, check in the Files section where the old-style suffix rules are listed.
The next section catalogs the pattern-specific variables defined in the makefile. Recall that pattern-specific variables are variable definitions whose scope is precisely the execution time of their associated pattern rule. For example, the pattern variable YYLEXFLAG
, defined as:
%.c %.h: YYLEXFLAG := -d %.c %.h: %.y $(YACC.y) --defines $< $(MV) y.tab.c $*.c $(MV) y.tab.h $*.h
would be displayed as:
# Pattern-specific variable values %.c : # makefile (from `Makefile', line 1) # YYLEXFLAG := -d # variable set hash-table stats: # Load=1/16=6%, Rehash=0, Collisions=0/1=0% %.h : # makefile (from `Makefile', line 1) # YYLEXFLAG := -d # variable set hash-table stats: # Load=1/16=6%, Rehash=0, Collisions=0/1=0% # 2 pattern-specific variable values
The Files section follows and lists all the explicit and suffix rules that relate to specific files:
# Not a target: .p.o: # Implicit rule search has not been done. # Modification time never checked. # File has not been updated. # commands to execute (built-in): $(COMPILE.p) $(OUTPUT_OPTION) $< lib/ui/libui.a: lib/ui/ui.o # Implicit rule search has not been done. # Last modified 2004-04-01 22:04:09.515625 # File has been updated. # Successfully updated. # commands to execute (from `../mp3_player/lib/ui/module.mk', line 3): ar rv $@ $^ lib/codec/codec.o: ../mp3_player/lib/codec/codec.c ../mp3_player/lib/codec/codec.c .. /mp3_player/include/codec/codec.h # Implicit rule search has been done. # Implicit/static pattern stem: `lib/codec/codec' # Last modified 2004-04-01 22:04:08.40625 # File has been updated. # Successfully updated. # commands to execute (built-in): $(COMPILE.c) $(OUTPUT_OPTION) $<
Intermediate files and suffix rules are labeled “Not a target”; the remainder are targets. Each file includes comments indicating how make
has processed the rule. Files that are found through the normal vpath
search have their resolved path displayed.
The last section is labeled VPATH Search Paths and lists the value of VPATH
and all the vpath
patterns.
For makefiles that make extensive use of user-defined functions and eval
to create complex variables and rules, examining this output is often the only way to verify that macro expansion has generated the expected values.
—warn-undefined-variables
This option causes make
to display a warning whenever an undefined variable is expanded. Since undefined variables expand to the empty string, it is common for typographical errors in variable names to go undetected for long periods. The problem with this option, and why I use it only rarely, is that many built-in rules include undefined variables as hooks for user-defined values. So running make
with this option will inevitably produce many warnings that are not errors and have no useful relationship to the user’s makefile. For example:
$ make --warn-undefined-variables -n
makefile:35: warning: undefined variable MAKECMDGOALS
makefile:45: warning: undefined variable CFLAGS
makefile:45: warning: undefined variable TARGET_ARCH
...
makefile:35: warning: undefined variable MAKECMDGOALS
make: warning: undefined variable CFLAGS
make: warning: undefined variable TARGET_ARCH
make: warning: undefined variable CFLAGS
make: warning: undefined variable TARGET_ARCH
...
make: warning: undefined variable LDFLAGS
make: warning: undefined variable TARGET_ARCH
make: warning: undefined variable LOADLIBES
make: warning: undefined variable LDLIBS
Nevertheless, this command can be extremely valuable on occasion in catching these kinds of errors.
The —debug Option
When you need to know how make
analyzes your dependency graph, use the --debug
option. This provides the most detailed information available other than by running a debugger. There are five debugging options and one modifier: basic
, verbose
, implicit
, jobs
, all
, and makefile
, respectively.
If the debugging option is specified as --debug
, basic debugging is used. If the debugging option is given as -d
, all
is used. To select other combinations of options, use a comma separated list --debug=
option1
,option2
where the option can be one of the following words (actually, make
looks only at the first letter):
basic
-
Basic debugging is the least detailed. When enabled,
make
prints each target that is found to be out-of-date and the status of the update action. Sample output looks like:File all does not exist. File app/player/play_mp3 does not exist. File app/player/play_mp3.o does not exist. Must remake target app/player/play_mp3.o. gcc ... ../mp3_player/app/player/play_mp3.c Successfully remade target file app/player/play_mp3.o.
verbose
-
This option sets the
basic
option and includes additional information about which files where parsed, prerequisites that did not need to be rebuilt, etc.:File all does not exist. Considering target file app/player/play_mp3. File app/player/play_mp3 does not exist. Considering target file app/player/play_mp3.o. File app/player/play_mp3.o does not exist. Pruning file ../mp3_player/app/player/play_mp3.c. Pruning file ../mp3_player/app/player/play_mp3.c. Pruning file ../mp3_player/include/player/play_mp3.h. Finished prerequisites of target file app/player/play_mp3.o. Must remake target app/player/play_mp3.o. gcc ... ../mp3_player/app/player/play_mp3.c Successfully remade target file app/player/play_mp3.o. Pruning file app/player/play_mp3.o.
implicit
-
This option sets the
basic
option and includes additional information about implicit rule searches for each target:File all does not exist. File app/player/play_mp3 does not exist. Looking for an implicit rule for app/player/play_mp3. Trying pattern rule with stem play_mp3. Trying implicit prerequisite app/player/play_mp3.o. Found an implicit rule for app/player/play_mp3. File app/player/play_mp3.o does not exist. Looking for an implicit rule for app/player/play_mp3.o. Trying pattern rule with stem play_mp3. Trying implicit prerequisite app/player/play_mp3.c. Found prerequisite app/player/play_mp3.c as VPATH ../mp3_player/app/player/ play_mp3.c Found an implicit rule for app/player/play_mp3.o. Must remake target app/player/play_mp3.o. gcc ... ../mp3_player/app/player/play_mp3.c Successfully remade target file app/player/play_mp3.o.
jobs
-
This options prints the details of subprocesses invoked by
make
. It does not enable thebasic
option.Got a SIGCHLD; 1 unreaped children. gcc ... ../mp3_player/app/player/play_mp3.c Putting child 0x10033800 (app/player/play_mp3.o) PID 576 on the chain. Live child 0x10033800 (app/player/play_mp3.o) PID 576 Got a SIGCHLD; 1 unreaped children. Reaping winning child 0x10033800 PID 576 Removing child 0x10033800 PID 576 from chain.
all
-
This enables all the previous options and is the default when using the
-d
option. makefile
-
Normally, debugging information is not enabled until after the makefiles have been updated. This includes updating any included files, such as lists of dependencies. When you use this modifier,
make
will print the selected information while rebuilding makefiles and include files. This option enables thebasic
option and is also enabled by theall
option.
Writing Code for Debugging
As you can see, there aren’t too many tools for debugging makefiles, just a few ways to dump make
’s internal data structures and a couple of print statements. When it comes right down to it, it is up to you to write your makefiles in ways that either minimize the chance of errors or provide your own scaffolding to help debug them.
The suggestions in this section are laid out somewhat arbitrarily as coding practices, defensive coding, and debugging techniques. While specific items, such as checking the exit status of commands, could be placed in either the good coding practice section or the defensive coding section, the three categories reflect the proper bias. Focus on coding your makefiles well without cutting too many corners. Include plenty of defensive coding to protect the makefile against unexpected events and environmental conditions. Finally, when bugs do arise, use every trick you can find to squash them.
The “Keep It Simple” Principle (http://www.catb.org/~esr/jargon/html/K/KISS-Principle.html) is at the heart of all good design. As you’ve seen in previous chapters, makefiles can quickly become complex, even for mundane tasks, such as dependency generation. Fight the tendency to include more and more features in your build system. You’ll fail, but not as badly as you would if you simply include every feature that occurs to you.
Good Coding Practices
In my experience, most programmers do not see writing makefiles as programming and, therefore, do not take the same care as they do when writing in C++ or Java. But the make
language is a complete nonprocedural language. If the reliability and maintainability of your build system is important, write it with care and use the best coding practices you can.
One of the most important aspects of programming robust makefiles is to check the return status of commands. Of course, make
will check simple commands automatically, but makefiles often include compound commands that can fail quietly:
do: cd i-dont-exist; echo *.c
When run, this makefile does not terminate with an error status, although an error most definitely occurs:
$ make
cd i-dont-exist;
echo *.c
/bin/sh: line 1: cd: i-dont-exist: No such file or directory
*.c
Furthermore, the globbing expression fails to find any .c files, so it quietly returns the globbing expression. Oops. A better way to code this command script is to use the shell’s features for checking and preventing errors:
SHELL = /bin/bash do: cd i-dont-exist && shopt -s nullglob && echo *.c
Now the cd
error is properly transmitted to make
, the echo
command never executes, and make
terminates with an error status. In addition, setting the nullglob
option of bash
causes the globbing pattern to return the empty string if no files are found. (Of course, your particular application may prefer the globbing pattern.)
$ make
cd i-dont-exist &&
echo *.c
/bin/sh: line 1: cd: i-dont-exist: No such file or directory
make: *** [do] Error 1
Another good coding practice is formatting your code for maximum readability. Most makefiles I see are poorly formatted and, consequently, difficult to read. Which do you find easier to read?
_MKDIRS := $(shell for d in $(REQUIRED_DIRS); do [[ -d $$d ]] || mkdir -p $$d; done)
or:
_MKDIRS := $(shell for d in $(REQUIRED_DIRS); do [[ -d $$d ]] || mkdir -p $$d; done)
If you’re like most people, you’ll find the first more difficult to parse, the semicolons harder to find, and the number of statements more difficult to count. These are not trivial concerns. A significant percentage of the syntax errors you will encounter in command scripts will be due to missing semicolons, backslashes, or other separators, such as pipe and logical operators.
Also, note that not all missing separators will generate an error. For instance, neither of the following errors will produce a shell syntax error:
TAGS: cd src ctags --recurse disk_free: echo "Checking free disk space..." df . | awk '{ print $$4 }'
Formatting commands for readability will make these kinds of errors easier to catch. When formatting user-defined functions, indent the code. Occasionally, the extra spaces in the resulting macro expansion cause problems. If so, wrap the formatting in a strip
function call. When formatting long lists of values, separate each value on its own line. Add a comment before each target, give a brief explanation, and document the parameter list.
The next good coding practice is the liberal use of variables to hold common values. As in any program, the unrestrained use of literal values creates code duplication and leads to maintenance problems and bugs. Another great advantage of variables is that you can get make
to display them for debugging purposes during execution. I show a nice command line interface in the section “Debugging Techniques,” later in this chapter.
Defensive Coding
Defensive code is code that can execute only if one of your assumptions or expectations is wrong — an if
test that is never true, an assert
function that never fails, or tracing code. Of course, the value of this code that never executes is that occasionally (usually when you least expect it), it does run and produce a warning or error, or you choose to enable tracing code to allow you to view the inner workings of make
.
You’ve already seen most of this code in other contexts, but for convenience it is repeated here.
Validation checking is a great example of defensive code. This code sample verifies that the currently executing version of make
is 3.80:
NEED_VERSION := 3.80 $(if $(filter $(NEED_VERSION),$(MAKE_VERSION)),, $(error You must be running make version $(NEED_VERSION).))
For Java applications, it is useful to include a check for files in the CLASSPATH
.
Validation code can also simply ensure that something is true. The directory creation code from the previous section is of this nature.
Another great defensive coding technique is to use the assert
functions defined in the section Flow Control in Chapter 4. Here are several versions:
# $(call assert,condition,message) define assert $(if $1,,$(error Assertion failed: $2)) endef # $(call assert-file-exists,wildcard-pattern) define assert-file-exists $(call assert,$(wildcard $1),$1 does not exist) endef # $(call assert-not-null,make-variable) define assert-not-null $(call assert,$($1),The variable "$1" is null) endef
I find sprinkling assert
calls around the makefile to be a cheap and effective way of detecting missing and misspelled parameters as well as violations of other assumptions.
In Chapter 4, we wrote a pair of functions to trace the expansion of user-defined functions:
# $(debug-enter) debug-enter = $(if $(debug_trace), $(warning Entering $0($(echo-args)))) # $(debug-leave) debug-leave = $(if $(debug_trace),$(warning Leaving $0)) comma := , echo-args = $(subst ' ','$(comma) ', $(foreach a,1 2 3 4 5 6 7 8 9,'$($a)'))
You can add these macro calls to your own functions and leave them disabled until they are required for debugging. To enable them, set debug_trace
to any nonempty value:
$ make debug_trace=1
As noted in Chapter 4, these trace macros have a number of problems of their own but can still be useful in tracking down bugs.
The final defensive programming technique is simply to make disabling the @
command modifier easy by using it through a make
variable, rather than literally:
QUIET := @
...
target:
$(QUIET) some command
Using this technique, you can see the execution of the silent command by redefining QUIET
on the command line:
$ make QUIET=
Debugging Techniques
This section discusses general debugging techniques and issues. Ultimately, debugging is a grab-bag of whatever works for your situation. These techniques have worked for me, and I’ve come to rely on them to debug even the simplest makefile problems. Maybe they’ll help you, too.
One of the very annoying bugs in 3.80 is that when make
reports problems in makefiles and includes a line number, I usually find that the line number is wrong. I haven’t investigated whether the problem is due to include files, multiline variable assignments, or user-defined macros, but there it is. Usually the line number make
reports is larger than the actual line number. In complex makefiles, I’ve had the line number be off by as much as 20 lines.
Often the easiest way to see the value of a make
variable is to print it during the execution of a target. Although adding print statements using warning
is simple, the extra effort of adding a generic debug
target for printing variables can save lots of time in the long run. Here is a sample debug
target:
debug: $(for v,$(V), $(warning $v = $($v)))
To use it, just set the list of variables to print on the command line, and include the debug
target:
$ make V="USERNAME SHELL" debug makefile:2: USERNAME = Owner makefile:2: SHELL = /bin/sh.exe make: debug is up to date.
If you want to get really tricky, you can use the MAKECMDGOALS
variable to avoid the assignment to the variable V
:
debug: $(for v,$(V) $(MAKECMDGOALS), $(if $(filter debug,$v),,$(warning $v = $($v))))
Now you can print variables by simply listing them on the command line. I don’t recommend this technique, though, because you’ll also get confusing make
warnings indicating it doesn’t know how to update the variables (since they are listed as targets):
$ make debug USERNAME SHELL
makefile:2: USERNAME = Owner
makefile:2: SHELL = /bin/sh.exe
make: debug is up to date.
make: *** No rule to make target USERNAME. Stop.
In Chapter 10, I briefly mentioned using a debugging shell to help understand some of the activities make
performs behind the scenes. While make
echos commands in command scripts before they are executed, it does not echo the commands executed in shell
functions. Often these commands are subtle and complex, particularly since they may be executed immediately or in a deferred fashion, if they occur in a recursive variable assignment. One way to see these commands execute is to request that the subshell enable debug printing:
DATE := $(shell date +%F) OUTPUT_DIR = out-$(DATE) make-directories := $(shell [ -d $(OUTPUT_DIR) ] || mkdir -p $(OUTPUT_DIR)) all: ;
When run with sh
’s debugging option, we see:
$ make SHELL="sh -x" + date +%F + '[' -d out-2004-05-11 ']' + mkdir -p out-2004-05-11
This even provides additional debugging information beyond make warning
statements, since with this option the shell also displays the value of variables and expressions.
Many of the examples in this book are written as deeply nested expressions, such as this one that checks the PATH
variable on a Windows/Cygwin system:
$(if $(findstring /bin/, $(firstword $(wildcard $(addsuffix /sort$(if $(COMSPEC),.exe), $(subst :, ,$(PATH)))))),, $(error Your PATH is wrong, c:/usr/cygwin/bin should precede c:/WINDOWS/system32))
There is no good way to debug these expressions. One reasonable approach is to unroll them, and print each subexpression:
$(warning $(subst :, ,$(PATH))) $(warning /sort$(if $(COMSPEC),.exe)) $(warning $(addsuffix /sort$(if $(COMSPEC),.exe), $(subst :, ,$(PATH)))) $(warning $(wildcard $(addsuffix /sort$(if $(COMSPEC),.exe), $(subst :, ,$(PATH)))))
Although a bit tedious, without a real debugger, this is the best way (and sometimes the only way) to determine the value of various subexpressions.
Common Error Messages
The 3.81 GNU make
manual includes an excellent section listing make
error messages and their causes. We review a few of the most common ones here. Some of the issues described are not strictly make
errors, such as syntax errors in command scripts, but are nonetheless common problems for developers. For a complete list of make
errors, see the make
manual.
Error messages printed by make
have a standard format:
makefile:n: *** message. Stop.
or:
make:n: *** message. Stop.
The makefile
part is the name of the makefile or include file in which the error occurred. The next part is the line number where the error occurred, followed by three asterisks and, finally, the error message.
Note that it is make
’s job to run other programs and that, if errors occur, it is very likely that problems in your makefile will manifest themselves as errors in these other programs. For instance, shell errors may result from badly formed command scripts, or compiler errors from incorrect command-line arguments. Figuring out what program produced the error message is your first task in solving the problem. Fortunately, make
’s error messages are fairly self-evident.
Syntax Errors
These are usually typographical errors: missing parentheses, using spaces instead of tabs, etc.
One of the most common errors for new make
users is omitting parentheses around variable names:
foo: for f in $SOURCES; do ... done
This will likely result in make
expanding $S
to nothing, and the shell executing the loop only once with f
having a value of OURCES
. Depending on what you do with f
, you may get a nice shell error message like:
OURCES: No such file or directory
but you might just as easily get no message at all. Remember to surround your make
variables with parentheses.
missing separator
The message:
makefile:2:missing separator. Stop.
or:
makefile:2:missing separator (did you mean TAB instead of 8 spaces?). Stop.
usually means you have a command script that is using spaces instead of tabs.
The more literal interpretation is that make
was looking for a make
separator such as :
, =
, or a tab, and didn’t find one. Instead, it found something it didn’t understand.
commands commence before first target
The tab character strikes again!
This error message was first covered in the section Parsing Commands in Chapter 5. This error seems to appear most often in the middle of makefiles when a line outside of a command script begins with a tab character. make
does its best to disambiguate this situation, but if the line cannot be identified as a variable assignment, conditional expression, or multiline macro definition, make
considers it a misplaced command.
unterminated variable reference
This is a simple but common error. It means you failed to close a variable reference or function call with the proper number of parentheses. With deeply nested function calls and variable references, make
files can begin to look like Lisp! A good editor that does parenthesis matching, such as Emacs, is the surest way to avoid these types of errors.
Errors in Command Scripts
There are three common types of errors in command scripts: a missing semicolon in multiline commands, an incomplete or incorrect path variable, or a command that simply encounters a problem when run.
We discussed missing semicolons in the section “Good Coding Practices,” so we won’t elaborate further here.
The classic error message:
bash: foo: command not found
is displayed when the shell cannot find the command foo
. That is, the shell has searched each directory in the PATH
variable for the executable and found no match. To correct this error, you must update your PATH
variable, usually in your .profile (Bourne shell), .bashrc (bash), or .cshrc (C shell). Of course, it is also possible to set the PATH
variable in the makefile itself, and export the PATH
variable from make
.
Finally, when a shell command fails, it terminates with a nonzero exit status. In this case, make
reports the failure with the message:
$ make
touch /foo/bar
touch: creating /foo/bar: No such file or directory
make: *** [all] Error 1
Here the failing command is touch
, which prints its own error message explaining the failure. The next line is make
’s summary of the error. The failing makefile target is shown in square brackets followed by the exit value of the failing program. Sometimes make
will print a more verbose message if the program exits due to a signal, rather than simply a nonzero exit status.
Note also that commands executed silently with the @
modifier can also fail. In these cases, the error message presented may appear as if from nowhere.
In either of these cases, the error originates with the program make
is running, rather than make
itself.
No Rule to Make Target
This message has two forms:
make: *** No rule to make target XXX. Stop.
and:
make: *** No rule to make target XXX, needed by YYY. Stop.
It means that make
decided the file XXX needed to be updated, but make
could not find any rule to perform the job. make
will search all the implicit and explicit rules in its database before giving up and printing the message.
There are three possible reasons for this error:
-
Your makefile is missing a rule required to update the file. In this case, you will have to add the rule describing how to build the target.
-
There is a typo in the makefile. Either
make
is looking for the wrong file or the rule to update the file specifies the wrong file. Typos can be hard to find in makefiles due to the use ofmake
variables. Sometimes the only way to really be sure of the value of a complex filename is to print it out either by printing the variable directly or examiningmake
’s internal database. -
The file should exist but
make
cannot find it either, because it is missing or becausemake
doesn’t know where to look. Of course, sometimesmake
is absolutely correct. The file is missing—perhaps you forgot to check it out of CVS. More often,make
simply can’t find the file, because the source is placed somewhere else. Sometimes the source is in a separate source tree, or maybe the file is generated by another program and the generated file is in the binary tree.
Overriding Commands for Target
make
allows only one command script for a target (except for double-colon rules, which are rarely used). If a target is given more than one command script, make
prints the warning:
makefile:5: warning: overriding commands for target foo
It may also display the warning:
makefile:2: warning: ignoring old commands for target foo
The first warning indicates the line at which the second command script is found, while the second warning indicates the location of the original command script that is being overridden.
In complex makefiles, targets are often specified many times, each adding its own prerequisites. One of these targets usually includes a command script, but during development or debugging it is easy to add another command script without realizing you are actually overriding an existing set of commands.
For example, we might define a generic target in an include file:
# Create a jar file. $(jar_file): $(JAR) $(JARFLAGS) -f $@ $^
and allow several separate makefiles to add their own prerequisites. Then in a makefile we could write:
# Set the target for creating the jar and add prerequisites jar_file = parser.jar $(jar_file): $(class_files)
If we were to inadvertently add a command script to this makefile text, make
would produce the overriding warning.
Last modified: Fri Nov 27 09:56:46 2020
You can buy me a coffee, please
I would appreciate it if you occasionally
buy me a coffee
A note to readers — clicking on a header will navigate to and from the index.
Table Of Contents
- Introduction to Make
- You can buy me a coffee, please
- What is make? What does the command do?
- Why use make for shell scripts?
- Debugging shell scripts without make
- The default input file to make — Makefile
- My First Makefile
- Makefile and targets
- Dependencies and execution order in makefiles
- Using input and output files as targets and dependencies
- How does make know when to execute a recipe?
- Integrating make with your editor
- Using vi/vim and make
- Using GNU Emacs with make
- Variables in Makefiles
- Including a dollar sign $ in a Makefile
- Setting variables in Makefiles
- Setting a variable inside a Makefile
- Setting makefile variables on the command line
- Evaluating Shell Variables with Makefile variables
- Don’t confuse shell variables with Makefile variables
- Errors when executing make
- Syntax errors in the Makefile
- Errors during execution
- while loops in a Makefile
- Line Continuation in Makefiles
- Conditional execution in Makefiles
- Common command line arguments
- Using a different Makefile
- Keep going with the -k argument
- Do Nothing — Make with the -n argument
- Recipe prefixes
- Ignoring errors with the — prefix
- Hiding recipes with the @ prefix
- Ignoring errors with the — prefix
- Conclusion to Part 1 — Using Make with Shell scripts
What is make? What does the command do?
The original Make documentation by Feldman had a wonderful description of the make utility.
When editing code — there are four steps:
- Think
- Edit
- Make
- Test
In other words, wouldn’t it be nice to have a program that takes care of all of the nit-picky details in step 3 for you?
Make
is an excellent program that can help you compile programs.
But suppose
you are just writing shell scripts? Make isn’t really needed,
right?
Well, please allow me to convince you otherwise. I use make all
the time with shell scripts. And the more complicated the process, the
more likely I will create a Makefile for the process.
Why use make for shell scripts?
Some of you might say to yourself I just write shell scripts. Why should I use make?
There are many reasons I use make for shell scripts. Here are some of my reasons:
- • My work flow has
steps
, or an order of execution. - • The shell scripts have multiple commands, options, and/or arguments.
- • I want to document what I did.
- • I am operating on a remote machine and don’t have access to a GUI environment.
- • I work with a team, and I want to share project-related
shell commands without having to have other team members install my
shell scripts and aliases. - • I work with several projects, but want to have a uniform management technique.
- • I don’t want to, or can’t, install additionals tools.
Whenever I create a directory for a new project, even if it’s a
handful of shell scripts, I create a Makefile. The best
documentation is the source code, and make is a great place to
start. And if I visit an old project, looking at the Makefile
refreshes my memory of what I did, how I used the scripts, and why I did certain things.
After all, once you find the optimum
command and arguments, the steps in a Makefile can immortalize
the exact arguments and sequence of operations.
Sometimes I type in long lines that execute a complex shell
script. I want to document it, but I don’t want to create a full-blown
shell script, especially if it’s just a single line. I could make it
an alias, but repeating this a hundred times makes an alias file
unwieldy. I could put it in a text file to document/remember it, but I also want to execute it.
I could write a shell script, but I want to make it easy to share with team members. I could
create a directory called scripts that contains all of my
recipes, but then I have to get team members to install this in their environment.
Or, I could use make. (rimshot)
Debugging shell scripts without make
Let’s examine a simple
programming work flow without make.
I could just type in a shell script in one line, but if it’s complicated, and I might want to do it again, I should keep it handy by putting it into a shell script.
Normally I have two windows
open — one is editing the file, and the other is using
the
file. For example — I could be editing a awk, bash, sed or python
script using My Favorite Editor™. And the other window is
a terminal window running a shell.
I edit the file in one window, then save the edit. I then go to
the other window and run
the program, typically by typing an up
arrow or !!
history command. Or I use command line editing to make changes. Standard operating procedure.
Sometimes you have to switch between two or more commands in the
shell window. This is easy to do with a shell that understands your
history of commands. However, your work flow might have steps. And you might
forget a step, or perhaps do them in the wrong sequence. Get it wrong and
your result won’t be right.
Perhaps the exact shell command you type is
very long and complicated, and you have to look at a manual page to remember what is needed.
Or suppose you want to use various combination of commands as
you experiment with the results, such as changing an option for one of the arguments. This can be complicated in a multi-step process.
And then there’s the classic problem of executing your program or
script, but you forgot to tell the editor to save the changes. We’ve all done this.
These are the problems make is made for.
System Adminitrators — You want to look
at log files and look for patterns and odd activity? Keep reading.
Let’s get started.
The default input file to make — Makefile
If you simply type the command
make
You will get the error make: Nothing to be done for
. This is because you haven’t provided
‘all’.make
with
enough information, and since there is no direct brain input (yet), it
normally gets this information from a file with the fiendish name
of makefile
.
To be more precise, the make
command first looks for the
file with the name makefile
and if this file does not exist, it
then looks for the file Makefile
. Note the second choice starts with a capital letter.
Most of the time, when programmers create a project, they provide the file
Makefile
instead ofmakefilefor a couple of reasons. Normally, the ls
command will sort filenames so that files that start with a
capital letters are shown before those beginning with
lower-case letters. Therefore this file is often listed before
the others. This is also why people provide a file called
README— to make it more noticeable.The second reason people use
Makefileovermakefileis to
make it easier for another person to override the default
behavior. If you get a source package with aMakefileyou want
to change, (e.g. to change the destination directory) just copyMakefiletomakefileand edit that
one. This keeps the original one around as a backup. It also allows you to test changes easily.
Sounds like a good convention. Let’s begin.
My First Makefile
Makefile and targets
Let’s create a very simple Makefile. Edit a file and give it the name Makefile
. The file should contain the following:
all:
./script1
The syntax is important, and a little fussy.
Let me provide a more precise description of the syntax, which in general terms is:
<target>: <prerequisites>
<tab> <recipe>
<tab> ...
Let’s break it down.
The word all
is the name of a target. The recipe in our makefile is ./script1
The prerequisites
is optional. I’ll cover that later.
For now, the recipe makes
the target, or in our case, you execute ./script1
to make all
.
Or to put it another way, when you type
make all
the make program executes ./script1
for you.
Congratulations on your first makefile, by the way. Yeah, I know. Underwhelming. Let’s build up on this.
As the specifications suggest, you can have more than one
recipe for a target. Each recipe is executed by passing the line to
the shell for execution. However, and here is the fussy part, there
must be a tab character before each recipe. Note that you
must use a tab character, and not space characters, before
recipes.
If you forget this, make will report an error such as:
Makefile:2: *** missing separator. Stop.
In this example, make reports the filename, and the line number of the error — in this case 2
.
If you are using an editor that changes tabs to spaces, this will cause
your Makefile to break. And if you cut and paste my examples, you
may have to change leading spaces into a tab.
With the basics out
of the way, we are ready to use make in a more realistic sense.
Suppose you have three independent scripts, named script1, script2, and script3.
all: step1 step2 step3
step1:
./script1
step2:
./script2
step3:
./script3
I’ve added three new targets and recipes. You can type make step1
, etc. to execute just one of the programs.
Notice, however, that we added step1, step2 and step3 as a prerequisite to the all
target.
If you type make all
— make builds all three targets.
This is because the target all
has three sub-targets — step1, step2 and step3.
If you don’t give make a target, it builds the first one in the file.
It’s very common to name the first targetallso make builds everything by default.
If for example, you are a system administrator that has to check
several log files, and you perform the same actions each time,
you can put them all into a Makefile.
You can use make to help you remember complex recipes that are project related, and be consistent in operations.
For example, suppose to had to generate a report for multiple projects in multiple directories. The shell commands may be different, but you can use make report
— that is — the same command — for all projects as long as you adjust the Makefile in each directory.
Or suppose you had to do a log or data analysis for several projects/programs. You could use the command make logs
for each project, without having to install or share scripts between teammates.
And there are times when you have to experiment with a long and
complex shell command just to get something working, and once it’s
done, it’s done. Yes, you can forget about it, but sometimes it’s a
good idea to capture these long and complicated commands, to make it
easier for you to re-use it the next time. For example, I’ve used a
wget
command with 8 to 10 command-line options, but I may not
remember these exact commands six months later.
So let’s learn to master the Makefile and add some dependencies to the recipes.
Dependencies and execution order in makefiles
Let’s assume that these programs/scripts have to be executed in a
certain order, in this case, step1, step2 and step3 in that sequence. We can
do this with a minor change to the earlier makefile:
all: step3
step1:
./script1
touch step1
step2: step1
./script2
touch step2
step3: step2
./script3
touch step3
clean:
rm step1 step2 step3
Note that my Makefile
creates three files named step1, step2, and step3 using
the touch command. This updates the time stamp on a file, and if the
file doesn’t exist, it creates an empty file.
Also note that the all target just has step3 as
a prerequisite, or dependency. And if you look at step3, it has
step2 as a dependency. That is, before script3 is
executed, step2 has to exist. And step2 is created
when script2 is executed. But that won’t happen until step1 is created.
In other words, we’ve created not just a set of steps, but an explicit sequence or oder of operation.
Make understands these dependencies and keeps track of them for you. If you execute make
it will perform the following commands
./script1
touch step1
./script2
touch step2
./script3
touch step3
However, if you execute make a second time, it will do nothing, because everything is already made. In otherwords, the prerequisites have been met.
The files step1, step2 and step3 have to be removed if you want to execute the three scripts again.
That’s why I added a target called clean
that when executed, it removes these files. You can type make clean;make
and the three scripts will be executed again.
The
make cleantarget is a common convention in makefiles.
Using input and output files as targets and dependencies
I used the files step1 etc., as targets. This works, and is simple, but it can cause problems. These files can get out of sync with your scripts and programs. It’s better to have targets and dependencies that are part of the flow of data.
We have three scripts above. Let’s assume that they are normally piped together, like this:
./script1 <data.txt | ./script2 | ./script3 >script3.out
I’m going to modify the makefile to perform the above script, in steps:
all: script3.out
script3.out: script2.out
./script3 <script2.out >script3.out
script2.out: script1.out
./script2 <script1.out >script2.out
script1.out: data.txt
./script1 <data.txt >script1.out
clean:
rm *.out
This eliminates the need for those do-nothing
step files. Instead, we are using files than contain real data.
If we type make
the scriptram will execute
./script1 <data.txt >script1.out
./script2 <script1.out >script2.out
./script3 <script2.out >script3.out
This is closer to a real makefile, but there is an important
feature missing. Make can deal with data files, but its real
power is dealing with source code. In this case, if script1
etc. is a script, we want make to realize that if the script
changes, the output will as well. So a more useful makefile
would look like:
all: script3.out
script3.out: ./script3 script2.out
./script3 <script2.out >script3.out
script2.out: ./script2 script1.out
./script2 <script1.out >script2.out
script1.out: ./script1 data.txt
./script1 <data.txt >script1.out
clean:
rm *.out
If you look at the target script3.out
, you will see two prerequisites now. As before, we have script2.out
, but we added script3
as well. That is, if you edit the script or change the input data — you have to rerun the script.
Now we’re ready. This version will detect when the program (or script) changes, and when it does, it re-runs that step.
Don’t forget that your scripts might have additional dependencies.
For example, if script2
executed a awk script called script2.awk
, you may want to add this as a dependency for the script2.out
target. Make doesn’t parse your files, it just looks at timestamps, so you have to add these dependencies yourself.
While you could just pipe the three scripts together, this methodology is very handy when debugging — especially if you don’t know which script has the bug, and if the scripts take a long time to execute. Not only does it just rerun the minimum steps after a script change, but it captures all the data that would normally be discarded if the scripts were piped together.
How does make know when to execute a recipe?
The make utility uses the time stamps and the recipes to
determine when it has to regenerate a target. In the example above, it
looks at the time stamps of the input file and the executable
program/script to determine if any action is needed. Make also
has some build-in rules it uses, but we will cover that in a future tutorial.
If you are a system administrator, you can see that this can be
useful, because make
will detect if a log file changes, and
only generate new data if it does.
Integrating make with your editor
Integrating your editor
with make can improve your efficiency a lot. How many times
have you tried a code fix to discover that you forgot to save the
changes? Instead, your editor will automatically save files for you,
and even better, auto-guide you to the line in the file that has the
error? You want to do this, gang. Let’s go.
Using vi/vim and make
First of all, I have to apologize that this is just an
intro into vim/make integration. To be honest, I used vi
for about a decade before vim 2.0 was released, and by then I
switched to Emacs. Consequently, I never mastered the advanced
features of vim. But I’ll do the best I can to get you started.
For you vim users, a first step you may want to do is to map
a function key to the make command. Once example is to add this
to your ~/.vimrc file — in this case the F9 key:
" You want to save files when you execute make. You have two choices. Either:
" (1) set autowrite whenever you execute :make
set aw
" or (2) Press F8 to save all open files
:map <f9> :wa
" and to execute make, press the F9 key:
" Press F9 to execute make
:map <f9> :make
It true that binding a function key to make just saves a few steps, as you can always type :make
in vi.
When you want to execute make, you do have to make sure your changes are saved to the file system.
If you like option 2, you have to press the F8 function key.
If you have option 1, by setting autowrite on, then this will happen automatically.
Now, when you press the F9 function key, vim will execute the make command,
allow you
to specify the target and arguments, and wait for you to press the
ENTER key. When this is done, vim will show you the results.
You can review the results of the compile using the :copen
(abbreviation :co
)
command which opens up a quickfix window. If there are errors that vim can parse, you can press ENTER and
jump to the different errors, and edit the file. For example, if you
have an error in your Makefile, you can use the quickfix window
to jump to the line that caused the error.
The command :cclose
will close this window, and you can toggle this using the :cw
command.
Vim allows you to defined the makeprg variable to
execute make with command line options, or to use some program
other than make. I saw one example where someone changed
makeprgto execute the java compiler. Personally, I think this
is a bad idea. First of all, if a second person wishes to duplicate your
steps, they have to modify their own vim startup file the same
way. Second, they also have to use vim instead of their
preferred editor.
These options belong in a Makefile, which becomes documented as part of the source code,
rather than hidden in a personal preference.
Using GNU Emacs with make
GNU Emacs has support for make built in. However, you may want to map a keystroke to the make command.
I use the following to map Control-C m
and Control-C M
to make.
;;; map Control-C m and Control-C M to make
(global-set-key C-cm" 'compile)
(global-set-key "C-cM" 'compile)
Sometimes I have shift on by accident, so I’ve mapped both upper
and lower case M
to make.
When I press Control-C
, Emacs will ask me if I want to save any files that have not
m
been saved. It then prompts me with the default make command
with arguments. I can edit this and it will remember this for next
time. Then Emacs will launch the make command and pipe the results
into a buffer called *compilation*
. If this process takes a
while, you can still edit your scripts and data while watching the results. The status is updated
on the status line, which says Compilation:exit
when make is finished. I find this handy for long-running jobs.
If you repeat the command, and it is still running, Emacs will ask you if you want to terminate the current compile.
The command Control-C Control-K
will kill the current compilation. I do this often when I realized I goofed and want to re-run the compile job.
Once the compilation is done, you can press Control-X `
,
which is command-next-error. This will read the errors in the
compilation buffer, locate the first error, and go to the line in the
file that caused the error. If you repeat this command, it will go to
the next error, even if it’s in a different file. Using this, you can
quickly navigate through your errors and then recompile. This won’t
work very well in programs that don’t generate errors that can be
parsed.
If you are dealing with more than one Makefile
, such as
nested directories, or multiple projects, the compile command
will run in the current directory of the file you are actively
editing.
If you are using Emacs in graphics mode, and you edit a Makefile
, the menubar will show current Makefile commands. I’ll try to describe these in a later tutorial.
I wanted to make sure you understood the normal workflow. It’s time to learn how to adjust your Makefile to be more a more productive user.
Variables in Makefiles
Make was an early program in the history of Unix systems, and the support for variables is unusual. Variables are indicated with a dollar sign, but the name of the variable is either a single letter, or a longer variable name surrounded by parentheses. Current versions of make also allow you to use curly braces as well. Some examples of variables in a recipe are:
all:
printf "variable B is $Bn"
printf "variable Bee is $(Bee)n"
printf "variable Bee is also ${Bee}n"
You have to be careful, because if you used $Bee
in a make file, you are referring to the variable $B
with the letters ee
appended to it. So if variable $B had the value Whoop
, then $Bee
has the value Whoopee
.
Consequently, I always use parentheses or curly braces (it doesn’t make any difference which one you use) around variables, to make sure no one confuses $(B)ee
with $(Bee)
There are special variables whose names are special characters. I’m not going to cover those yet.
Including a dollar sign $ in a Makefile
Since the dollar sign indicates a variable, what do you do if you want a dollar sign left alone? In this case, simply use $$
to indicate a single dollar sign. For instance, if you wanted to use a regular expression containing a dollar sign in a makefile, you would need to double the dollar sign. Suppose you wanted to search for all lines that had a number as the last character, using the regular expression«[0-9]$»
, a sample recipe would be:
grepthelines:
grep '[0-9]$$' data.txt
Note that make interprets the lines and evaluated variables before it passes the results to your shell. The shell only sees a single $
. Normally, meta-characters within Strong Quotes are left alone. But make processes the lines before it sends them to the shell.
Setting variables in Makefiles
There are two ways to set Makefile variables. I’m not talking about shell or environment variables.
Setting a variable inside a Makefile
Setting variables is simple; simply put them on a line (not starting with a TAB character), with an equals sign. Variables can refer to other variables. An example might be
# Example of a Makefile with variables
OUTFILES = script1.out script2.out script3.out
TMPFILES = script1.tmp script2.tmp script3.tmp
TEMPORARYFILES = $(OUTFILES) ${TMPFILES)
# Which programs am I going to install?
PROGS = script1 script2 script3
# Note I am using a shell environment variable here
INSTALL = /usr/local/bin
clean:
rm $(TEMPORARYFILES)
install:
cp -i ${PROGS} ${INSTALL}
The variable TEMPORARYFILES is the value of
two other variables concatenated — as strings. It does make sense that
make variables are string-based.
Setting makefile variables on the command line
In the last example, if you executed the command make install
, make would copy files to the /usr/local/bin directory. You can over-ride this on the command line. The syntax is
make target [variable=value ...]
For example, if you typed
make install INSTALL=~/bin
Then make would install the programs into ~/bin.
Evaluating Shell Variables with Makefile variables
When you define variables, the shell is used to evaluate the line before it passes the results to make. For example, you can use the following definitions:
OUTFILES = *.out
INSTALL = $${HOME}/bin
OUTFILES will be equal to all of the files that match the pattern *.out
while the INSTALL variable is based on the environment variable HOME.
Don’t confuse shell variables with Makefile variables
Note that I used $${HOME}
in the previous example. The double dollar sign tells make to pass a single dollar sign to the shell, which uses it to get an environment variable.
Don’t make the mistake of trying to set a variable in a recipe and assume it works the same as a definition. Here’s is an example with both:
#Define a variable in a Makefile
A = 123
all:
A=456; printf "A = ${A}n";
A=456; printf "A = $${A}n";
printf "A = $${A}n";
This will print
% make
A=456; printf "A = 123n";
A = 123
A=456; printf "A = ${A}n";
A = 456
printf "A = ${A}n";
A =
The first time printf is executed, A single $ is used, so the definition in the Makefile is used. The second time a double dollar sign is used, so the shell variable is used, and 456
is printed.
The third printf prints the shell variable A
as an empty string, because it is undefined. Each line of the recipe is executed by a new shell. In this example, three shells are executed — one for each of the lines containing printf. And each shell has it’s own view set of variables.
Please note that when a recipe has multiple lines, each line is executed with its own shell. Variables set in one line are not passed to a second line.
Also note that you can execute recipes which launch a background job by ending the recipe with an ampersand.
Errors when executing make
There are a few ways you can make a mistake, and have make complain.
Syntax errors in the Makefile
First of all, Make will complain about errors in a makefile.
If you don’t have a tab before a recipe, it will complain
Makefile:linenumber: *** missing separator. Stop.
If you ask it to make a target and it doesn’t know how, it will tell you there is no rule. It can detect loops in dependancies as a few other errors.
Errors during execution
The second type of error occurs when make is executing other programs.
If you execute a complex series of steps or recipes, and an error occurs, make will stop. For example, in my earlier example:
all: script3.out
script3.out: ./script3 script2.out
./script3 <script2.out >script3.out
script2.out: ./script2 script1.out
./script2 <script1.out >script2.out
script1.out: ./script1 data.txt
./script1 <data.txt >script1.out
clean:
rm *.out
If the script script1 has an error, then make will stop execution right after it tries to create script1.out
If there are more than one recipe for a target, make will stop after the first error. If there were four lines (or recipes) to execute, and the third one aborts because of an error, the fourth line won’t get executed.
This is intuitive, because it’s what you want to happen. But there are some subtle points.
Suppose you had the following recipe:
test:
false;true
exit 1;exit 0
The false
program is a standard Unix command that simple exits with an error. If it was the only command on that line, then make will halt execution. However, make asks the shell to execute several commands on the line, (false
and true
), and the last command didn’t fail. So the shell tells make that the line executed successfully. This ignores
the previous error on the line. So make then executes the second line.
In this case, the shell executes exit 1
directly, just like the false command does, but this time the shell — which is executing the programs, itself exits, which causes make to stop. The exit 0
is never seen.
while loops in a Makefile
Sometimes you need to write a script that processes a file, and it
only processes a single file at a time. Yet you need to repeat
this for several files.
As an example, I wanted to search a directory for PDF files and
calculate the SHA1 hash for each one. I used this to see if any PDF
files changed, or new ones were created. I used the following recipe:
genpdfsums:
./FindPDFFiles | while IFS= read line;do sha1sum "$$line";done >PDFS.hashs
Note that I set IFS to an empty string in case any filename has a space as part of the name. This assumes each input line is a filename.
Line Continuation in Makefiles
Because make executes a shell, you can use the same conventions for line continuation as the shell. For example, here is a recipe I used when I examined the results of a wget command to extract URLs:
TRACE = cat
#TRACE = tee /tmp/trace.out
all_urls.out: Makefile
cat */wget.err | grep '^--' |
grep -v '(try:' | awk '{ print $$3 }' | ${TRACE} |
grep -v '.(png|gif|jpg)$$' | sed 's:?.*$$::' | grep -v '/$$' | sort | uniq >all_urls.out
When my makefiles get complicated, I often put Makefile
as a dependancy in the target. In the above example, if I edit the Makefile, then all_urls.out
is out of date.
That is, my source code
is my Makefile.
You may also notice I included an example of how I integrate make with shell script debugging.
In this case I created a variable called TRACE
which I can use to debug my shell script. By changing the definition of TRACE
, I can capture the output in the middle of a complex shell script.
Conditional execution in Makefiles
Sometimes you need to test for a condition and execute part of a
shell script. The shell has the commands «||» and «&&» which can
be used to execute simple flow
control. The characters «&&» execute the rest of the line if
the condition is true. To test for a false condition, use «||».
Let’s suppose you are testing to make sure two programs generate the same results, but you only want to test this when the programs change.
Here’s one way to do it. If either «script1» or «script2» change, then
execute them. If the results differ, execute «report»
diff: script1.out script2.out
diff -b script1.out script2.out >/dev/null || ./Report
script1.out: script1
./script1 >script1.out
script2.out: script2
./script2 >script2.out
I redirected the output of diff to /dev/null to hide the
info when I run the make command. I’m just interested in the status of
the command, not the data.
Here is another example where I examine a directory for files, looking
for new files that I haven’t scanned before. I keep track of all
of the files I have scanned in the file «scanned.log». If I find a
file that is not in that file, I check it using a program canned
«Scan». That program will append the filename to the file
«scanned.log». This Makefile recipe does this by using two while
loops. The first one is given a list of filenames using find, and the second one only sees
filenames that are not in the file «scanned.log». I’m
using grep with the «-q» quick and the «-s» silent option
to suppress error messages about unreadable files. To repeat, I just want the job status of grep, not the data or errors.
To elaborate, the first while loop executes a test if the
filename is in the scanned.log file. If it is not in the file, the
echo command sends the filename to standard output, which is, in
this case, the second while loop. The second loop scans the
file.
This way, a complex makefile will execute commands only if it
hasn’t been executed before, but using data managed by your own
program.
all:
find . -name *.txt | while read f;do grep -sq "$$f" scanned.log || echo "$$f" ;done |
while IFS= read n;
do Scan "$$n";
done
Common command line arguments
I’ll discuss some of the common command line arguments I use with
make. Note that some of these options have multiple arguments to
specify the same option, (e.g. -f filename
or —file=filename
) . Check the manual page for more details.
Using a different Makefile
As I mentioned before, make first looks for the file makefile
, and if this
isn’t found, it looks for Makefile
. You can override this with the -f
option:
# execute make with the default name file
make
# Execute make using the file Makefile.new
make -f Makefile.new
You can Use this to debug your makefiles, or manage variations.
Keep going with the -k argument
Normally, make stops when an error occurs. You can use the
-k
argument to tell make to ignore the error and keep
going. You can to be careful with this if errors cause data to be
invalid. Buf if you have a long-running build process, you may want to
keep going as long as possible.
Do Nothing — Make with the -n argument
When you start using make, you may want to see what
make will do before you execute it. I often have this issue
when I look at someone else’s Makefile. Typing sudo make
on someone else’s code should make you very
install
nervous. Besides examining the contents of the Makefile, there
is another choice: execute make -n
. The -n
option is a
do-nothing option, It just prints the commands that would be executed,
without executing them. Nothing is changed and no files are created.
Recipe prefixes
Certain characters can be used in the front of a recipe to control how make execute the recipe. These can make make (I couldn’t resist), less annoying.
Ignoring errors with the — prefix
You can use the -k
option to ignore errors, but that’s not always what you want. Suppose, for instance, you want to delete all *.out files. If the files exist, no problem. But if you execute a rm *.out
recipe when no files that match that pattern exist, this will generate an error, and make will stop.
You can put the prefix —
at the begnnning of a line to tell make to ignore any run-time errors on that line. Here’s an example:
all:
....
clean:
# The - in the next line is a prefix
-/bin/rm *.out
printf "this executes even if no files are deleted"
However, you will still see the error. It just won’t cause make to exit.
Hiding recipes with the @ prefix
I’ve used printf
in some of these examples to document my makefiles. However, when you execute make, we end up seeing make show us the printf command before it executes, and then the printf command executes — so we end up seeing this notice twice. You can prevent make from echoing lines to the terminal using the @
prefix:
all:
....
clean:
# The - in the next line is a prefix
-/bin/rm *.out
@printf "All clean now"
You can combine these prefixes.
test:
@-/bin/rm *.out >/dev/null 2>1
@printf "done"
However, this may still generate errors. For instance, I still get the message:
Makefile:2: recipe for target 'test' failed
make: [test] Error 1 (ignored)
If I add a touch junk.out
to the recipe:
test:
@touch junk.out
@-/bin/rm *.out >/dev/null 2>1
@printf "done"
Then when I execute this recipe, the only thing I see is done
Conclusion to Part 1 — Using Make with Shell scripts
I think I’ve given you enough to start to combine make with your shell scripts.
Make‘s greatest power is used when compiling source code, but I find it helpful when writing complex shell scripts.
For example, I had to create dozens of long, complex shell commands when I worked on
a project where I had to search an external web server for confidential information
I used a dozen different tools to scan the server, extract and download all files, look for duplicates, extract the metadata from these servers, and then scan the metadata for keywords. I had to get this done as quickly and efficiently as possible, and I wanted to keep track of everything I tried, so I could reuse it in the future. My main tool was make.
A future tutorial will cover the advanced features used for programmers.