ZenSRC

This is a follow up article to my C++ Byzantine Generals Coding Sample which focuses on the Makefile part of the project.

Over the years, I’ve heard many complaints directed at Make and Makefiles (and I’ve done a bit of it myself). I think the biggest problem coders have with Makefiles is that they are nearly a black art in and of themselves. I think most programmers would rather spend time on the actual code they write than on figuring out the sometimes complex syntax that gets their compiler to build their source files in an efficient manor.

This post is intended to pull together some of the best practices I’ve found and to discuss some of the neat tasks that I have my Makefile doing in the hopes that it helps get you to a solution faster.

First up, the help target. You’ll notice in the full Makefile that the help target appears first. Many makefiles typically have “all” first, which usually makes the default behavior perform a complete build. I’ve found that printing usage information on the available targets far more preferable. It gives new users an idea of what the makefile does, and what to expect from each target.

.PHONY: help
help:
	@echo 'Commonly used make targets:'
	@echo '  all          - build program (use release and debug targets instead)'
	@echo '  check        - build program and run valgrind memory check tests'
	@echo '  clean        - remove files created by other targets'
	@echo '  debug        - build program with debugging enabled and optimizations disabled'
	@echo '  docs         - run doxygen to produce documentation'
	@echo '  graph        - build program and run, generating output graphs'
	@echo '  print        - creates a pdf listing of the source code'
	@echo '  release      - build program with debugging disabled and optimizations enabled'
	@echo '  TAGS         - build TAGS database for source code'

Next, we provide targets for release and debug. All we’re doing here is modifying the $(CFLAGS) variable such that it uses the correct compilation flags for the relevant target. These get modified and passed on to the “all” target and that way we keep the rest of the rules clean.

#
# We use target-specific variable values for debug and release build targets,
# see <http://www.gnu.org/software/make/manual/make.html#Target_002dspecific>
#
.PHONY: debug
debug: CFLAGS += $(CDEBUG)
debug: all

.PHONY: release
release: CFLAGS += $(CRELEASE)
release: all

Next up is the heart of the compilation part of the makefile. This line literally says that for each of the object files we need to build, use the corresponding C++ (.cpp) file from the source directory. Prior to constructing and passing the compilation line to the shell, we perform static code analysis on our input source files. Essentially, we’re stacking the deck in our favor by attempting to catch subtle errors early and often. In the event that the static analysis tool (cppcheck) finds anything, it will force the compilation to halt and require the developer to address the relevant issue. Finally, I found a great resource concerning correctly generating the list of dependencies, and thats where the dependency generation lines originate.

# Simple brute force rule to build obj/*.o based on their src/*.cpp counterparts
# Dependency generation reference <http://make.paulandlesley.org/autodep.html>
#
# Also perform static analysis on all of the files, in another effort to
# stack the deck in our favor and catch errors early (Force our static
# analysis tool to return an error and halt the makefile when it finds
# problems).
.PHONY: $(objdir)%.o
$(objdir)%.o: $(srcdir)%.cpp
	@mkdir -p $(dir $@)
	@echo "============="
	@echo -n "Static analysis: "
	cppcheck -q --error-exitcode=1 $<
	@echo "Compiling $<"
	$(CC) $(CFLAGS) -c $< -o $@
	@echo "Dependency generation $<"
	@cp $(objdir)$*.d $(objdir)$*.P; \
		sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\$$//' \
		-e '/^$$/ d' -e 's/$$/ :/' < $(objdir)$*.d >> $(objdir)$*.P; \
		rm -f $(objdir)$*.d

This next target, “check” is fairly straight forward, but immensely useful. It runs the excellent Valgrind memory analysis and profiling tool over our project. Ideally, this would be incorporated into a batch of automated unit tests, but more on that later. It’s enough to say that we’re taking another step towards detecting issues early and often by providing a common and simple way for all developers to run this tool via the make system.

.PHONY: check
check: debug
	@echo "============="
	@echo "Performing valgrind analysis"
	valgrind --track-origins=yes --leak-check=full $(TARGET) $(TARGET_BASE_ARGUMENTS)

Another simple target is the automated generation of Doxygen documentation. I have the Doxyfile set to spit out warnings to a text file, doxy_warn.txt, so the developer can visit it after the make process and take care of any issues that were found. Doxygen is an immensely useful tool, and when leveraged correctly, can make the process of providing API documentation to third parties especially nice. Here is the doxygen documentation that my coding sample generated.

.PHONY: docs
docs: graph
	@echo "============="
	@echo "Creating Documentation using doxygen"
	doxygen ../doc/Doxyfile
	cat ../doc/doxy_warn.txt

Finally, the “TAGS” target. Exuberant Ctags is another terrific development tool that allows developers to quickly navigate source code. I have the TAGS target rebuilt on “all”. This keeps our source code database fresh.

# Do not add phony rule, we want TAGS to be checked to save a little build time.
TAGS:
	@echo "============="
	@echo "Creating TAGS"
	etags -o $(srcdir)TAGS $(srcdir)*.cpp  $(srcdir)*.h

So here are a few things that this makefile doesn’t do, which it normally could. These aren’t done mostly because the coding sample I wrote didn’t require it.

• Automate unit testing (run unit tests, verify passage, generate and distribute reports).
• The makefile isn’t setup to do builds which require multiple components built across multiple directories.
• Doesn’t perform installation.
• Doesn’t dependency checks.

Occasionally, I’m asked to interview new college graduates for software engineering positions at my company. Over time, I have accumulated what I believe to be a good set of interview questions that check breadth of knowledge more than depth. I do spend time digging into depth for what the interviewee feels they are strongest at just to make sure they do indeed know what they claim to. This question on Stackoverflow caught my eye yesterday, and the top rated answer took me to the the Programmer Competency Matrix. This table is a terrific reference for testing programmer’s ability and I haven’t seen anything else that’s quite so concise yet covers so many of the bases.

As hard as it is, I try to keep my resume, cover letter, and letter of references up to date. It gives me an idea of how my skill set is growing over time and also keeps me relatively in tune with what the job market looks like for Software Engineers.

Several months ago, while going through the motions of updating my resume package, I realized that I didn’t have an up to date coding sample. So began the process of becoming intimately familiar with the Byzantine Generals Problem. For a great analysis of the problem and one potential implementation, check out
Mark Nelson’s post on this very problem.

Although Mark provides his own sample C++ implementation, I went in a slightly different direction and leveraged a few APIs, tools, and technologies that I’ve been meaning to try in a project for some time.

I version controlled my solution to this problem: C++ coding sample on BitBucket and also uploaded the associated Doxygen Generated Documentation.

I’ll likely spend a few posts talking about some of the neat things I ran into along the way, but please take a look and don’t hesitate to leave some feedback!

Update for additional parts:

• Byzantine Generals Coding Sample, Part 2 (Makefiles)

This is a follow up to a previous post about GitHub. While Git + GitHub have worked relatively well for my needs, I’ve found one deal breaker that’s missing from GitHub: http push. Http push simply means that you can commit your changes to a given repository over native http. That is a must have, since most of the time I sit behind a rather pernicious corporate firewall.

Lately, I’ve been reading about Mercurial, which in many ways is very similar to Git. For some good reference on the differences, check out this Stack Overflow question. If you’re interested in learning Mercurial (Hg), check out the tutorials at HgInit. It turns out there are some very similar services to GitHub for Mercurial, one of which is BitBucket. As you’ve probably guessed by now, the first listed feature of BitBucket is both http push and http pull. So, I used the built in git-mercurial conversion utility and moved my stuff over and have been happily coding away.

A good friend of mine recently pointed me to an excellent article over at Gamasutra : Book Excerpt: Game Engine Architecture. I liked the excerpt so much that I picked Game Engine Architecture up a few weeks ago. I’m slowly making my way through it, and will have a full review once completed.

Here are some initial thoughts:

• The book is well laid out, with a brief introduction on C++ programming for those that are unfamiliar.
• The book acts as a great jumping off point to further reading. Each topic covered has in-line references to much more in depth coverage of said topic.
• The book is a terrific high level overview of the common problems encountered in Game Engines (Soft real time systems).

One other quick note: even if you work mostly in hard real time systems, as I do, I think you’d be surprised at how useful it is to read about other programming domains. As computer scientists and programmers, we all run into the same sorts of underlying problems. Solutions that work in one domain can often be modified to apply to other domains.

So, the basic premise is this: I work on a variety of computers at work and at home. Many of the aforementioned computers run different operating system (including Windows XP, Windows 7, Ubuntu, Debian, Sun, and OSX). On all of the computers, I use an application that provides a native console (usually bash) or provides an SSH terminal used to access a *nix-like environment. I’m a big linux guy, and like many linux guys, I’ve managed to heavily customize my environment. As you can probably guess, its a pain in the butt to keep a uniform environment going across all of those setups.

Some time ago, I setup my own Subversion (SVN) server on a personal NAS device. Using SVN, I’ve been pulling down my latest environment to each existing and new machine that I work on. Then, I use SVN to easily synchronize my changes across all of the machines that I encounter. This solution was working great up until a few weeks ago when my employer blocked an IP address range that included my home network.

While running my own SVN server has been relatively painless, I think it makes sense to pick a more robust solution going forward. After doing some reading, I’m giving GitHub a try to host those files that I don’t mind the world viewing. The process was relatively painless, and now my environment is stored here. I’ll follow up with my GitHub experiences after I’ve tried it out for a few weeks.