Parallelized Make

June 10, 2008

Since the introduction of dual core machines the number of mono core machines is declined and almost no computer of that kind are sold at all. Having such a machine is clearly a big advantage, but for many things you must tell programs that you are on a multi-cpu machines to make them use all the power. So i wrote a very little and simple script to replace the default make.

Just place this script in a path before the standard make. I made a folder ~/.bin in my home directory and modified the PATH accordingly export PATH=~/.bin:$PATH


#!/bin/sh

# verbosity check
if [ "$VERBOSE" = "yes" ] ; then
	verbose=yes
fi

# Get the number of cores
if [ -z $NCPU ] ; then
    tmp=$(sysctl hw.ncpu)
    NCPU=${tmp#hw.ncpu: }
	if [ "$VERBOSE" = "yes" ] ; then
		echo "Parallelization enabled with $NCPU threads"
	fi
fi

j=$(($NCPU*5/2))
if [ ! -z $NOPMAKE ] ; then
    echo "Parallelization Disabled" >&2
    /usr/bin/make $@
    exit $?
fi

# Start the compilation in parallel
if [ ! -z "$DISTCC_HOSTS" ] ; then
    if [ -z "$HOST_COUNT" ] ; then
        export HOST_COUNT=$(echo $DISTCC_HOSTS | wc -w)
    fi
    /usr/bin/make CC='distcc /usr/bin/gcc' CXX='distcc /usr/bin/g++' -j$((j+$HOST_COUNT)) $@
else
    /usr/bin/make -j$j $@
fi

To make it work on linux the sysctl command must be replaced with a count of the lines containing “processor” in the file /proc/cpuinfo. Something like grep processor /proc/cpuinfo | wc -l