Note: GNU make (as well as some other make implementations) offer the -j [number] option, which launches up to number compilation jobs in parallel, dependencies permitting. Leaving out the optional number means launching as many independent jobs as available.
I had to modify the makefiles for my DESO BLAS library a bit in order to make sure that parallel execution of certain commands was safe (the mechanism used by GNU make only looks at dependencies).
Comparison: The following diagram compares the
compilation times on my old desktop (lazy), on a dual processor
Sun Ultra-SPARC-2 server, on a 1GHz Pentium III and finally on my
new machine charis, which is a dual Athlon 1600+. Note that all
compilations were run over NFS (100 Mbit/s ethernet connection to
server).
Conclusions

With acknowledgement to Tony Fisher http://www-users.cs.york.ac.uk/~fisher/ .