Flocking condor pools over firewalls

At Imperial we have a number of condor pools, some of which are behind firewall (e.g. Saturn pool, Viking pools), while some are outside the firewall (DoC pool, Thor pool).

Since version 6.4.6, one can specify LOWPORT and HIGHPORT in Condor's config file(s) which controls (most of) the communication used by condor. Moreover, there are some control ports used by condor (as defined in $CONDOR_DIRECTORY/src/condor_includes/condor_network.h):

#define NEGOTIATOR_PORT 9614
#define ACCOUNTANT_PORT 9616
#define START_PORT 9611
#define START_UDP_PORT 9611
#define COLLECTOR_PORT 9618
#define COLLECTOR_UDP_PORT 9618
#define COLLECTOR_COMM_PORT 9618
#define COLLECTOR_UDP_COMM_PORT 9618

We have successfully managed to flock condor pools from both sides of the firewall. Furthermore, using this knowledge and experience we have managed to flock to and from a condor pool at Southampton University (also behind firewall at Southampton). Below is what one needs to modify in the firewall configuration in order to enable condor flocking.

Modifications in the firewall(s) and condor config file(s)

    Of course, steps described below need to be done on both ends.

  1. First thing is to modify HOSTALLOW_WRITE and HOSTALLOW_READ in the condor_config file. For example, we have:

    HOSTALLOW_READ = site1_pool_IP, site2_pool_IP
    HOSTALLOW_WRITE = site1_pool_IP, site2_pool_IP

    where siteX_pool_IP corresponds to IP address of resources on siteX (e.g site1_pool_IP = 146.169.*.*).

  2. Next, you adjust FLOCK_FROM and FLOCK_TO values in condor_config file:

    FLOCK_FROM = master_host_site1, master_host_site2
    FLOCK_TO = master_host_site1, master_host_site2

  3. In the same file one needs to add values which will represent Condor port range:

    HIGHPORT = value_hp
    LOWPORT = value_lp

  4. And finally, some additions are need in the firewall configuration. If host1_PORT_RANGE and host2_PORT_RANGE correspond to LOWPORT-HIGHPORT on host1 and host2 respectively, then the following communication occurs:

    Modify the firewall configuration accordingly.