7.8 Parallel UDF Example

The following is an example of a serial UDF that has been parallelized , so that it can run on any version of ANSYS FLUENT(host, node, serial). Explanations for the various changes from the simple serial version are provided in the /* comments */ and discussed below. The UDF, named face_av, is defined using an adjust function, computes a global sum of pressure on a specific face zone, and computes its area average.

Example: Global Summation of Pressure on a Face Zone and its Area Average Computation

#include "udf.h"

DEFINE_ADJUST(face_av,domain)
{
/* Variables used by serial, host, node versions */
 int surface_thread_id=0;   
 real total_area=0.0;
 real total_force=0.0;

/* "Parallelized" Sections */
#if !RP_HOST   /* Compile this section for computing processes only (serial
                  and node)  since these variables are not available
                  on the host */
 Thread* thread; 
 face_t face;                         
 real area[ND_ND];
#endif /* !RP_HOST */


/* Get the value of the thread ID from a user-defined Scheme variable */
#if !RP_NODE /* SERIAL or HOST */
 surface_thread_id = RP_Get_Integer("pres_av/thread-id");
 Message("\nCalculating on Thread # %d\n",surface_thread_id);
#endif /* !RP_NODE */

 /*  To set up this user Scheme variable in cortex type */
 /* (rp-var-define 'pres_av/thread-id 2 'integer #f) */
 /* After set up you can change it to another thread's ID using : */
 /* (rpsetvar 'pres_av/thread-id 7) */

 /* Send the ID value to all the nodes */
 host_to_node_int_1(surface_thread_id); /* Does nothing in serial */

#if RP_NODE
 Message("\nNode %d is calculating on thread # %d\n",myid,
          surface_thread_id);
#endif /* RP_NODE */

#if !RP_HOST  /* SERIAL or NODE */
 /* thread is only used on compute processes */
 thread = Lookup_Thread(domain,surface_thread_id);

 begin_f_loop(face,thread)

/* If this is the node to which face "officially" belongs,*/
/* get the area vector and pressure and increment         */
/* the total area and total force values for this node    */
  if (PRINCIPAL_FACE_P(face,thread)) /* Always TRUE in serial version */
  {
   F_AREA(area,face,thread);
   total_area += NV_MAG(area);
   total_force += NV_MAG(area)*F_P(face,thread);
  }
 end_f_loop(face,thread)

 Message("Total Area Before Summing %f\n",total_area);
 Message("Total Normal Force Before Summing %f\n",total_force);

# if RP_NODE /* Perform node synchronized actions here
                Does nothing in Serial */
  total_area = PRF_GRSUM1(total_area);
  total_force = PRF_GRSUM1(total_force);
# endif /* RP_NODE */

#endif /* !RP_HOST */

/* Pass the node's total area and pressure to the Host for averaging  */
 node_to_host_real_2(total_area,total_force); /* Does nothing in SERIAL */

#if !RP_NODE /* SERIAL or HOST */
Message("Total Area After Summing: %f (m2)\n",total_area);
Message("Total Normal Force After Summing %f (N)\n",total_force);
Message("Average pressure on Surface %d is %f (Pa)\n",
                 surface_thread_id,(total_force/total_area));
#endif /* !RP_NODE */
}

The function begins by initializing the variables surface_thread_id, total_area, and total_force for all processes. This is done because the variables are used by the serial, host, and node processes. The compute nodes use the variables for computation purposes and the host uses them for message-passing and displaying purposes. Next, the preprocessor is directed to compile thread, face, and area variables only on the serial and node versions (and not the host), since faces and threads are only defined in the serial and node versions of ANSYS FLUENT. (Note that in general, the host will ignore these statements since its face and cell data are zero, but it is good programming practice to exclude the host. See Section  7.5 for details on compiler directives.)

Next, a user-defined Scheme variable named pres_av/thread-id is obtained by the host (and serial) process using the RP_Get_Integer utility (see Section  3.6), and is assigned to the variable surface_thread_id. (Note that this user-defined Scheme variable was previously set up in Cortex and assigned a value of 2 by typing the text commands shown in the comments.) After a Scheme-based variable is set up for the thread ID, it can be easily changed to another thread ID from the text interface, without the burden of modifying the source code and recompiling the UDF. Since the host communicates with Cortex and the nodes are not aware of Scheme variables, it is essential to direct the compiler to exclude the nodes from compiling them using #if !RP_NODE. Failure to do this will result in a compile error.

The surface_thread_id is then passed from the host to compute node-0 using the host_to_node macro. Compute node-0, in turn, automatically distributes the variable to the other compute nodes. The serial and node processes are directed to loop over all faces in the thread associated with the surface_thread_id, using #if !RP_HOST, and compute the total area and total force. Since the host does not contain any thread data, it will ignore these statements if you do not direct the compiler, but it is good programming practice to do so. The macro PRINCIPAL_FACE_P is used to ensure that faces at partition boundaries are not counted twice (see Section  7.2). The nodes display the total area and force on the monitors (using the Message utility) before the global summation. PRF_GRSUM1 (Section  7.5.4) is a global summation macro that is used to compute the total area and force of all the compute nodes. These operations are directed for the compute nodes using #if RP_NODE.