Practical Software Development: TEFEL 2: Go-Style Interfaces in C

Welcome to Duncan White's Practical Software Development (PSD) Pages.

I'm Duncan White, an experienced and professional programmer, and have been programming for well over 30 years, mainly in C and Perl, although I know many other languages. In that time, despite my best intentions:-), I just can't help learning a thing or two about the practical matters of designing, programming, testing, debugging, running projects etc. Back in 2007, I thought I'd start writing an occasional series of articles, book reviews, more general thoughts etc, all focussing on software development without all the guff.

See all my Practical Software Development (PSD) Pages

TEFEL #2: Another Example of a TEFEL tool: Go-style Interfaces in C
My previous article (no 11) discussed a new tactic that can be used when you want to add a single well-defined feature to a large existing language like C. I call this new technique Tool-Enhanced Features for Existing Languages (TEFEL). The idea is simple:

Work out what new feature you want.

Propose an exact syntax for it.

Work out exactly how to translate the new syntax into Plain C.

Graft the new feature into C by writing a simple line-by-line pre-processor that copies most lines through unchanged (hoping they're valid C), but locates specially marked Extension Directives, turning each into a corresponding chunk of plain C.

Thus, C with directives will come in, and standard C will go out.

Now, while explaining this idea on the Plain Ordinary C Programming forum on LinkedIn, I asked for other ideas for features that could be added to C using a TEFEL pre-processor. Nigel Evans and I knocked some ideas around, and came up with the following: As I said back in PSD Article 8: An excellent general principle is to transfer cool features from one language into another - and I've done this several times in this series:

Article 8 itself went on to describe my tool DataDec, which transposes the Functional Programming recursive data types concept to C.

Similarly, of course, article 11 built a client-side extension to C to make programming with DataDec even easier and more pleasant.)

So, it will not surprise you to find that my new example here is another feature that we can take from some other language, and add to C. But what language, and what feature? Google's Go language (see http://golang.org/), was partially designed by such C luminaries as Brian Kernighan and Rob Pike, and Rob Pike gave an excellent Keynote speech about the practical design ideas here .

Go has several rather controversial, but entirely deliberate, design decisions embodied in it. One such design decision is Go's approach to Object Oriented Programming, favouring composition via interfaces over inheritance from parental classes. As long time readers of these articles will know, I am not a fully paid-up member of the Object Oriented Programming (OOP) squad. In particular, I have come to heartily dislike inheritance. So, obviously, Go's approach to OOP is almost designed to attract me!

Go handles OO via three concepts: packages, interfaces and receivers. In Go, a package is, roughly speaking, a module, ie. a package contains a set of public functions, data types (all optional private additional functions to help). Next, a Go interface simply defines a set of named functions (and the parameter and return type signature that each takes). If a particular package implements all those functions (with the same signatures), then that package is compatible with the interface (of course, a single package may be compatible with any number of interfaces).
You may then write code that takes an object of a particular interface type, and uses that value to call those common functions that the interface guarantees exist. Each such call is routed automatically to the particular package's function. This acts just like dynamic method dispatch in full blown OO, which is of course a form of polymorphism.
Note that interfaces don't tackle the object data part of OO, ie. self and attributes, Go handles that via functions having an optional receiver type, often a pointer to a structure which stores the instance attributes, and allowing you to write receiverobjectvariable.function(params). I'm going to completely ignore receivers in this article, and concentrate on making post-hoc interfaces work.
(Actually, in rereading this discussion alongside the Go Programming book, I see some of the above comments are approximations, based on my lack of in-depth familiarity with Go. In particular, I realise that in Go an interface is a type within a package, and they are actually matched with other types (not packages). So what I've said is a simplification, but this will do for the purposes of this article).

Go interfaces are, then, vaguely like Java interfaces, or C++ abstract classes, except that you define Go interfaces independently of (and often after writing) the concrete "packages" that happen to implement them. In his Go keynote speech Rob Pike argues very strongly that this time reversal is very important - that working this way allows interfaces to be added later, once common patterns of packages and their functions emerge, rather than having to plan the interfaces and inheritance hierarchies long before it is clear what you will need - and, having decided upon a new interface, having to intrusively modify each "class" that implements this interface (which leads to fragile inheritance hierarchies and interfaces).

So, with a Go-style interface you can "bind" a package to an interface, which either fails if the package is not compatible with that interface (i.e. the package does not contain at least of the required interface functions), or succeeds, giving you an interface value. That interface value may be passed around as a parameter, and calls "through it" to any of the interface functions can be made at any later time, but of course each such call is actually made to the underlying package's function of the same name.

Last year, when I thought about this fresh from reading the Go Programming Language book, I couldn't think of a way in portable C of implementing such interfaces at all. The basic question we need to get an answer to is does module M contain a function called M_F? (ideally, with the correct type signature).

Thinking about this in late July 2018, I realised that Unix's shared libraries and dynamic linking library (aka libdl, dlopen() and dlsym()) could give us a way to implement the basic idea of Go-style interfaces: The key insight is that if module M is dynamically linked, then dlsym() can answer the above question for us!
Of course, this still means that we don't have a completely portable method, as Unix's libdl is not available on (for example) Windows, let alone small embedded systems. But Unix still covers a lot of ground (Linux, Solaris, HP-UX etc).

Anyway, having had the idea of implementing something like Go-style interfaces in C, as a TEFEL tool, the place to start is to work out what libdl allows us to do, and work out what sort of code we might write manually to implement some form of interface. This is the first step before building a TEFEL tool - working out the style of code we might want to generate.
After all, we can't build a tool to generate code before knowing what sort of code we may wish to generate! So, let's start by thinking about Unix shared libraries, libdl and what we can usefully do with them. Don't worry, we'll get back to interfaces in a little while!

Introduction to Unix Shared Libraries
Suppose we start by writing a plain C file called pkg1.c, containing a bunch of functions of varying types:
 /*
  *	pkg1: a collection of C functions, that may accidentally satisfy
  *	      one or more interfaces that don't yet exist..
  */
 
 #include <stdio.h>
 #include <stdlib.h>

 #include "pkg1.h"

 void pkg1_f1( void )
 {
	printf( "pkg1::f1\n" );
 }

 int pkg1_f2( void )
 {
	printf( "pkg1::f2, returning 1\n" );
	return 1;
 }

 void pkg1_f3( char *s, int x )
 {
	printf( "pkg1::f3, s='%s', x=%d\n", s, x );
 }

 void *pkg1_f4( int n )
 {
	printf( "pkg1::f4, x=%d, returning NULL\n", n );
	return NULL;
 }
Then we generate the matching modular header file pkg1.h containing the public prototypes, either manually or using my proto tool:
  extern void pkg1_f1( void );
  extern int pkg1_f2( void );
  extern void pkg1_f3( char * s, int x );
  extern void * pkg1_f4( int n );
Then we write a simple program usepkg1.c that imports pkg1 and uses those functions:
  #include <stdio.h>
  #include <stdlib.h>

  #include "pkg1.h"

  int main( void )
  {
	pkg1_f1();
	int n = pkg1_f2();
	printf( "pkg1::f2 returned %d\n", n );
	pkg1_f3( "hello", n );
	void *p = pkg1_f4( n );

	return 0;
  }
So far, this is all perfectly normal C, that you have no doubt written many times before, and (on Unix systems) we might expect to compile each .c file to a .o, and then link them together into an executable, by gcc commands such as:
  gcc -Wall -c pkg1.c
  gcc -Wall -c usepkg1.c
  gcc -o usepkg1 usepkg1.o pkg1.o
Instead, let's build pkg1 as a shared library. First, we compile all our C code using gcc's -fPIC (position independent code flag):
  gcc -Wall -fPIC -c pkg1.c
  gcc -Wall -fPIC -c usepkg1.c
and then we create a shared library libpkg1.so from pkg1.o:
  gcc -shared -Wl,-soname,libpkg1.so -o libpkg1.so pkg1.o
then finally we link usepkg1.o against -lpkg1, ie. our newly created libpkg1.so. But of course with shared libraries this really records the linkage, because most of the work really happens later, at run-time:
  gcc -o usepkg1 usepkg1.o -L. -lpkg1
Of course, really, we wouldn't type those complex gcc invocations direct at the command line each time. We'd set them up as a Makefile:
  CC		=	gcc
  CFLAGS	=	-Wall -fPIC
  BUILD		=	libpkg1.so usepkg1

  all:	$(BUILD)

  clean:
	/bin/rm -f $(BUILD) *.o lib*so core a.out
  
  usepkg1:	usepkg1.o libpkg1.so
	$(CC) -o usepkg1 usepkg1.o -L. -lpkg1

  libpkg1.so:	pkg1.o
	$(CC) -shared -Wl,-soname,libpkg1.so -o libpkg1.so pkg1.o
Now, compile everything up via make:
  make
Next, I tried running the executable:
  ./usepkg1 
When I did this, I saw the confusing error message:
  ./usepkg1: error while loading shared libraries: libpkg1.so: cannot open shared object file: No such file or directory
To see why, I used the following command
  ldd usepkg1
and saw that libpkg1.so reported as missing:
  libpkg1.so => not found
To fix this, we have to tell the runtime linker to search the current directory for shared libraries: Bash/sh users write:
  export LD_LIBRARY_PATH=".:$LD_LIBRARY_PATH"
Whereas csh users write:
  setenv LD_LIBRARY_PATH ".:$LD_LIBRARY_PATH"
To check, use "ldd" again:
  ldd usepkg1
and you should now see something like:
  libpkg1.so => ./libpkg1.so (0x00007fd5566d6000)
Now you can run usepkg1 as normal:
  ./usepkg1
and you'll see:
  pkg1::f1
  pkg1::f2, returning 1
  pkg1::f2 returned 1
  pkg1::f3, s='hello', x=1
  pkg1::f4, x=1, returning NULL
If you want to follow along with this, you can download a tarball 01pkg1a.tgz containing this code so far.
Indirect access via libdl
Next, we want to add a layer of indirection - we don't want to link usepkg1 with the shared library at compile-time. Instead, we want to leave that completely to run-time: we want to use libdl to open the shared library, then use dlsym() to find out whether the shared library contains the functions that we want to call, and return function pointers to them, and then call via those function pointers.
Ok, so now leaving pkg1.c completely unchanged, we construct a new program: usepkg1_via_dl.c as follows: First we include libdl's header file, dlfcn.h:
  /*
   *	usepkg1_via_dl: let's try to use pkg1 via libdl
   */

  #include <stdio.h>
  #include <stdlib.h>
  #include <dlfcn.h>
Then, in main(), we call dlopen() to open a connection via libdl to our particular named shared library:
  int main( void )
  {
	char *module = "pkg1";

	char libname[1024];
	sprintf( libname, "lib%s.so", module );

	// open the shared library
	void *dl = dlopen( libname, RTLD_NOW );
	if( dl == NULL )
	{
		fprintf( stderr, "Can't dlopen %s\n", libname );
		return 1;
	}
  }
(Don't worry about what the constant RTLD_NOW is, read the manual page ("man dlopen") for more details, but for our purposes, it's just part of the furniture of how we call dlopen()).
Now that we have a non-NULL pointer dl, we can use it to lookup a named public symbol in the shared library. Of course a symbol x could be an externally visible variable called x (of any type), or an externally visible function called x() (with any return type, and any number of parameters of any combination of types). We'll want to check these details rather better later on, when we develop full-blown interfaces, but for now let's just assume a symbol we find is the sort of symbol that we need.
So we can write:
	char *funcname = "f1";

	// can we look up the symbol funcname inside the dl?
	void *p = dlsym( dl, funcname );
	if( p != NULL )
	{
		printf( "Have found %s in %s\n", funcname, module );
	} else
	{
		printf( "Not found %s in %s\n", funcname, module );
	}
Let's wrap this up into a reusable function lookup_function().
  //
  // void *p = lookup_function( void *dl, char *module, char *funcname );
  //	look within dl, a dynamic library opened by dlopen(), for
  //	the global symbol funcname.
  //	If we find it, return a pointer to it.  If not, return NULL.
  //
  void *lookup_function( void *dl, char *module, char *funcname )
  {
	// can we look up the symbol funcname inside the dl?
	void *p = dlsym( dl, funcname );
	if( p != NULL )
	{
		printf( "Have found %s in %s\n", funcname, module );
		return p;
	}

	printf( "Not found %s in %s\n", funcname, module );
	return NULL;
}
Back in main(), we can now call lookup_function() for f1. and die unless we find it.
	void *f1 = lookup_function( dl, module, "f1" );
	if( f1 == NULL )
	{
		fprintf( stderr, "Failed to find f1 or %s_f1 in %s\n",
			module, libname );
		return 1;
	}
If we have found it (and assuming that it's a symbol of the right type, which we said earlier we can't yet check), we can now define a pointer to function type for any void->void function:
  // void_void_f: a pointer to a void->void function
  typedef void (*void_void_f)( void );
Modify the void *f1 to read void_void_f f1, and now we should be able to call it:
	// ok, now we have f1, a pointer to pkg1's f1 function.
	// let's just call it:
	(*f1)();
Note that modern C compilers also allow you to write the simpler:
	f1();
but personally I think that this syntax, which obscures the fact that f1 is a pointer to a function, is just... wrong. It's a pointer, dammit, so you should use * when derefencing it.
To recap, our whole program now reads:
  /*
   *	usepkg1_via_dl: let's try to use pkg1 via libdl
   */

  #include <stdio.h>
  #include <stdlib.h>
  #include <dlfcn.h>


  //
  // void *p = lookup_function( void *dl, char *module, char *funcname );
  //	look within dl, a dynamic library opened by dlopen(), for
  //	the global symbol <funcname>.
  //	If we find it, return a pointer to it. If not, return NULL.
  //
  void *lookup_function( void *dl, char *module, char *funcname )
  {
	// can we look up the symbol funcname inside the dl?
	void *p = dlsym( dl, funcname );
	if( p != NULL )
	{
		printf( "Have found %s in %s\n", funcname, module );
		return p;
	}

	printf( "Not found %s in %s\n", funcname, module );
	return NULL;
  }

  // void_void_f: a pointer to a void->void function
  typedef void (*void_void_f)( void );

  int main( void )
  {
	char *module = "pkg1";

	char libname[1024];
	sprintf( libname, "lib%s.so", module );

	// open the shared library
	void *dl = dlopen( libname, RTLD_NOW );
	if( dl == NULL )
	{
		fprintf( stderr, "Can't dlopen %s\n", libname );
		return 1;
	}

	void_void_f f1 = lookup_function( dl, module, "f1" );
	if( f1 == NULL )
	{
		fprintf( stderr, "Failed to find f1 or %s_f1 in %s\n",
			module, libname );
		return 1;
	}

	// ok, now we have f1, a pointer to pkg1's f1 function.
	// let's just call it:
	(*f1)();

	return 0;
  }
You can download this version via: the tarball 02dl1.tgz, containing the code so far. It should compile fine on any Linux system, and hopefully also on any POSIX-compliant Unix system:
  make
If you then examine the output of ldd:
  ldd usepkg1_via_dl
You will no longer see a reference to libpkg1.so, but now you will see a new reference to libdl:
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f3711517000)
If you now run this, will it work? Let's see:
  ./usepkg1_via_dl
(Don't forget you need "." on your LD_LIBRARY_PATH as discussed earlier). Unfortunately, it says:
  Not found f1 in pkg1
  Failed to find f1 or pkg1_f1 in libpkg1.so
Why didn't it work? If you inspect pkg1.c you may notice the solution immediately. But suppose you don't. A useful diagnostic tool to track this sort of problem down is nm, which displays all defined symbols (and external symbol used) in an object file or shared library:
  nm libpkg1.so|grep f1
reveals the answer:
  0000000000000760 T pkg1_f1
Our function inside pkg1.c is not called f1, it's called pkg1_f1. No wonder we couldn't find it!
The best way of fixing this is to extend lookup_function() so that it searches for both funcname and modulename_funcname, stopping as soon as it finds one:
  //
  // void *p = lookup_function( void *dl, char *module, char *funcname );
  //	look within dl, a dynamic library opened by dlopen(), for
  //	either the global symbol <funcname> or the global symbol
  //	<module_funcname>.
  //	If we find either, return a pointer to the first one we found.
  //	If not, return NULL.
  //
  void *lookup_function( void *dl, char *module, char *funcname )
  {
	// can we look up the symbol funcname inside the dl?
	void *p = dlsym( dl, funcname );
	if( p != NULL )
	{
		return p;
	}

	//printf( "Not found %s in %s\n", funcname, module );

	// can we look up the module-qualified symbol inside the dl?
	char fullname[1024];
	sprintf( fullname, "%s_%s", module, funcname );
	p = dlsym( dl, fullname );
	if( p != NULL )
	{
		return p;
	}
	//printf( "Not found %s in %s either\n", fullname, module );
	return NULL;
  }
(I've also removed some of the printf()s in the function, and commented out others).
You'll find this version ready for download in the tarball 03dl2.tgz, Using that version, compile it and run it:
  make
  ./usepkg1_via_dl
and you should see:
  pkg1::f1
which is the message printed by pkg1_f1 in pkg1.c. If you're not convinced, feel free to edit pkg1.c and change the message that is printed. Recompile (noticing that only pkg1.c is recompiled and libpkg1.so regenerated), and rerun.
Next, let's try looking up f2 and calling that as well, if we find it. Here, we must remember that pkg2's f2 function (actually called pkg1_f2 of course) is a void->int function. Define another pointer to function type:
  // int_void_f: a pointer to a void->int function
  typedef int (*int_void_f)( void );
and then add our f2 stanza:
	// Let's do the same with function f2 now.. it's void->int

	int_void_f f2 = lookup_function( dl, module, "f2" );
	if( f2 == NULL )
	{
		fprintf( stderr, "Failed to find f2 or %s_f2 in %s\n",
			module, libname );
		return 1;
	}

	// ok, now we have f2, a pointer to pkg1's f2 or pkg1_f2 function.
	// let's just call it:
	int n = (*f2)();
	printf( "Called pkg1's f2 function, result is %d\n", n );
You'll find this version ready for download in the tarball 04dl3.tgz, Using that version, compile it and run it:
  make
  ./usepkg1_via_dl
and you should see:
  pkg1::f1
  pkg1::f2, returning 1
  Called pkg1's f2 function, result is 1
(unless of course you changed the pkg1::f1 message earlier:-)). If you look at pkg1_f2() in pkg1.c, you will see that indeed it does return 1. Feel free to change that return value and recompile.
One thing you might also want to try is editing pkg1.c and renaming pkg1_f2() as f2(). After all, lookup_function() contains code to lookup either function, so it should still work. Alternatively, you might like to alter pkg1.c so that it defines both f2 and pkg1_f2, printing different messages and returning different values, and satisfy yourself which function lookup_function() will find and invoke.
Next, let's build a new package which provides compatible f1() and f2() functions - create pkg2.c which reads as follows:
  /*
   *	pkg2: a second collection of C functions, that may accidentally satisfy
   *	      one or more interfaces that don't yet exist..
   */

  #include <stdio.h>
  #include <stdlib.h>

  void f1( void )
  {
	printf( "pkg2::f1\n" );
  }

  int f2( void )
  {
	int n = 42;
	printf( "pkg2::f2, returning %d\n", n );
	return n;
  }

  void f3( char *s, int x )
  {
	...
  }

  void *f4( int n )
  {
	...
  }
Note that this version happens to define the functions as fN() rather than pkg2_fN(), but of course lookup_function() contains code to deal with both cases, so that shouldn't matter. Next, we add a new test program usepkg2_via_dl.c which differs only from usepkg1_via_dl.c in the value of module at the top of main():
	char *module = "pkg2";
and we also need new Makefile rules to compile everything up.
You'll find this version ready for download in the tarball 05dl4.tgz, Using that version, compile it and run our new test program:
  make
  ./usepkg2_via_dl
and you should see:
  pkg2::f1
  pkg2::f2, returning 42
  Called pkg2's f2 function, result is 42
(Feel free to modify various parts of the f1() and f2() bodies in pkg2.c to change the messages printed, and the return value, if you want to further convince yourself that it's really calling the right functions in pkg2.c).
There seems very little point in having two versions of a test program that differ only in the name of the module under test. Let's take the name of the module from the command line instead of fixing it inside main(). While we're at it, let's allow multiple command line arguments, each specifying a different module name to test.
So we construct use_any_pkg.c, whose main() now reads:
  int main( int argc, char **argv )
  {
	for( int i=1; argv[i] != NULL; i++ )
	{
		f1_f2( argv[i] );
	}
	return 0;
  }
Calling a new function f1_f2( modulename ) to do all the libdl work for that module. This function comprises most of the rest of the old main() body, generalised slightly to print a message and continue rather than abort:
  //
  // f1_f2( char *module );
  //	load the given module via libdl, and try to
  //	find functions f1 and f2 in that module,
  //	invoking both of them if you find them.
  //
  void f1_f2( char *module )
  {
	char libname[1024];
	sprintf( libname, "lib%s.so", module );

	// open the shared library
	void *dl = dlopen( libname, RTLD_NOW );
	if( dl == NULL )
	{
		fprintf( stderr, "Can't dlopen %s\n", libname );
		return;
	}

	// Can we find (void->void) function f1 in module?
	void_void_f f1 = lookup_function( dl, module, "f1" );
	if( f1 == NULL )
	{
		fprintf( stderr, "Failed to find f1 or %s_f1 in %s\n",
			module, libname );
		return;
	}

	// Can we find (void->int) function f2 in module..
	int_void_f f2 = lookup_function( dl, module, "f2" );
	if( f2 == NULL )
	{
		fprintf( stderr, "Failed to find f2 or %s_f2 in %s\n",
			module, libname );
		return;
	}

	// ok, now we have f1, a pointer to the module's f1 or
	// module_f1 function. let's just call it:
	printf( "Calling %s's f1 function\n", module );
	(*f1)();

	// ditto, now we have f2, let's just call it:
	int n = (*f2)();
	printf( "Called %s's f2 function, result is %d\n", module, n );
  }
You'll find this version ready for download in the tarball 06dl5.tgz. Using that version, compile it and run our new test program with pkg1 as the argument:
  make
  ./use_any_pkg pkg1
and you should see:
  Calling pkg1's f1 function
  pkg1::f1
  pkg1::f2, returning 1
  Called pkg1's f2 function, result is 1
Then run it with pkg2 as the argument:
  ./use_any_pkg pkg2
and you should see:
  Calling pkg2's f1 function
  pkg2::f1
  pkg2::f2, returning 42
  Called pkg2's f2 function, result is 42
Note that the old test programs usepkg[12]_via_dl.c have been removed.
If you think about this for a moment, this should prove that we are well on the way to some form of plugin architecture via libdl now, because we can name any module (or sequence of modules) on the command line, and our program will attempt to find the required functions - f1() (or module_f1()), and f2() (or module_f2()) - in each dynamic library named. In fact, version 06dl5 also contains pkg3.c, which does not contain a f2() or pkg3_f2() function, so is not compatible. Run:
  ./use_any_pkg pkg3
and you should see:
  Failed to find f2 or pkg3_f2 in libpkg3.so
If you give an entirely non-existent module name:
  ./use_any_pkg pkg10
you should see:
  Can't dlopen libpkg10.so
A prototype "f12.interface"..
Suppose we now return to our fullblown idea of interfaces. You could construct an interface in hypothetical TEFEL form like: f12.interface:
  %func void f1( void );
  %func int f2( void );
this describes what functions, with what type signatures (number of parameters, parameter types, and return type), you'd need to have exist in your "interface" which I've imaginatively called f12. Here, any module which implements f1 and f2, ideally with matching type signatures (to be checked later), is compatible with interface f12, and can be bound to interface f12.
You could translate this (by hand) to a plain C module, f12.[ch], that implements the critical f12_bind() function which tries to bind a package to the required interface. This contains most of the code from our earlier f1_f2() function - except that instead of calling the functions at the end, it stores the function pointers in fields of an interface structure:
f12.h reads:
  // void_void_f: a pointer to a void->void function
  typedef void (*f12_void_void_f)( void );

  // int_void_f: a pointer to a void->int function
  typedef int (*f12_int_void_f)( void );

  // This represents the "interface f12" at run-time.
  // It's a container of SLOTS for the f12 functions..
  typedef struct
  {
	f12_void_void_f   f1;
	f12_int_void_f    f2;
  } f12;

  /*
   * f12 *x = f12_bind( char *modulename, char *errmsg );
   *	Attempt to "bind" lib<module>.so to the f12 interface:
   *	Load "lib<module>.so" into memory, and attempt to locate the
   *	required symbols f1 and f2 (or <module>_f1 and <module>_f2...)
   *	within it's namespace.  For now, we just check for existence
   *	of those function symbols, later on we'll try to check the
   *	compatibility of the function signatures with the interface.
   *
   *	If we fail: strcpy an error message into errmsg and return NULL
   *	If we succeed: return an newly malloc()d f12 object with the
   *	slot function pointers bound to the corresponding functions in
   *	lib<module>.so (now in memory)
   */
  extern f12 *f12_bind( char * modulename, char * errmsg );
You'll notice that we've added an f12_ prefix to our various function pointer types, to avoid any potential namespace clashes, and created the f12 structure type containing one function pointer per required function, to represent a module bound to the f12 interface at runtime.
Next, to implement the f12_bind() function, we will reuse most of our code. Let's start by moving lookup_function() into a separate module, as it is a helper that can be used when binding any interface: to reduce the number of parameters to lookup_function(), and move the responsibility for constructing an error message inside, lookup.h defines a struct type pkg_info which combines the information about a package - the module name, the library name, the interface name, the dl pointer and an error message pointer:
  // pkg_info: information record about what we're binding to/from..
  typedef struct
  {
	void *dl;      		// result of dlopen on libname
	char *module;		// name of the module we're binding to
	char *interface;	// name of this interface
	char *errmsg;		// pointer to space for error message
	char *libname;		// name of the library we're binding to
  } pkg_info;

  //
  // void *p = lookup_function( pkg_info *info, char *funcname );
  //	look within info->dl, a dynamic library opened by dlopen(), for
  //	either the global symbol <funcname> or the global symbol
  //	<modulename_funcname>.
  //	If we find either, return a pointer to the first one we found.
  //	If not, return NULL.
  //
  extern void *lookup_function( pkg_info *info, char *funcname );
lookup.c then reads:
  /*
   *	lookup: helper to search for a function symbol in a dl
   */

  #include <stdio.h>
  #include <stdlib.h>
  #include <dlfcn.h>

  #include "lookup.h"

  //
  // void *p = lookup_function( pkg_info *info, char *funcname );
  //	look within info->dl, a dynamic library opened by dlopen(), for
  //	either the global symbol <funcname> or the global symbol
  //	<modulename_funcname>.
  //	If we find either, return a pointer to the first one we found.
  //	If not, return NULL.
  //
  void *lookup_function( pkg_info *info, char *funcname )
  {
	// can we look up the symbol funcname inside the dl?
	void *p = dlsym( info->dl, funcname );
	if( p != NULL )
	{
		return p;
	}

	// can we look up the module-qualified symbol inside the dl?
	char fullname[1024];
	sprintf( fullname, "%s_%s", info->module, funcname );
	p = dlsym( info->dl, fullname );
	if( p != NULL )
	{
		return p;
	}

	sprintf( info->errmsg,
		"No symbol '%s' or '%s' in %s",
		funcname, fullname, info->libname );

	return NULL;
  }
f12.c then reads:
  /*
   *	interface f12: a collection of C functions.
   *	manually translated into plain C, using libdl
   */

  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <assert.h>
  #include <dlfcn.h>

  #include "f12.h"
  #include "lookup.h"

  /*
   * f12 *in = f12_bind( char *module, char *errmsg );
   *	Attempt to "bind" lib<module>.so to the f12 interface:
   *	Load "lib<module>.so" into memory, and attempt to locate the
   *	required symbols f1 and f2 (or <module>_f1 and <module>_f2...)
   *	within it's namespace.  For now, we just check for existence
   *	of those function symbols, later on we'll try to check the
   *	compatibility of the function signatures with the interface.
   *
   *	If we fail: strcpy an error message into errmsg and return NULL
   *	If we succeed: return an newly malloc()d f12 object with the
   *	slot function pointers bound to the corresponding functions in
   *	lib<module>.so (now in memory)
   */
  f12 *f12_bind( char *module, char *errmsg )
  {
	char libname[1024];
	sprintf( libname, "lib%s.so", module );
	void *dl = dlopen( libname, RTLD_NOW );
	if( dl == NULL )
	{
		sprintf( errmsg, "f12_bind: dlopen of %s failed", libname );
		return NULL;
	}

	f12 *in = malloc(sizeof(*in));
	if( in == NULL )
	{
		strcpy( errmsg, "f12_bind: malloc() failed" );
		return NULL;
	}

	pkg_info   info;
	info.dl     	= dl;
	info.module 	= module;
	info.libname	= libname;
	info.errmsg 	= errmsg;

	in->f1 = (f12_void_void_f) lookup_function( &info, "f1" );
	if( in->f1 == NULL )
	{
		free(in);
		return NULL;
	}

	in->f2 = (f12_int_void_f) lookup_function( &info, "f2" );
	if( in->f2 == NULL )
	{
		free(in);
		return NULL;
	}

	return in;
}
Client code (such as our new f12_any_pkg.c program) can attempt to bind a named module to the interface "f12", check it works, and then invoke the bound functions, via:
  /*
   *	f12_any_pkg: let's try to access a package via interface "f12"
   */

  #include <stdio.h>
  #include <stdlib.h>

  #include "f12.h"

  int main( int argc, char **argv )
  {
	for( int i=1; argv[i] != NULL; i++ )
	{
		char *module = argv[i];
		char errmsg[1024];
		f12 *f = f12_bind( module, errmsg );
		if( f == NULL )
		{
			fprintf( stderr, "%s\n", errmsg );
		} else
		{
			f->f1();
			int n = f->f2();
			printf( "f2 returned %d\n", n );
			free( f );
		}

		printf( "\n" );
	}
	return 0;
  }
You'll find this version ready for download in the tarball 07int1.tgz. Using that version, compile it and run our new test program with pkg1 as the argument:
  make
  ./f12_any_pkg pkg1
and you should see:
  pkg1::f1
  pkg1::f2, returning 1
  f2 returned 1
Then run it with pkg2 as the argument:
  ./f12_any_pkg pkg2
and you should see:
  pkg2::f1
  pkg2::f2, returning 42
  f2 returned 42
Building a TEFEL tool to create f12.[ch]

Now that we know what C code we would like to generate from the f12.interface file, the next step is to carry on and build the TEFEL tool that does that.

Let's carry on and do that in part 2 of this article.

d.white@imperial.ac.uk
Back to PSD Top
Written: August 2019