Developing a Shader/Texture plug-in for OpenFX


The plug-in shaders come in two forms, one for the Software scanline software renderer and one for the Hardware GPU renderer.  The shaders are defined in the Modeller as part of a material applied to a surface, they are mainly used by the Renderer module,  The GPU renderer requires a vertex and a fragment shader,  the software renderer uses the same DLL as is used to set-up the shader.  (Note we use the terms shader and texture interchangeably, because the original software only version of OpenFX called its plugin surface appearances "Textures" the modern term "Shader" is used for GPU generated surface appearances. Since OpenFX offers both CPU and GPU procedural surface appearance generators the definitions are a little blurred.)  

Storing and using Shader settings

Plug-in textures/shaders (and plug-in effects and  image processors)l have user configurable parameters. The number and type of the configurable parameters necessarily support different numbers of and different types of data.  To be able to store these in the .MFX and .OFX files OpenFX stores these parameters as a text string - each shader etc.plug-in is expected to begin by reading its parameters from the parameter text string (if it exists, or create it from defaults, if it does not exist.) In that part of the plug-in used by the Designer or Animator, before it terminates it is expected to write into the text string its parameters. This can normally be done using the C functions sscanf() and sprintf().  By adopting this strategy OpenFX imposes no constraints on the type or number of shader (etc.) parameters.

Building and Files

Each shader/texture plugin is built as a Windows DLL, (and a pair of GLSL shaders) it will have one or more C or C++ language source file, one or more project specific header files, a Visual Studio project file, a resource file (.RC) ,  a module definition file (.DEF) a file with filename extension .TFX.  The project will build to a .DLL of the same name as the TFX file the pair of GLSL codes will have the same name as the TFX file but with filename extensions .VERT and  .FRAG .  The build files can have any location, but the TFX, .FRAG .VERT and DLL files must be placed in the "TEXTURES" folder.

The plugin source files need to include four header files. Two of the headers define the memory allocation functions used by the current OpenFX, these are located in the "animate" folder. The "defines.h" file defines global variables and renderer functions that can be used by the plugin when it is executing within the renderer. (It is located in the "textures" folder. The final essential header file is "rstruct.h"  (located in the "render2" folder) defines the structure used to pass information from the renderer to the plug-in.) (This is discussed in the IMPORTANT section below).  The Renderer's functions that the developer can use are not defined in a library, they are accessed through a pointer that is passed to the plugin at run time. However, they can be used in the plugin as if they were ordinary functions. There is no need to link any of the plugins with a special library. (We will return to the source code later.) 

The resource file (.RC) will contain a dialog box description to allow the shaders's parameters to be set by the user.  To get a feel for a plugin shader we shall look at a plugin for a simple checkerboard texture. The external textures/shaders are defined as part of a material (up to 4 external textures can be used as part of one material).

The .DEF file defines the exported entry point functions used by the Designer and the Renderer:

_ExternalTextureProcedure @1
_ExternalTextureStartup @2
_ExternalTextureMorph @3
_ExternalTextureClose @4
_SetExternalParameters @5
_ExternalTextureProcedureGL @6

The purpose of these functions will be discussed in the context of the "checker" example. The screen below shows the material dialog and the texture specific dialog for the Checkerboard texture.

shaders.png (237930 bytes)

The files associated with the checker shader are:


these are located in the "textures" folder. The build project  file is located in the "textures" folder and all the current shaders may be built using the Visual Studio SLN file in the same folder.

The build process will create a DLL file called checks.dll in the plugins folder.  The checks.tfx is a dummy file containing nothing, it is used for the external  texture selection box to provide a list of all external textures and can be regarded as a simple place holder.   If you want to start writing a new texture  plugin then we suggest you copy the files from the example plugin to a new set.  It's possible to copy the .VCPROJ files and modify it using notepad  replacing all instances of the string  "checks" with the name of your new texture.

This concludes all we have to say about building. Most of the other plugins in OpenFX follow a similar strategy.

The plugin-code

The plugin code follows the basic design of a Windows DLL . In this section we will examine the parts of the checks source file that are common to all shader plugins. The most important items to be aware of are the names of the functions called by OpenFX's designer and render module.  These are listed in the .DEF file. These functions use pointers to two data structures through which access to the OpenFX main module's (designer/animator/renderer) memory management, and other functions may be made.    

The OpenFX method of accessing global variables and main program functions   (VERY IMPORTANT)

OpenFX uses a pointer mechanism for allowing the plugins to access functions in the main program and program global variables. Since all OpenFX plugins are DLLs that are loaded into the main applications address space. The plugin DLLs are NOT loaded when the main application starts (many of the built-in actions are also implemented as DLLs that ARE loaded when the program starts) they are loaded when required and unloaded when they have finished.
In the case of plugins that are used in the Designer and Animator AND are needed by the Renderer (e.g. shaders) then the Renderer loads all the needed plugins when it reads the script file during initialisation. The Renderer unloads the plugins when it is finished with the script.

When the Designer requires to configure a texture it calls  the function _SetExternalParameters(), with three arguments:

char *_SetExternalParameterss(char *Op, ,HWND hWnd,X__MEMORY_MANAGER *lpevi);

typedef struct tagX__MEMORY_MANAGER {
void *(*fpMalloc)(long size);
void (*fpFree)(void *buffer);
void *lpAni;
void *lpMod;

The void structure members "lpAni" and "lpMod"   are used to pass pointers to the internal structures ANI_STRUCTURE(Animator)   and X__STRUCTURE(Designer) when appropriate!  (When not used these pointers are set to NULL.)

The Designer, Animator and Renderer  keep only one copy of this structure, it is initialised at start-up and passed to all the plugins.  By using the members of this structure any plugin can access the memory functions used internally.   This method is very simple and very flexible for extension and allowing plug-ins to access global data and functions in the calling programs.

In order that the contents of structures like this can be made available to all functions and files in a plugin the pointers passed in to the functions should normally  be copied into the global variable:

The function that actually carries out the generation of the effect is called:  _ExternalTextureProcedure(..) and the one that generates the effect using the GPU is called  _ExternalTextureProcedureGL(..).  The last argument of the calls to both these functions is a pointer to a Renderer data structure called   X__SHADER.  Like the memory structure and the Designer's X__STRUCTURE the X__SHADER structure contains pointers to important data structures in the Designer and useful internal functions like the Peril noise funcitons.

Part of the X__SHADER structure looks like this:

typedef struct tagX__SHADER {
void (*fp_sNoise)(double x, double y, double z, double *result);
void (*fp_sTurbulence)(double x, double y, double z, double *result);
vector *fp_Wave_Sources;
Glint (*fp_GetAttibuteLocation)(nit id, char *name);
void (*fp_DrawShaderPolygons)(long k, nit pass, Glint prodigy, 
     glint attribloc, long Face, void *maintop, long Invert, void *maintop); 
long *dummy[16];
long version;
vector *fp_axis_o,*fp_axis_u,*fp_axis_v,*fp_axis_n;
long fp_Nface;
void *fp_MainFp;
long *dummy[21];

Accessing variables and functions in a DLL module through direct use of a pointer the X__SHADER or X__MEMORY_MANAGER  is very tiresome.  For example to use the Designer's number of selected  vertices variable (called "NvertSelect")  it would be necessary to use the code (  (*(lpevi->NvertSelect))    )  to  gain access to this variable.   This is very messy!  To overcome this issue and provide an interface to the functions that make them look (in the code) just like normal function calls, a number of #defines are included in the header file "defines.h" , which is one of the ".h" files that every plugin must include.

Using these defines, calls to functions and use of global variables can be used as if they were part of the module code.

Keeping this very important information in mind we can return to considering the Checker texture plugin and look at the key code constructs.

Essential headers, defines and global variables
include <stdlib.h>
#include <windows.h>

// handle to resource file to allow resources to be loaded
static HINSTANCE hDLLinstance=NULL;  

#include "..\animate\memory.h" /* for memory definition */
#include "..\animate\memdef.h" /* local names */
#include "defines.h"
#include "rstruct.h"
#include "checks.h"  // identifiers for the dialog box
DLL standard entry code for Visual Studio compiler
BOOL WINAPI DllMain(HANDLE hDLL, DWORD dwReason, LPVOID lpReserved){
switch (dwReason) {
hDLLinstance = hDLL; /* handle to DLL file */
return (nit)TRUE;
Function called by the Designer to allow the user to set the shader's parameters,
char * _SetExternalParameters(char *Op, /* string for the parameters */
     HWND hWnd, /* parent window */
     X__MEMORY_MANAGER *Lpevi /* pointer to structure with memory functions */
// Create a dialog box to set the effects
// use the incoming parameters Op if not NULL
// Free Op, create new string to hold the parameters and return pointer to new
// parameter string
return Op;
Exported function called by Renderer when the shader  is first loaded.
long _ExternalTextureStartup(long frame,long nframes,
    char *parameter_list, 
     X__SHADER *Lpevi){
if(parameter_list[0] != '#'){ /* NO parameters use default */}
else {
sscanf(parameter_list,  /* read the parameters for this texture */, ...);
// load the shaders for this effect (standard renderer code)
return LoadAndCompileShader("checks");
Exported function called by Renderer when the shader is unloaded.)
void _ExternalTextureClose(X__SHADER *Lpevi){
Exported function called by Renderer to morph any of the texture's parameters that are time dependendent.
long _ExternalTextureMorph( char *parameter_list, double mr){
// parameter list contains the same parameters as the main list but
// from the previous key frame
// "mr" is the morth ration when mr=0 then each parameter should
// equal the value passed to this function. Whe mr=1 then each 
// parameter should have the value read when the texture is loaded
return 1;
Exported function called by Renderer to implement the texture created by the software renderer.
long _ExternalTextureProcedure(
long coord_type, vector p, vector n,
double alpha, double beta, double gamma,
double bump, double map_x, double map_y,
double *alpha_channel, unsigned char sc[3], double colour[3],
double *reflectivity, double *transparency,
// The parameters above are defined in the Renderer code
// Note in the renderer  "double" is re-defined as a "float" to save
// on memory.
X__SHADER *Lpevi  // Important structure 
// implement the effect
return 1;
Exported function called by Renderer to configure any attribute or uniform varibles used by the shader as part of the GPU hardware renderer.
long _ExternalTextureProcedureGL(
double bump_scale,
unsigned char sc[3],
unsigned char ac[3],
X__SHADER *Lpevi
// Set any shader specific uniform variables here.
SetUniformVector( ...);
// Callback to Render the Polygons - this allows for multiple passes - through same shaders
// possibly with different parameters - including vertex offset - blending depth enabling 
// to allow for multipass textures e.g. hair and fur (shell model) it's arguments must not be altered!!!
return 1;


The Checker texture/shader example

To return to the checker example, the algorithm is very simple: we just use the texture coordinates passed to the shader and modulo arithmetic, per unit of texture coordinates in the X and Y directions used to determine whether the material's base or secondary colour is used. In this section we will  look at a the key steps in the code:

static double x=1.0,y=1.0,z=1.0;
BOOL CALLBACK DlgProc(HWND hwnd,UINT msg,WPARAM wparam,LPARAM lparam); 
char * _SetExternalParameters(
  char *Op, /* string for the parameters */
  HWND hWnd, /* parent window */
  X__MEMORY_MANAGER *Lpevi /* pointer to structure with memory functions */
  char szbuf[255],*Op1;
  if(Op != NULL){ /* parameters exist so read them off the list */
   sscanf(Op1,"%f %f %f",&x,&y,&z);
//  Do a modal dialog
if(DialogBox(hDLLinstance,MAKEINTRESOURCE(DLG),hWnd, (DLGPROC)DlgProc) == FALSE)return Op;
if(Op != NULL)CALL_FREE(Op); /* free the old string */
  sprintf(szbuf,"# %.3f %.3f %.3f", x,y,z);  / write the new scaling parameters into the text string
  if((Op=(char *)CALL_MALLOC(strlen(szbuf)+1)) == NULL)return NULL;
  strcpy(Op,szbuf);  // copy from the local temporary string
  return Op;  // return the string pointer
long _ExternalTextureStartup(long frame,long nframes,char *parameter_list, X__SHADER *Lpevi){
if(parameter_list[0] != '#'){;}
else {
sscanf(parameter_list,"%f %f %f",
return LoadAndCompileShader("checks");    // this renderer function loads the shaders from files checks.vert and checks.frag
void _ExternalTextureClose(X__SHADER *Lpevi){
UnloadCompiledShader(tGLshaderID);    // unload the GLSL shaders
long _ExternalTextureMorph(char *parameter_list, double mr){
double x_m,y_m,z_m;
sscanf(parameter_list,"%f %f %f",&x_m,&y_m,&z_m);
x=x_m+(x-x_m)*mr;   // morph the shader parameters
return 1;
long _ExternalTextureProcedure(
    long coord_type, vector p, vector n,
   double alpha, double beta, double gamma,
   double bump, double map_x, double map_y,
   double *alpha_channel, unsigned char sc[3], double colour[3],
    double *reflectivity, double *transparency,
   X__SHADER *Lpevi
long id;
double xx,yy,zz,noise;
double xr,xl,yr,yl,zr,zl;
xx=alpha / x;    // converte the texture to units of the texture cell
yy=beta / y;
zz=gamma / z;
id = (long)FLOOR(xx) + (long)FLOOR(yy) + (long)FLOOR(zz);
if(id & 1){   // if in this cell then switch to the material's secondary colout
return 1;
long _ExternalTextureProcedureGL(
double bump_scale, 
unsigned char sc[3],
unsigned char ac[3], 
X__SHADER *Lpevi
// Set any shader specific uniform variables here.  (thies are the colour and the scaling aplied to the texture cell.
// Callback to Render the Polygons - this allows for multiple passes - through same shaders
// possibly with different parameters - including vertex offset - blending depth enabling 
// to allow for multipass textures e.g. hair and fur (shell model) it's arguments must not be altered!!! 
return 1;


// vertex shader for plugin textures 
const vec4 fc = vec4(0.0,0.0,1.0,1.0); 
varying vec3 tnorm;
varying vec4 PP;
varying vec3 texpos; 
varying vec3 uvec;
varying vec3 vvec;
varying vec3 wvec; 
uniform vec3 Uvector; 
uniform vec3 Vvector; 
uniform vec3 Wvector; 
attribute vec3 ShaderPosition; // vertex shader coordinate 

void main(void){
// copy the vertex position
// compute the transformed normal
tnorm = gl_NormalMatrix * gl_Normal ;
gl_Position = ftransform(); 
PP = gl_ModelViewMatrix * gl_Vertex; 
texpos = ShaderPosition;
vvec = normalize(gl_NormalMatrix * Vvector);
wvec = normalize(gl_NormalMatrix * Wvector);
uvec = normalize(gl_NormalMatrix * Uvector);
gl_FrontColor = gl_Color; 
gl_BackColor = gl_Color; 


// fragment shader for plugin textures 
// uniforms - note they must be used if they are to be accessed in the calling program 
uniform nit Clip1;
uniform vec3 MaterialC; // Material second colour
uniform vec3 ScalingV;
uniform float SpecularPower; // specularity (range 10 - 80 )
uniform float SpecularContribution; // mix of specular contribution - turns on/off spec 0.0/1.0
uniform float MixRatio; // Reflection Fraction 
uniform vec3 AntiNormal1; // Transform matrix for normal to undo camera
uniform vec3 AntiNormal2; // Connection to a mat3 uniform 
uniform vec3 AntiNormal3; 
uniform sampler2D RefMap; // Reflection sampler 
const float DiffuseContribution=1.0;
const vec3 Xunitvec = vec3 (1.0, 0.0, 0.0);
const vec3 Yunitvec = vec3 (0.0, 1.0, 0.0); 
varying vec3 tnorm; // interpolated surface normal
varying vec4 PP; // vertex global coord
varying vec3 texpos; // shader coord relative to material shader axis 
varying vec3 wvec;
varying vec3 uvec;
varying vec3 vvec; 
void CalcLight(in vec3, in vec3, in vec3, inout vec4, inout vec4); // Light function (External) 
void main(void){
if(Clip1 == 1){ // Use this as alternative to use the buid in clipping to make ATI/nVidia Compatible
float d=dot(,gl_ClipPlane[0].xyz) + gl_ClipPlane[0].w;
if( d < 0.0)discard;
// compute the fragment position in eye coordinates
vec3 ecPosition = vec3(PP);
// compute a unit vector in direction of viewing position
vec3 viewVec = normalize(-ecPosition);
// must re-normalize interpolated vector to get Phong shading
vec3 vnorm = normalize(tnorm);
vec3 reflectDir = reflect(ecPosition,vnorm);
mat3 AntiNormal; 
vec2 index;
index.y = dot(normalize(reflectDir), Yunitvec);
reflectDir.y = 0.0;
index.x = dot(normalize(reflectDir), Xunitvec) * 0.5;
if (reflectDir.z >= 0.0)index = (index + 1.0) * 0.5;
else {
index.t = (index.t + 1.0) * 0.5;
index.s = (-index.s) * 0.5 + 1.0;
vec3 envColor = vec3 (texture2D(RefMap, index)); 
vec4 surface_colour=gl_Color; 
vec3 p=floor(texpos);
nit id = nit(p.x)+nit(p.y)+nit(p.z);
if(((id/2)*2) != id){

vec4 diffuse=vec4(0.0,0.0,0.0,1.0);
vec4 spec=vec4(0.0,0.0,0.0,0.0);

gl_FragColor = mix(surface_colour * DiffuseContribution * diffuse, vec4(envColor,1.0),MixRatio)
+ spec * SpecularContribution;


This completes our discussion of the "Checker"  plug-in shader/texture as part of the material.

Go back to the developer page...