BPT: The BeanShell Preprocessor Tutorial

Warren MacEvoy


Table of Contents

Introduction
1. A (yawn) preprocessor?
2. Installation
3. Caio Mondo!
4. BPP Basics
5. BPP Intermediates
6. BPP Patterns
7. Symmetric Preprocessing in BPP
A. Running BPP (part 2)
B. Docbook

List of Tables

4.1. BPP to BeanShell translation
7.1. Multi-stage BPP translation

List of Examples

2.1. hello.bpp: Hello, World!
2.2. hello: Hello, World!
3.1. HelloWorld.java.bpp: Hello Ciao привет
3.2. dictionary.xml: translation table
3.3. HelloWorld.java.bpp: Ciao
4.1. HelloWorld.java: traditional java hello world
4.2. helloworld.bsh: BeanShell hello world
4.3. BeanShell: "Any" variable types
4.4. BeanShell: static import and iterators example program
4.5. BeanShell: static import and iterators example output
4.6. dictionary.bsh: language translation setup file
4.7. HelloWorld.java.bpp: better translation version
5.1. C header file: bad example 1
5.2. C headerfile: bad example 2
5.3. C header file: correct
6.1. Pattern: Version control
6.2. Pattern: Template Definition
6.3. Pattern: Template Instantiation
6.4. Pattern: Tempalate Instantiation (multiple redirects)
6.5. Pattern: Document/View XML Root Document
6.6. Pattern: Document/View HTML Renderer
6.7. Pattern: Document/View Java Renderer
7.1. Useless ### example
7.2. Document/View Renederer without ##-preprocessor
7.3. Document/View Renederer with ##-preprocessor
A.1. BPP in a batch file

Introduction

The BeanShell Preprocessor, or BPP for short, is the first of a type of preprocessor known as a symmetric preprocessor. It allows Java™ programmers to use all of Java as a preprocessor for either Java source files or to produce other text (possibly UTF-8 encoded unicode) documents.

The fact that you can use a full-featured language such as Java as a preprocessor greatly strengthens the expressiveness available in the preprocessor. The same idea could be applied to other programming languages; but so far only Perl has a similar preprocessing language (see PerlPP: CPP on Steriods in the Linux Gazette). The real value of using the preprocessor is to leverage skills: if you already know Java, then, in the short time it takes to read and understand this tutoral, you will know how to use Java to write Java (or XML, SQL and COBOL for that matter).

Chapter 1. A (yawn) preprocessor?

Yes, BPP is a preprocessor. BPP supports (and expects) UTF-8 encoded unicode, but its real goal in life is producing source files. Its intention is to take marked up (mostly ASCII) text documents and produce similar (but somehow better) text documents, which are also probably also mostly ASCII.

Didn't preprocessors die out a long time ago? Yes and no. If preprocessor makes you think of M4 and CPP, then there hasn't been much innovation in those kind of preprocessors for some time. If you think of JSP, PHP and XSLT, you might realize that automated preprocessing is alive and kicking, but changed its name to keep away from the Unfashionable Lingo police.

BPP can be thought of as an extraordinarily superior version of the C preprocessor for the Java (or any other) programming language. I know that sounds like an unsupportable claim, but you can be the judge after reading this tutorial.

Chapter 2. Installation

BPP uses Java, BeanShell, and some of the examples in this tutorial use JDOM. You can get the latest JDK from Sun at java.sun.com. BeanShell is a Java interpreter available from beanshell.org, and JDOM is a Java XML document object model available from www.jdom.org. The versions available at the time of writing this tutorial are: JDK SE 1.4.2_03, BeanShell 2.0b1, and JDOM Beta 10. BPP is available from SourceForge at bpp.sourceforge.net. The jar filese for BeanShell, JDOM, and BPP are in the bin folder of the full BPP distribution.

If you are using a version of BPP after 0.8.5.2b, then the easiest way to run bpp is:

  • Put bpp.jar somewhere handy, in the same folder, place all the jar files of stuff you want BPP to use, especially BeanShell's jar file, since BPP can't run without it.
  • If there are class files you would like BPP to use, then the class my.package.MyClass should be placed in
    path/to/bpp/my/package/MyClass.class
    
  • Once these are set,
    java -jar /path/to/bpp.jar args...
    
    Should work fine.

Note that, when jar files are executed with the -jar option, the CLASSPATH is ignored. This is just the rules for jar files. If you want to use a classpath, or have a version of BPP older than 0.8.5.2b, then read on...

If you want the CLASSPATH to be used, then your only option is to include the path to BPP and BeanShell in your CLASSPATH, and then execute BPP with

java -cp BPP_CLASSPATH bpp.BPP args...

One way to assure these are in your classpath, is to place them in the jre/lib/ext directory of the JRE installation you are using. Be warned that there may be more than one of these on your system. For example, a default installation of the Java SE SDK under windows creates two folders: c:/j2sdk1.X.Y_ZZ/jre/lib/ext and c:/Program Files/Java/j2re1.X.Y_ZZ/lib/ext. Commonly used jar files should be placed in both locations to avoid any suprizes about missing class files.

While copying the jar files into the jre/lib/ext directory is the most convenient, you will probably want to explicitly use a classpath argument if you are working on a multi-user system, or if you have concerns about version control of jar files during your development. See Appendix A about other ways to run BPP.

If Java is set up properly, then typing java -version at the console should produce version information. See the documentation on Sun's website (or the thousands of other resources on the net) if you are having trouble getting Java to work.

Once Java is set up, you can check to see if beanshell is working properly by typing,

java -jar /path/to/bsh-2.0b1.jar

To check that beanshell sees JDOM, type

who="World"; 
print("Hello, "+who);

in the BeanShell console. BeanShell should respond with no errors. BPP uses beanshell indirectly, so you can close the console for now.

Finally, you can check if BPP is properly set up. Create a file called hello.bpp containing:

Example 2.1. hello.bpp: Hello, World!

#name="World";
#greet="Hello";
$greet, $name!

Type

java -jar /path/to/bpp.jar hello.bpp

If this works correctly, then you have just created a file named hello containing:

Example 2.2. hello: Hello, World!

Hello, World!

Contratulations! We are now ready to explore the power and expressiveness of BPP.[1]



Chapter 3. Caio Mondo!

Suppose you wanted to deliver an application in various specific languages. The product you deliver to your customer is a simple small application with information in their language, without the bulk and burden of carrying information about all languages. Consider this modified "hello world" program, presumably written in HelloWorld.java.bpp:

Example 3.1. HelloWorld.java.bpp: Hello Ciao привет

# //
# // get some useful Java formatting tools:
# //
# static import bpp.Format.*;
#
# //
# // use the lang property to get the current language
# // default is english (en)
# //
# String lang=System.getProperty("lang");
# if (lang==null) lang="en";
#
# //
# // read in our dictionary using JDOM
# //
# org.jdom.Element dictionary = 
#      new org.jdom.input.SAXBuilder().build(
#         new BufferedInputStream(
#             new FileInputStream("dictionary.xml")))
#                .getRootElement();
#
# //
# // stick all the words in a hash map
# //
# Map words = new HashMap();
# for (i:dictionary.getChildren()) { 
#    words.put(i.getAttributeValue("name"),i); 
# }
#
#
# // shorthand function for looking up a word
# // according to its name and encoding it
# // as a Java string literal
# String word(String name) { 
#    return literal(words.get(name)
#       .getAttributeValue(lang)); 
# }
#
public class HelloWorld
{
  public static final String LANGUAGE="$lang";
  public static void main(String [] args) 
  {
    System.out.println($(word("helloworld")));
  }
}

Now create the dictionary.xml file with the following contents:

Example 3.2. dictionary.xml: translation table

<?xml version="1.0" encoding="UTF-8"?>
<dictionary>
  <word name="helloworld" 
    en="Hello World!" 
    it="Ciao Mondo!" 
    ru="&#x043F;&#x0440;&#x0438;&#x0432;&#x0435;&#x0442; &#x043C;&#x0438;&#x0440;"
  />
</dictionary>

You may now produce an English, Italian, and Russian version of HelloWorld with:

java -jar /path/to/bpp.jar -Dlang=lang HelloWorld.java.bpp

Note that each time you run BPP, it will overwrite HelloWorld.java with a new version. You can redirect the ouput of BPP with the -o option if you want to. For example:

java  -jar /path/to/bpp.jar -Dlang=it -o it/HelloWorld.java HelloWorld.java.bpp

Let's look at the Java source file generated as the Italian translation:

Example 3.3. HelloWorld.java.bpp: Ciao

public class HelloWorld
{
  public static final String LANGUAGE="it";
  public static void main(String [] args)
  {
    System.out.println("Ciao Mondo");
  }
}

There are several important points here:

  • First, the code does not suffer from any bloat: it can be built and distributed independently of JDOM, XML, BeanShell or BPP. Even though it takes advantage of all three in the build process.
  • Second, there is no run-time overhead for the language support. Most solutions to this ship all the languages and looks up specific entries at run time. All the table creation and lookups happen at build time with BPP.
  • Third, if a competitor got all of the versions of the HelloWorld source files (or reverse engineered the class files), they would still have no particular insight as to how you support three languages in your application. A typical distribution would allow a competitor to reverse engineer your IP and deliver an application in a similarly economic way.
  • Fourth, there is absolutely no possibility of doing this using a simple preprocessor like CPP. At least, last time I checked I could not read in an XML document and stuff it in a hash table in CPP.
  • Fifth, the preprocessor code is just Java (well, BeanShell) code with a # in front of it. That means all the tools and tricks you have built up for Java you may now use to write Java.

These are all significant payoffs. Now aren't you glad that you read up to here?

For being the first example, I know this was fairly complex, but you probably wanted to see up-front why you should spend any more time learning BPP. From here on out, we will start from the beginning and work our way up.

Chapter 4. BPP Basics

Seeing something happen isn't as easy as making it happen (and you haven't seen everything anyway), so lets start with some simple exercises to get the basic ideas. We will move on to some power tools later on.

BeanShell

I mentioned already that BPP uses beanshell under the hood. If you are familar with Java it should take you a very short time to become familar with BeanShell, and there are some excellent tutorials on the official beanshell website. However, here are a few samples good enough to get through this tutorial on BPP.

BeanShell example 1: Hello World, version 3

Writing a console application in Java takes a bit of typing:

Example 4.1. HelloWorld.java: traditional java hello world

public class Main {
  public static void main(String [] args) {
    System.out.println("hello world!");
  }
}

Typing this in BeanShell will work, but here's BeanShell's shorthand version:

Example 4.2. helloworld.bsh: BeanShell hello world

print("hello world!");

You can run this by typing the above into the beanshell console (the thing we started by typing java -cp ... bsh.Console), or directly interpret it withn

java -cp /path/to/bsh-2.0b1.jar bsh.Interpreter hello.bsh

In short, BeanShell is a get-to-the-point Java. It allows you to use native Java in an interpreted environment, and is written in Java. In the example above, print("hello world") in BeanShell is equivalent to System.out.println("hello world"), and there are a number of other useful shorthand notations as well, but if you write Java statements beanshell will interpret them the same way Java does.

BeanShell example 2: Types

One important simplification BeanShell supports is the idea of an "any" type. "Any" types are simply used without a declaration, and take on whatever value is assigned to them. For example, running

Example 4.3. BeanShell: "Any" variable types

x=1;
print("x="+x);
x="two";
print("x="+x);
x=new Date();
print("x="+x);

will produce

x=1
x=two
x=Tue Feb 17 12:53:30 MST 2004

"Wait!" You say. What about type safety? Java saves my cookies all the time by remembering what is what. You can declare x and give it a type and BeanShell will check its type during assignments, but you don't have to. You can also set strict java modes if you are paranoid. Don't be too paranoid in the preprocessor; any bad code generated by the preprocessor must also pass the plain old (strict) Java compiler.

BeanShell example 3: JDK 1.5 extensions

There are two convenient syntaxes which will be supported in the JDK 1.5 that beanshell already supports. The first is the idea of a static import, and the second is an iterator over a collection notation. These two extensions are illustrated in the following:

Example 4.4. BeanShell: static import and iterators example program

// get static methods and values from the java.Math class
static import java.lang.Math.*;

x=cos(PI/6); // JDK1.5 extension
y=Math.sin(Math.PI/6); // the old painful way.

print("(x,y)=(" + x + "," + y + ")"); 

// Make a list of things:
list = new LinkedList();
list.add("thing one");
list.add("thing two");

// print out the things:
for(item:list) print("new: "+ item); // JDK1.5 extension

// print out the things the old way
for (Iterator i=list.iterator(); i.hasNext(); ) {
  item=i.next();
  print("old: "+item);
}

Running this example through beanshell produces the output

Example 4.5. BeanShell: static import and iterators example output

(x,y)=(0.8660254037844387,0.49999999999999994)
new: thing one
new: thing two
old: thing one
old: thing two

Most people aren't paid more to type more, and the new syntax is easier to read, so why not use it?

BeanShell example 4: Downcasts and exceptions.

I slipped something by you on the last example. I assign a LinkedList to an "any" type variable called list. Later on, I ask for list.iterator(). Generally speaking, I might get a ClassCastException if I had, say, made list an array instead. There are several aspects of loosening up strict java syntax here:

  • BeanShell uses reflection to invoke methods and inspect or change attributes. This means that, if you know a list contains only StringBuffer elements, you can append an "x" to each of them with:
    for(item:list) item.append("x");
    
    
    Instead of the standard Java:
    for(Iterator i=list.iterator(); i.hasNext(); ) {
      StringBuffer item = (StringBuffer)i.next();
      item.append("x");
    }
    
    
    In the beanshell specific example, you will get an exception if the list contains an object with no appropriate append() method. This is an example of automatic down-casting, which isn't automatic in plain Java.
  • BeanShell allows you to invoke a method without necessarily catching any exceptions that might be thrown. Again, you can write try .. catch blocks, but if you don't, and no exceptions are thrown, the script will execute fine. For example, working with files usually requires that you account for IOException exceptions ocurring, but in beanshell you can write a script that ignores those possibilities. Such a script will generate a RuntimeException if an exception happens.

BeanShell is an extremely useful tool independently of BPP, and you will need to be modestly comfortable with it before understanding and using BPP. I suggest you spend a little time working through some tutorials on BeanShell. It is sure to get into your development process somewhere.

A simplified reduction of BPP.

Let me begin by stating that I am lying to you. BPP does more than what you will read in this section, but it is a convenient jumping-off point for understanding BPP.

Let's begin by inspecting what BPP does to transform the HelloWorld.java.bpp BPP source file into the HelloWorld.java java source file. You can see this by running BPP with the -b option; which means "generate a beanshell script, but don't execute it." On the left is the origional source file (HelloWorld.java.bpp) and on the right is the generated beanshell script (HelloWorld.java.bsh):

Table 4.1. BPP to BeanShell translation

HelloWorld.java.bppHelloWorld.java.bsh
# //
# // get some useful Java formatting tools:
# //
# static import bpp.Format.*;
#
# //
# // use the lang property to get the current language
# // default is english (en)
# //
# String lang=System.getProperty("lang");
# if (lang==null) lang="en";
#
# //
# // read in our dictionary using JDOM
# //
# org.jdom.Element dictionary = 
#      new org.jdom.input.SAXBuilder().build(
#         new BufferedInputStream(
#             new FileInputStream("dictionary.xml")))
#                .getRootElement();
#
# //
# // stick all the words in a hash map
# //
# Map words = new HashMap();
# for (i:dictionary.getChildren()) { 
#    words.put(i.getAttributeValue("name"),i); 
# }
#
#
# // shorthand function for looking up a word
# // according to its name and encoding it
# // as a Java string literal
# String word(String name) { 
#    return literal(words.get(name)
#       .getAttributeValue(lang)); 
# }
#
public class HelloWorld
{
  public static final String LANGUAGE="$lang";
  public static void main(String [] args) 
  {
    System.out.println($(word("helloworld")));
  }
}

 //
 // get some useful Java formatting tools:
 //
 static import bpp.Format.*;

 //
 // use the lang property to get the current language
 // default is english (en)
 //
 String lang=System.getProperty("lang");
 if (lang==null) lang="en";

 //
 // read in our dictionary using JDOM
 //
 org.jdom.Element dictionary = 
      new org.jdom.input.SAXBuilder().build(
         new BufferedInputStream(
             new FileInputStream("dictionary.xml")))
                .getRootElement();

 //
 // stick all the words in a hash map
 //
 Map words = new HashMap();
 for (i:dictionary.getChildren()) { 
    words.put(i.getAttributeValue("name"),i); 
 }


 // shorthand function for looking up a word
 // according to its name and encoding it
 // as a Java string literal
 String word(String name) { 
    return literal(words.get(name)
       .getAttributeValue(lang)); 
 }

out.println("public class HelloWorld");
out.println("{");
{out.print("  public static final String LANGUAGE=\"");out.print(lang);out.println("\";");}
out.println("  public static void main(String [] args) ");
out.println("  {");
out.println("    System.out.println("+(word("helloworld"))+");");
out.println("  }");
out.println("}");

At the first level,

  • All lines which begin with a # are copied to the beanshell script verbatim (except for the # itself).
  • Other normal lines are translated into out.print() statements, which, if executed, reproduce the origional line of text. BPP automatically defines out as a UTF-8 encoded OutputStream to the output file.
  • $JAVA_IDENTIFIER and $(JAVA_EXPRESSION) are translated in a magical way so that values in the beanshell script can be easily used to generate text output.

Executing the script on the right will produce the various versions of the HelloWorld class given the proper lang system property. However, it is easier to maintain the BPP code on the left, because it knows how to encode odd characters like the double quote without you worrying about it.

Without the -b option, BPP generates and executes the beanshell script on the right (without ever explicitly creating the file HelloWorld.java.bsh). For a number of substantial problems, this is enough to use BPP effectively.

Knowing this allows us to simplify the maintenence of several source files all of which have to allow for various language translations. Since reading the dictionary is a common feature of such an application, it would be smarter to have a single beanshell source file with the dictionary code in it, called dictionary.bsh. This is just the top part of the HelloWorld.java.bpp file with the #'s removed:

Example 4.6. dictionary.bsh: language translation setup file

//
// get some useful Java formatting tools:
//
static import bpp.Format.*;

//
// use the lang property to get the current language
// default is english (en)
//
String lang=System.getProperty("lang");
if (lang==null) lang="en";

//
// read in our dictionary using JDOM
//
org.jdom.Element dictionary = 
     new org.jdom.input.SAXBuilder().build(
        new BufferedInputStream(
            new FileInputStream("dictionary.xml")))
               .getRootElement();

//
// stick all the words in a hash map
//
Map words = new HashMap();
for (i:dictionary.getChildren()) { 
   words.put(i.getAttributeValue("name"),i); 
}


// shorthand function for looking up a word
// according to its name and encoding it
// as a Java string literal
String word(String name) { 
   return literal(words.get(name)
      .getAttributeValue(lang)); 
}

With this handy file, the typical translated source file would look like:

Example 4.7. HelloWorld.java.bpp: better translation version

#
# source("dictionary.bsh");
#
public class HelloWorld2
{
  public static final String LANGUAGE="$lang";
  public static void main(String [] args) 
  {
    System.out.println($(word("helloworld")));
  }
}

If you are new to BeanShell, think that source() in beanshell is equivalent to #include in the C preprocessor. Now it looks almost like plain Java, and we have seperated any details about our dictionary support to one script file and one dictionary. A production-level solution in a few dozen lines of code.

A few parting words before I let the translate-at-preprocess-time idea go. First, for a large dictionary, a better solution would be to use a database to store the translations. In our model that only implies changing the contents of the dictionary.bsh file to reflect using JDBC to access the new flavor of dictionary. If you are used to the power of the C preprocessor, you probably see how BPP broadens your vision what a preprocessor can do for you.

In the next Chapter, we'll talk about generating other kinds of source files (like C programs) using BPP. This will bring a few more details about BPP syntax for non-default translation rules.

Chapter 5. BPP Intermediates

Suppose you wanted certain "unusual" numerical constants to be defined in a C program, but defined in such a way that these constants were computed at compile time. The natural place to put these in a C program is in a header file. As an example, suppose we wanted something along the lines of the following header file MyConstants.h:

Example 5.1. C header file: bad example 1

#ifndef MY_CONSTANTS_H
#define MY_CONSTANTS_H

#define X 12.0
#define CUBE_ROOT_X   2.2894284851066637356160844238794
#define SINE_X       -0.536572918004349716653742282424

#endif

The problem with this technique is pretty obvious: there is no way to inspect the correctness of this header file. In fact it is wrong; the correct value to SINE_X is -0.5365729180004349716653742282424 (three zeros instead of two). This could easily happen with an accidental press of the delete key and would create a very subtle error. How about this instead (now defined in MyConstants.h.bpp, and not yet quite correct):

Example 5.2. C headerfile: bad example 2

#static import java.lang.Math.*;
#static import bpp.Format.*;
#ifndef MY_CONSTANTS_H
#define MY_CONSTANTS_H

#x=12.0

#define X $(literal(x))
#define CUBE_ROOT_X   $(literal(pow(x,1.0/3.0)))
#define SINE_X        $(literal(sin(x)))

#endif

How much more confident would you be about the correctness of the results of this header file? If it worked that is. This source file suffers from a conflict of interest: are the # symbols in column 1 indicating something for the C preprocessor (lines 3 and 4), or BPP (lines 1 and 2)?

BPP has some syntax to clear this up. Here are some more detailed translation rules (yes, I'm still leaving some things out):

  • Lines with a # (pound) in column 1 have the remainder of the line copied exactly to the BeanShell script. For example,
    #static import bpp.Format.*;
    
    
    will be translated into the beanshell code:
    static import bpp.Format.*;
    
    
    This is the problem rule for the example. The C preprocessor likes to interpret #-signs too, which compells us to put #-signs in column 1 for other reasons. How do we tell BPP to leave these lines alone? There are two explicit translation rules to force BPP to make other choices.
  • Lines with a " (double quote) in column 1 are forced to translate the remainder of the line magically, which means to convert it into a print statement, accounting for $JAVA_IDENTIFIER and $(JAVA_EXPRESSION). For example,
    "#define X $x
    "  String LANGUAGE="$lang";
    "  System.out.println($(word("helloworld")));
    
    
    will be translated into the beanshell code:
    {out.print("#define X ");out.println(x);}
    {out.print("  String LANGUAGE=\"");out.print(lang);out.println("\";");}
    {out.print("  System.out.println(");out.print(word("helloworld"));out.println(");");}
    
    
    Executing the first line of this in the context that x=12.0 would produce the C preprocessor-style definition
    #define X 12
    
    
    This is the reason we started down this road. By putting a double quote in column 1, you can force magic translation on lines which would appear by the default rules to take some other translation. While this solves the force, it is worth mentioning the prohibit syntax as well.
  • There are reasonable situations where you do not want to have a magic translation, even on lines that have lots of $'s in them. A good example of this would be to quote a part of PHP or Perl script, which use a $-sign in front of all (PHP) or many (Perl) variables. The way to force BPP to quote exactly is to place a single quote in column 1. Doing this causes BPP to encode that line as a print statement that will exactly reproduce the line (except for the leading single quote) exactly. For example,
    '#define X $x
    '  String LANGUAGE="$lang";
    '  System.out.println($(word("helloworld")));
    
    
    will be translated into the beanshell code:
    out.println("#define X $x");
    out.println("  String LANGUAGE=\"$lang\";");
    out.println("  System.out.println($(word(\"helloworld\")));");
    
    

What does BPP do with a line that has not translate directive in column 1? It applies the default. If BPP is run normally, then the default is to quote magially (like a double quote " in column 1). If, however, you pass the -q option to BPP, it will quote exactly by default.

So now we can fix the C header file (now MyHeader.h.bpp):

Example 5.3. C header file: correct

#//
#// for bpp to execute at preprocess time...
#//
#static import java.lang.Math.*;
#static import bpp.Format.*;
#//
#// for bpp to translate exactly (except the leading single quote)...
#//
'#ifndef MY_CONSTANTS_H
'#define MY_CONSTANTS_H
#//
#// for bpp to execute at preprocess time...
#//
#x=12.0
#//
#// for bpp to translate magically (except the leading double quote)
#//
"#define X $(literal(x))
"#define CUBE_ROOT_X   $(literal(pow(x,1.0/3.0)))
"#define SINE_X        $(literal(sin(x)))
#//
#// for bpp to translate exactly (except the leading double quote)
#//
'#endif

Now the world makes sense again. There is one value to adjust (x), and the preprocessor calculates the rest of the parameters in a way that is guaranteed to be consistent and easy to check the correctness of. For example deleting any character on the SINE_X line will cause a compile-time error, eithe because BPP fails to execute or it produces code that does not make sense.

There is one more translation rule you will eventually need to know about. What if you want to generate a line that uses preprocessor values (and so is magic), and yet has literal $ values which you want to have BPP leave alone? There are two ways around this:

First, on lines that are translated magically, two adjacent dollar-signs ($$) always means a non-magic $. So a section of perl script processed by BPP might have:

#x=12.0;
$$x=$x;

The first line is an assignment in BPP. The second line becomes the equivalent perl assignment statement:

#x=12.0;
$x=12.0;

Second, if you find yourself working too hard to use the BPP syntax, you can always just write your own out.print() statement:

#out.println("$x="+x+";");

Would generate the perl statement $x=12 just as well as $$x=$x would.

You now have seen a lot about BPP. You have seen how it works by translating the BPP source file in a BeanShell script, and executes that to produce an output file. You also know the small number of translation rules it has about how to translate a source line into a beanshell script line. There is really only one more thing to know: the preprocessor has a preprocessor (with a preprocessor, etc.).

I'm actually going to talk about that two chapters from here. For now it is more important to see how to use this syntax to do a lot of useful things. So the next chapter is a preprocessor patterns recipe book.

Chapter 6. BPP Patterns

This chapter is a "patterns of use" section. We're not saying you can't use BPP in other ways, but here are some places BPP has been useful.

Version Control

As much as Java tries to be a "build once, run anywhere" environment, there are still plenty of reasons to want to do different things under different circumstances. A good example of this is the transition between JDK 1.1 and JDK 1.2. It might be convenient to use Java 2 collections for your problem, and some of your customers would benefit from better performance (or perhaps adaptability) by writing code that uses the framework. However, out of backwards compatibility concerns, you may be limited to a Vector and Hashtable.

C programmers face this in spades, since there are potentially important variations on fundamental information (like the size of an int) accross platforms. The solution they came up with, and you can use with BPP in Java, is conditional compilation. The basic pattern of use is:

Example 6.1. Pattern: Version control

#
#String JAVA_VERSION=System.getProperty("java.version");
#boolean JAVA2=(version.compareTo("1.2")>=0);
#

interface Juggler {
#if (JAVA2) {
  /** JDK 1.2 or later. */
  void juggle(Collection balls);
#} else {
  /** JDK 1.1 or later. */
  void juggle(Vector balls);
#}
} // Juggler

Templates

Templates represents the idea of writing code that writes code. This is easy to set up in BPP: first write the template, second translate the template into BeanShell code, and third use it.

Write the template. Suppose we wanted mutable versions of the primative wrapper types. For consistency with the String and StringBuffer relation, we want each to have a XXXXBuffer class that has a toXXXX() method which returns the standard (immutable) wrapper class, along with a toXXXXPrimitive(). For times that we don't want to know which specific type is behind the mutable wrappers, we also want generically named bean-like get and set methods. Like this:

Example 6.2. Pattern: Template Definition

#makeWrapperBuffer(String primitive) {
# String Wrapper=bpp.Format.getWrapper(primitive);
public class $(Wrapper)Buffer {
  protected $primitive value;

  public $(Wrapper)Buffer($Wrapper _value) { setValue(_value); }
  public $(Wrapper)Buffer($(Wrapper)Buffer _value) { value=_value.get(); }
  public $(Wrapper)Buffer($primitive _value) { value=_value; }

  public $Wrapper to$Wrapper() { return new $Wrapper(value); }
  public $primitive to$WrapperPrimitive() { return value; }
  public $Wrapper getValue() { return new $Wrapper(value); }
  public $primitive get() { return value; }

  public void setValue($Wrapper _value) { value=_value.$(primitive)Value(); }
  public void set($primitive value) { value=_value; }
}
#}

Running this through BPP with the -b option creates a beanshell script template. I would save the above file as makeWrapperBuffer.bpt and execute BPP with

bpp -b -o makeWrapperBuffer.bsh makeWrapperBuffer.bpt

The -b option tells BPP to generate the beanshell script, but not to execute it.

From here, there are two options. The simplest is to create files like IntegerBuffer.java.bpp with the code:

Example 6.3. Pattern: Template Instantiation

#source("makeWrapperBuffer.bsh");
package mutable;

#makeWrapperBuffer("int");

So how many of these do I need to make? Did I forget to make one for Byte or Float? BPP can automate this as well with the handy RedirectBegin()..RedirectEnd() beanshell functions in the contrib/bsh directory of the full distribution. Here's the way I would have really done it:

Example 6.4. Pattern: Tempalate Instantiation (multiple redirects)

#source("makeWrapperBuffer.bsh");
#source("redirect.bsh");
#
#for (primitive:bpp.Format.PRIMITIVES.values) {
#  Wrapper = bpp.Format.getWrapper(primitive);
#  WrapperBuffer = Wrapper + "Buffer";
#  src="src/mutable/" + WrapperBuffer + ".java";
#  RedirectBegin(src);
package mutable;

#makeWrapperBuffer(primitive);
#  RedirectEnd(src);
#}

Mutable wrapper types for all the primitive types in three easy steps: write the template, convert it to beanshell, and use it.

Document/View

This pattern is very handy in software development. The idea is that there is some fundamental information stored in some "Root Document." From this fundamental information, different views are generated with "Renderers."

For software development, the root document may be in a database or XML document, and the renderers can be BPP scripts that write code according to whatever is in the root document.

As an example, suppose you wanted to view a form from both a Java application and a web page. The root document in this case could be a description of the form and the data behind it, and the views would generate HTML or Java code depending on the renderer. Let's place our data in an XML root document called MyForm.xml:

Example 6.5. Pattern: Document/View XML Root Document

<?xml version="1.0" encoding="UTF-8"?>
<form name="MyForm">
  <element type="label"    name="nameLabel"   value="Name?"/>
  <element type="textbox"  name="nameTextbox" value=""/>
</form>

The MyForm.html.bpp file might contain something like:

Example 6.6. Pattern: Document/View HTML Renderer

# //
# // read in our form using JDOM
# //
# org.jdom.Element form = 
#      new org.jdom.input.SAXBuilder().build(
#         new BufferedInputStream(
#             new FileInputStream("MyForm.xml")))
#                .getRootElement();
#
# String v(org.jdom.Element element, String attribute) {
#   return element.getAttributeValue(attribute);
# }
<html>
<form name="$(v(form,"name"))">
#for(e:form.getChildren("element")) {
#  String type=v(e,"type");
#  String name=v(e,"name");
#  String value=v(e,"value");
#  if (type.equals("label")) {
     <label id="$name">$value</label>
#  } else if (type.equals("textbox")) {
     <input id="$name" type="textbox" name="$name" value="$value"/>
#  } else {
     <br/><font color="red">Unknown element type: $type</font>
#  }
#} // for each element
</form>
</html>

The equivalent Java-source renderer might be called Form.java.bpp:

Example 6.7. Pattern: Document/View Java Renderer

# //
# // read in our form using JDOM
# //
# org.jdom.Element form = 
#      new org.jdom.input.SAXBuilder().build(
#         new BufferedInputStream(
#             new FileInputStream("MyForm.xml")))
#                .getRootElement();
#
# String v(org.jdom.Element element, String attribute) {
#   return element.getAttributeValue(attribute);
# }
#
# static import bpp.Format.*;

import java.awt.*;
import javax.swing.*;

public class $(v(form,"name")) extends JFrame {

// declare components
//
#for(e:form.getChildren("element")) {
#  String type=v(e,"type");
#  String name=v(e,"name");
#  String value=v(e,"value");
# if (type.equals("label")) {
   JLabel $name = new JLabel($(literal(value)));
# } else if (type.equals("textbox")) {
   JTextField $name = new JTextField($(literal(value)));
# } else { // throw error at preprocess time...
#   throw new RuntimeException("Unknown element type: $type");
# }
#} // for each element
#

  // add them to the content pane:
  public void init() {
    Container cp = getContentPane();
#for(e:form.getChildren("element")) {
#   String name=v(e,"name");
    cp.add($name);
#} // for each element
  } // init()
}

Think about this. I know the above renderers are toys compared to a real renderer for either case, but BPP gives a systematic way to keep several views up to date simultaneously. Even a single renderer will pay for itself through reuse, but that's back to the Template idea.

Chapter 7. Symmetric Preprocessing in BPP

If preprocessing is useful, it is because it enhances the expressiveness of the language that it preprocesses. In the case of BPP, it enhances Java with Java (well, BeanShell). But if Java is better with a preprocessor, shouldn't the preprocessor be better with one too?

In fact, BPP is a symmetric preprocessor, meaning that it does have a preprocessor (which is also BeanShell), with a preprocesser, etc. The following source file illustrates (in an unuseful way) using the preprocessor up to three levels:

Example 7.1. Useless ### example

a=0;
#b=1;
##c=2;
###d=3;
##e=c+$d;
#f=b+$e;
g=a+$f;

The actual steps BPP takes to preprocess this is a little complex, but the results are reasonably easy to understand, and boils down to this: magic $-information at the k-#-level comes from the k+1-# level. This can be seen as processing the highest #-level first, and passing that to the next lower level, etc.

In fact, the above script can be dissected by stopping the filtering process at the various stages (there are 6 for this example. Seeing the output from a given stage of the filtering process is done by passing the -s stage option to BPP.

###-decorator. Passing -s 6 to BPP while it processes the above example generates only the output of the ###-decorator filter, which generates the following[]

Table 7.1. Multi-stage BPP translation

InputOutput from stage 6: ###-decoratorOutput from stage 5: ###-interpreterOutput from stage 4: ##-decoratorOutput from stage 3: ##-interpreterOutput from stage 2: #-decoratorOutput from stage 1: #-interpreter
a=0;
#b=1;
##c=2;
###d=3;
##e=c+$d;
#f=b+$e;
g=a+$f;
out.println("a=0;");
out.println("#b=1;");
out.println("##c=2;");
d=3;
{out.print("##e=c+");out.print(d);out.println(";");}
out.println("#f=b+$e;");
out.println("g=a+$f;");
a=0;
#b=1;
##c=2;
##e=c+3;
#f=b+$e;
g=a+$f;
out.println("a=0;");
out.println("#b=1;");
c=2;
e=c+3;
{out.print("#f=b+");out.print(e);out.println(";");}
out.println("g=a+$f;");
a=0;
#b=1;
#f=b+5;
g=a+$f;
out.println("a=0;");
b=1;
f=b+5;
{out.print("g=a+");out.print(f);out.println(";");}
a=0;
g=a+6;

The decorator stages create beanshell scripts which are executed in the interpreter stages. Eventually the first-level interpreter executes to generate the final output[2].

[2]

Just because you've never seen a preprocessor with a preprocessor before doesn't mean they are not useful, but it takes a little brain stretching to see where and why you want to use it. I do think there is one of those 80-20 rules at hand though:

  • 80% of the code that BPP preprocesses is level 0 (no pound signs).
  • 80% of what remains is level 1 (one pound sign).
  • 80% of what remains is level 2 (two pound signs).
  • ...

In fact, we have not yet found a practical application of the ###-level of the preprocessor. But I'm sure that it exists! If you find one please tell me about it, and I will get it in this tutorial.

All the code examples in this book only show the #-level of the preprocessor, but we have so far found two useful applications of the ##-level, and only one of those seems generically useful enough to put in this tutorial.

Uses of ##

When applying the document/view pattern in software development, specific BPP renderers often look like:

#
#FORALL ELEMENTS IN ROOT DOCUMENT
  GENERATE HEADER
#END FORALL
#
#FORALL ELEMENTS IN ROOT DOCUMENT
  GENERATE BODY
#END FORALL
#
#FORALL ELEMENTS IN ROOT DOCUMENT
  GENERATE FOOTER
#END FORALL
#

The actual implementation of this pseudo-code usually have a fairly complex set of nested loops that implement the FORALL part of the pseudo-code. Changing the root document then involves changing all the loops, and for a complex root document, the loops are complex in any case. The ##-preprocessor can help it look more like the pseudo-code:

#
##void forall_begin() {
#FORALL ELEMENTS IN ROOT DOCUMENT
##} // forall_begin()
#
##void forall_end() {
#END FORALL
##} // forall_end() {
#
#
##forall_begin();
  GENERATE  HEADER
##forall_end();
#
##forall_begin();
  GENERATE BODY
##forall_end();
#
##forall_begin();
  GENERATE FOOTER
##forall_end();
#

To be specific, suppose you had a Collection of classes containing a transaction (String), along with a debit (negative) or credit (positive) number, and you wanted a report with all credits followed by all debits. With one level of the preprocessor, you could write:

Example 7.2. Document/View Renederer without ##-preprocessor

#static import bpp.Format.*;
CREDITS:
#int count=0;
#for (Iterator i = account.iterator(); i.hasNext(); ) {
#  Transaction transaction = (Transaction) i.next();
#  String information = transaction.information;
#  double amount = transaction.amount;
#  if (amount >= 0) {
$(++count). $information $(N(amount,"$#,###.00"))
#  } // if credit
#} // account

DEBITS:
#int count=0;
#for (Iterator i = account.iterator(); i.hasNext(); ) {
#  Transaction transaction = (Transaction) i.next();
#  String information = transaction.information;
#  double amount = transaction.amount;
#  if (amount < 0) {
$(++count). $information ($(N(-amount,"$#,###.00")))
#  } // if credit
#} // account

The #-preprocessor certainly has a lot of kinda-the-same going on, so I am inclined to write this instead:

Example 7.3. Document/View Renederer with ##-preprocessor

#static import bpp.Format.*;
#
##void forall_begin() {
#int count=0;
#for (Iterator i = account.iterator(); i.hasNext(); ) {
#  Transaction transaction = (Transaction) i.next();
#  String information = transaction.information;
#  double amount = transaction.amount;
##} // forall_begin
#
##void forall_end() {
#} // account
##}

CREDITS:
##forall_begin();
#  if (amount >= 0) {
$(++count). $information $(N(amount,"$#,###.00"))
#  } // if credit
##forall_end();

DEBITS:
##forall_begin();
#  if (amount < 0) {
$(++count). $information ($(N(-amount,"$#,###.00")))
#  } // if credit
##forall_end();

Now all the looping logic is in one spot, and just recycled for each stage of the rendering. The more loops, the better this solution starts to look.

Uses of ###

Your ad here!



[2] I'm still lying, but less and less as time goes on. The only thing I'm leaving out now is that BPP dynamically brings preprocessors to life as they are required, but it fudges things so that it is equivalent to these six seperate passes.

Appendix A. Running BPP (part 2)

Using the

java -cp ... bpp.BPP args...

works, but gets tedious, and is sure to get scripted pretty quickly in your build process. Here are a few suggestions for simplifying the execution of BPP:

  • Unixish/cygwin users can get the full BPP distribution and put that somewhere and then setup BPP using:
    cd somewhere; . bin/profile
    . This will set BPP_HOME to somewhere, set BPP_CLASSPATH to all the jar files in $BPP_HOME/bin directory, and add $BPP_HOME/bin to your PATH. After this, typing
    bpp args...
    will do the trick for running bpp.
  • Windows users without cygwin. I shed a small tear for you. There is a batch file BPP.BAT in the full bpp distibution for you, but since windows couldn't even get a stinking for loop right in their scripting language, you have to set up your environment variables up yourself.
  • Unixes without the full bpp distribution. I suggest setting up an alias, such as: alias bpp='java -cp bpp_classpath bpp.BPP'. You don't need much of the distribution to get the unix scripts to work nice, just get the bin folder of the distribution, so that is probably the better solution.
  • Windows without the full bpp distribution. I suggest setting up an environment variable which behaves similarly to the unix alias above, so you can just type: %BPP% args.... One BPP project is set up like this:

    Example A.1. BPP in a batch file

    set JAVA=java lasspath "../common/lib/bpp.jar;../common/lib/bsh-2.0b1.jar;../common/lib/jdom.jar;../common/lib/ExcelXML.jar"w %JAVA_FLAGS%
    set BSH=%JAVA% bsh.Interpreter
    set BPP=%JAVA% bpp.BPP
    

Appendix B. Docbook

This tutorial was created using DocBook. See DocBook: The Definiative Guide and www.sagehill.net for a tutorial on using DocBook tools.