Copyright © 2002-2003 Isaac W. Foraker
Abstract
This is for the LibTPT template scripting language for C++, based on build 1.21. For the most up to date documentation, go to http://tazthecat.net/~isaac/libtpt/.
Table of Contents
List of Examples
Table of Contents
TPT is a template scripting language implemented in C++, designed to be integrated with C++ programs. It is intended as a fast, lightweight replacement for fullblown scripting languages, providing an easy interface for embedding TPT scripts in C++ code. The TPT language was originally based on PML.
Known uses for TPT include generating CGI web pages, C and C++ source code, Make files, and e-mail messages.
I find many of today's template languages frustrating and hard to work with. Originally, I simply wanted to perform on-the-fly variable substitution, but as my needs grew, so did the idea for TPT. For a time, I used the PML language, but that was written entirely in Perl and lost the support of its creator.
Though I initially only intended to use TPT for generating HTML responses for CGI scripts, it was quickly adopted by the clo++ project for C++ code generation, and I have since used it for generating E-mail, Makefiles, and C source code as well.
As with most language choices, the decision to use TPT is usually a matter of personal tastes. TPT has a low learning curve and can be integrated into a project quickly.
One important point to note is that TPT is not about I/O. It is optimized for output only (like generating source files or web pages), so if you are designing interactive applications, TPT is not for you. However, if you are looking for a simple solution for generating customized text files, give TPT a try.
Table of Contents
LibTPT has been compiled and tested under the following platforms:
| FreeBSD 4.5 GCC 3.x |
| Linux GCC 3.x |
| Win32 MSVC 6.0 |
LibTPT is also rumored to build on:
| Sparc Solaris Forte C++ |
| Sparc Solaris GCC 3.x |
| Mac OS X GCC 3.x |
| Win32 MSVC++.Net (VC 7.0) |
Currently, LibTPT officially supports Microsoft Visual C++ 6.0, but is known to work with Visual C++.Net (7.0) as well. For optimal performance, it is highly recommended that you install the STLPort (http://www.stlport.org/).
After installing the STLPort, add the following line to %INCLUDE%\stlport\stl_user_config.h:
#define _STLP_DO_IMPORT_CSTD_FUNCTIONS 1
The %INCLUDE% variable is the path where the stlport directory was installed. This will make the STL import the standard C library functions into the "std" namespace, making Visual C++ act more like a standard C++ compiler.
Make sure you set %INCLUDE%\stlport as your first include path, or the wrong version of the header files will be included. Also, since MSVC 6 does not properly handle the standard library in the std namespace, avoid broad statements like
using namespace std;
Once your build environment is set up correctly, open w32/libtpt.dsw. The libtpt workspace includes all projects necessary to build libtpt and its tests. The default build configuration is intended for a generic x86 platform. Change these options if you so desire.
Both debug and release configurations are available for build. Each project should build with "debug" in its name when compiled as a debug project.
Batch build all the release versions, then run tests:
cd test/ test.bat
At the time of the writing of this document, LibTPT has only been tested with GCC 3.x. GCC 2.95.x is known to have problems compiling LibTPT.
The LibTPT configuration system requires Perl 5.005 or later. The configure.pl script requires that either the CXX environment variable is set to the path of your C++ compiler, or that you specify the path to you C++ compiler with the --cxx command line option.
./configure.pl --cxx /path/to/g++ [--help] make ./test.sh make install
By default, configure.pl will use /usr/local as the install directory. Use --prefix=/new/install/path to override this.
Table of Contents
This introduction to the TPT template language is intended for users who are already familiar with programming. If you are not familiar with the concepts of numbers, strings, arrays, functions, and expressions, you may want to read an introductory level programming book.
TPT reserves the following characters.
@ { } " ' \
To use one of these characters in text, use the backslash (\) escape character. For example, \\ in TPT will evaluate to a single \. Not escaping these reserved characters can lead to undesirable effects.
Variables are objects that hold values. In TPT, a variable may hold a value of type integer, string, array of values, or hash of values.
A variable in TPT is identified in plain-text as ${id} and in an expression as just id, where id is the name of the variable. The value of a variable is set using the @set TPT command.
@set(id, 123)
@set(id, "The red fox runs.")
A variable may also be cleared using the @unset TPT command.
@unset(id)
See Example 2.2, “Contents of a fruit basket” for an example of how variables may be used in a template.
Example 2.2. Contents of a fruit basket
Listing of fruitbasket.tpt
@set(fruit, "Apple") @set(basket, 12) The basket holds ${basket} ${fruit}s.
Output of fruitbasket.tpt
The basket holds 12 Apples.
Variables can also hold more complex values, such as arrays or hashes. Array variables look like,
${array[index]}
and hashes look like,
${array.member}
In arrays, the index can be either a number or an expression that evaluates to a number. Example 2.3, “A fruit basket array” demonstrates how to set a variable to an array, then read the variable.
Example 2.3. A fruit basket array
Listing of fruity.tpt
@set(basket, "apple", "orange", "lime", "banana", "pear")\ The fruit basket contains \ @foreach fruit (basket) { ${fruit} }.
Output of fruity.tpt
The fruit basket contains apple orange lime banana pear .
Note the use of spaces in this example. Backslashes are used to join lines. For more details about controlling whitespace and carriage returns, see the section called “Whitespace & Carriage Returns”.
Example 2.4, “Random array index” demonstrates how to use an array index in a creative fashion.
Example 2.4. Random array index
Listing of randfruit.tpt
@set(basket, "apple", "orange", "lime", "banana", "pear")\ The fruit of the day is ${basket[@rand(@size(basket))]}.
Possible output of randfruit.tpt
The fruit of the day is banana.
To determine the type of variable, use @isarray, @isscalar, and @ishash.
Comments are denoted by the @# (at-pound) symbols. TPT only supports to end-of-line comments. I.e. everything on the line following @# is a part of the comment.
Hello World! @# This is the Hello World Comment.
In the above example, everything after Hello World! will be ignored by the TPT parser, including the whitespace. Comments affect whitespace depending on use. All whitespace immediately preceeding a comment is ignored. In the case of a full line comment,
@# This is a full line comment @# This is not
the entire line, including carriage returns, is ignored. In order to be classified as a full line comment, @# must be the very first characters on the line. Preceding the @# with whitespace will only cause the whitespace and comment to be ignored. This is useful if you need to insert a certain number of blank lines into a template.
@# Insert some blank lines @# blank line 1 @# blank line 2 @# blank line 3 @# blank line 4 @# blank line 5
Expressions in TPT can only be executed when placed inside a TPT wrapper like a function, macro, array [], loop, or conditional expression. TPT employs a full Recursive Descent parser, so complex infix expressions are supported. Use @eval to evaluate an expression in text.
Since the primary purpose of TPT is to format templates into text, controls are provided to remove whitespace and carriage returns. Using a backslash (\) at the end of a line will join lines. Any whitespace before @< will be ignored, and any whitespace after @> will also be ignored.
Macros are a powerful feature of TPT. User defined Macros in TPT are very similar to user defined functions or subroutines in other languages. A macro must be defined before it is used. A macro definition looks like:
@macro(mymacro, param1, param2, ...) {
macro body
}
To access this macro once its defined would look like:
@mymacro(param1, param2, ...)
Table of Contents
dumptpt.cxx
#include <libtpt/tpt.h> #include <iostream> #include <stdexcept> // Dump the parsed template output straight to std::cout void dumptemplate(const char* filename) { try { // Instantiate a parser TPT::Parser p(filename); // Add an include path (only ./ searched by default) p.addincludepath("./include"); // Parse the template directly to standard output if (p.run(std::cout)) { // handle reporting of template errors } } catch(const std::exception& e) { std::cout << "EXCETPION: " << e.what() << std::endl; } }
Table of Contents
In C++, it is sometimes handy to specify that a library function call a user defined function for customizable behavior. In the standard library, the generic binary search and quick sort functions do this.
void *bsearch( const void *key, const void *base, size_t num, size_t width, int (*compare ) ( const void *elem1, const void *elem2 ) ); void qsort( void *base, size_t num, size_t width, int (*compare )(const void *elem1, const void *elem2 ) );
Each of these functions allows the user to specify a custom compare function for a user defined data type. LibTPT provides a similar method to allow the user to specify a C++ function that will be called from a TPT template.
Callbacks give the user the power to extend TPT beyond its core functionality. For instance, a project may require TPT to do some floating point math, which is not available in the core language. On the more extreme, a user may decide to extend TPT with CGI or Database functionality. Whatever the reason, callbacks let the user implement fast TPT functions within the C++ code.
Remember, while it is tempting to write callback modules that allow TPT to do anything a full blown language can do, TPT was designed to take advantage of the speed of C++ for variable management. Therefore, functionality like database access or CGI processing should remain in your C++ code for optimal performance.
A TPT callback must be declared in the form:
bool mycallback(std::ostream& os, TPT::Object& params);
Note that the first parameter is a reference to an output stream. All communication between the callback and the LibTPT parser are handled with streams. So, to have the callback print a message would look like:
os << "My Message";
The second parameter is of type TPT::Object, which represents a generic object in LibTPT. The object may hold a type of scalar, array, or hash. The params variable passed to the callback is guaranteed to hold an array, but that array may hold scalars, arrays, hashes, or any combination of those types, depending on how the callback was called from within the TPT template. The enumerated types of object available to a callback function are TPT::Object::type_scalar, TPT::Object::type_array, and TPT::Object::type_hash, with the corresponding access methods TPT::Object::scalar(), TPT::Object::array(), and TPT::Object::hash(). If a call to the incorrect access method is made, an exception will be thrown. The parser does its best to handle these exceptions without interrupting the template translation.
When accessing a member of Object::array() or Object::hash(), be sure to use the get() method to get the member's real address. Member of Object arrays and hashes are stored in a smart pointer to ensure proper memory management. Refer to the example below for how to properly access a member of an Object array.
As mentioned earlier, TPT does not have any built-in functionality for dealing with floating point numbers. Suppose you needed a function to sum a list of floating point numbers populated into variables from a database or user form. See Example 4.1, “A callback example: @fsum()” for an example of what the code might look like for a floating point summation function.
Example 4.1. A callback example: @fsum()
#include <libtpt/tpt>
#include <ostream>
// Define a function for summing floating point numbers
bool fsum(std::ostream& os, TPT::Object& params)
{
// Make a reference to the array for easier typing.
TPT::Object::ArrayType& pl = params.array();
// initialize the work variable
double work=0;
// Set up iterators and loop.
TPT::Object::ArrayType::const_iterator it(pl.begin()), end(pl.end());
for (; it != end; ++it)
{
// Make a reference to the object for easier typing.
TPT::Object& obj = *(*it).get();
// Only add the object if it is a scalar.
if (obj.gettype() == TPT::Object::type_scalar)
work+= atof(obj.scalar().c_str());
else
return true; // expected a scalar!
}
os << work;
return false;
}
This callback is then registered with the TPT::Parser::addfunction() method.
TPT::Parser p(buf);
p.addfunction("fsum", &fsum);
The above example demonstrates how simple it is to loop through an array of scalar examples. As an exercise, modify fsum to be able to process sub-arrays of floats.
As with most programming interfaces, using TPT callbacks can open some serious security holes if you are not paying attention. It is recommended that you do not create callbacks that can execute arbitrary programs, and when processing user input, always bounds check the input. Bounds checking is already handled by the std::string class, so the best bet is to stick with std::string. Do not use sprintf for formatting strings. If you must format a string, and absolutely cannot use iostreams, use snprintf instead, and double check your format string for any errors.
Table of Contents
The following functions affect the flow of the Lexical Analyzer, and are never parsed.
Ignore everything until the end of the line. If the comment is the first item on the line, the entire line will be ignored.
@# This is a whole line comment, so this line will be ignored @# This is not @mymacro(a, b, c) @# describe my macro
When @tpt_ignoreindent is called (with no parenthesis), all indentation on the following lines will be ignored. Use @tpt_noignoreindent to stop ignoring indentation.
@itpt_ignoreindent\
@if (x) {
Four score and seven years ago...
} @else {
We the people of the United States...
}
Note: @ignoreindent and @noignoreindent are depricated.
When @tpt_ignoreblankline is called (with no parenthesis), any line with no content (including white-space) will be ignored.
@tpt_ignoreblankline\
@macro1() {...}
@macro2() {...}
@macro3() {...}
@tpt_noignoreblankline
Note: @ignoreblankline and @noignoreblankline are depricated.
The ignore space operators instruct the lexical analyzer to ignore any space preceding @<, or any space following @>.
@# ignore spaces after @> @> This should be flush to the left @# ignore spaces before @< @<This should be flush to the left
Include and parse a TPT header file, usually a filename ending in .tph.
@include("macros.tph")\
@include('heading.tph')\
Assign a value or values to a variable.
@set(identifier, expression)
@set(identifier, expression1, expression2, ...)
Assign a value or values to a variable only if the variable is empty.
@set(identifier, expression)
@set(identifier, expression1, expression2, ...)
Loop over a block of code while the loop expression is true.
@while (expression) { loop block }
Compare two strings, giving -1 if string1 is less than string2, 0 if string1 is equal to string2, or 1 if string1 is greater than string 2. @comp and @strcmp are aliases for @compare.
Concatenate a list of strings together, and return the new string.
@set(newstring, @concat("prefix", "suffix"))
Get a substring of the given string.
@# Extract one character from the alphabet @set(x, "abcdefghijklmnopqrstuvwxyz")\ @set(i, 9)\ @substr(x, i, 1)
This sample will return "j" from the x symbol.
Return 1 if string is empty or 0 if string is not empty. If given a hash or array, @empty will report an error and return nothing.
@if (@empty(x)) {
@set(x, "[empty]")\
}
Pad the left side of the specified string with spaces until the string is the specified width. If the string is already greater than or equal to the specified width, @lpad simply returns the original string.
@lpad(string, width)
Pad the right side of the specified string with spaces until the string is the specified width. If the string is already greater than or equal to the specified width, @rpad simply returns the original string.
@rpad(string, width)
Get the average of the specified list of integers. Numbers may be passed as part of an array, or individually as parameters. The result will be returned as a truncated integer (i.e. the result will always be rounded down).
@set(numlist, 1, 2, 3)\ @avg(numlist) @avg(1, 2, 3)
Check a variable to see if it holds an array. Gives 1 if the variable is array, or 0 if the variable holds a different type or is not defined.
@if (@isarray(x)) {
x is an array
}
Get the size of an array. Given a scalar, @size will return 0 if the scalar is an empty string (i.e. ""). Given a hash, @size will return the number of keys and values in the hash. So, a hash with 2 keys will have a size of 4.
When given multiple parameters, @size will return the sum of the sizes of all parameters.
@size is not recursive. The sum returned by @size will not include the sizes of sub-arrays or hashes.
Check a variable to see if it holds a hash. Gives 1 if the variable is a hash, or 0 if the variable holds a different type or is not defined.
@if (@ishash(x)) {
x is a hash
}
All LibTPT library classes, functions, and variables exist in the TPT C++ namespace.
All TPT header files can be included with:
#include <libtpt/tpt.h>
#include <libtpt/parse.h>
The TPT::Parser class is the primary interface to the TPT library. For basic template processing, TPT::Parser is the only class you will need.
Parser(const char* filename); Parser(const char* filename, const Symbols& st); Parser(const char* buf, unsigned long size); Parser(const char* buf, unsigned long size, const Symbols& st); Parser(Buffer& buf); Parser(Buffer& buf, const Symbols& st);
#include <libtpt/iparse.h>
The TPT::IParser class is the Interactive version of the TPT::Parser class. This version writes updates to the Symbols table back to the Symbols table passed in by the user. This is useful when there is a need to maintain state information between processing templates, or to pass state information back to the calling C++ program.
IParser(const char* filename, Symbols& st); IParser(const char* buf, unsigned long size, Symbols& st); IParser(Buffer& buf, Symbols& st);
#include <libtpt/buffer.h>
The TPT::Buffer class is the base IO class for reading TPT source. Instantiating a TPT::Buffer directly is a useful optimization when you will be parsing the same template repeatedly. A TPT::Buffer is not required to parse a TPT template.
explicit Buffer(const char* filename); explicit Buffer(std::istream* is); explicit Buffer(const char* buffer, unsigned long size); explicit Buffer(const Buffer& buf, unsigned long start, unsigned long end);
#include <libtpt/symbols.h>
The TPT::Symbols class may be used to predefine a symbols table for a TPT template before the template is passed to TPT::Parser. The TPT::Symbols object must be populated before it is passed to the TPT::Parser. A TPT::Symbols table is not required to parse a TPT template.
#include <libtpt/object.h>
The TPT::Object class is intended for intermediate to advanced users. The TPT::Object is the internal representation of data used within LibTPT. TPT::Object is designed with speed and exception safety in mind. The TPT::Object will automatically become whatever type it is referenced as.
There are two basic uses for TPT::Object in your code. First, to access variables passed to a user defined callback function. Second, to set up complex data structures before entering the TPT Parser.
An object may be a scalar (std::string), an array using integer indexes, or a hash keys on a string. For ease, the following members are provided for access to Objects.
TPT::Object::scalar(void) Returns a reference to the std::string
object.
TPT::Object::aval(index) Returns a reference to the TPT::Object
at the specified array index.
TPT::Object::hval(key) Returns a reference to the TPT::Object
at the specified hash key.
Using these functions, complex objects can easily be constructed for use inside templates.
TPT::Object fruits;
TPT::Symbols sym;
// Build a somewhat complicated hash of hashes
fruits.hval("good").hval("apple").hval("desc").aval(0) = "Red";
fruits.hval("good").hval("apple").hval("desc").aval(1) = "Green";
fruits.hval("good").hval("apple").hval("desc").aval(2) = "Delicious";
fruits.hval("good").hval("apple").hval("calories") = "150";
fruits.hval("good").hval("banana").hval("desc").aval(0) = "Yellow";
fruits.hval("good").hval("banana").hval("desc").aval(1) = "Yummy";
fruits.hval("good").hval("banana").hval("calories") = "300";
fruits.hval("bad").hval("grapefruit").hval("desc").aval(0) = "Yellow";
fruits.hval("bad").hval("grapefruit").hval("desc"). = "Bitter";
fruits.hval("bad").hval("grapefruit").hval("calories") = "150";
// Assign this object to a TPT symbol
sym.set("fruits", fruits);
Table of Contents
In PML, strings may be passed bare (i.e. without quotes) to macros and functions. TPT requires strings to be in enclosed in either single or double quotes (' or ").
TPT does not automatically remove carriage returns on lines that only contain function calls. To make TPT ignore the carriage return, use a backslash.
@concat is different in TPT. @concat concatenates a variable list of strings together, with no added spaces. The PML version concatenated values onto an existing symbol.
PML: @concat(id, value)
TPT: @set(id, @concat(id, "value"))
"value" may be either a string or an expression.
The new @concat also replaces prepend and append. Instead use:
PML: @prepend(id, value)
TPT: @set(id, @concat("value", " ", id))
PML: @append(id, value)
TPT: @set(id, @concat(id, " ", "value"))
Why is this different? Most functions in TPT are designed to be usable within the expression processor. Some functions of PML, like concat, prepend, and acted on a symbol, instead of returning a value that could be used by the expression parser.
TPT adds the following new functions for your pleasure.
@lc(expr) - lowercase string or expression @uc(expr) - uppercase string or expression @length(expr) - get number of characters in string or expression
| @need keyword supported by PML for loading PML modules. |
| @perl keyword for eval'ing perl expressions. |
| @unless keyword. Just use if (!expr) instead. |
| @until keyword. Just use while (!expr) instead. |
| @rib keyword for default value. Just use @else { nbsp; } for now. Another workaround is to use if (@empty(var)) if you were using @rib to only test a variable. |
| ${ARGV} array variable. |
| @append keyword for appending string to a symbol. Just use new @concat. |
| @prepend keyword for prepending string to a symbol. Just use new @concat. |
Table of Contents
LibTPT Template Language for C++
Copyright (C) 2002-2003 Isaac W. Foraker (isaac(at)tazthecat.net)
All Rights Reserved
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the Author nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE AND DOCUMENTATION IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
LibTPT Primary Programming and Documentation
Isaac W. Foraker (http://tazthecat.net/~isaac/)
Original PML Language, clo++ command line parser generator, and cxxtools
Peter J. Jones (http://pmade.org/pjones/)