The very first step to build a custom Syntax is (apart of having ready all the classes we talked about in the previous section, such as SettingsInitializers and TokenContexts) to extend the Syntax class, which is a little bit tricky.

Extending the Syntax class requires creating a public void constructor that updates a superclass variable (and this is the tricky part), and overriding the protected "parseToken()" method that translates text into tokens. The basic skeleton for building our custom Syntax object looks then like this:

public class SchemeSyntax
  extends Syntax
{
  public SchemeSyntax()
  {
    // The tricky part!
    tokenContextPath = SchemeTokenContext.contextPath;
  }
  // @Override
  protected TokenID parseToken()
  {
    // Some magic here!
    return null;
  }
}

The editor module infrastructure will make calls to our overriden "parseToken()" method for us, automatically. And it will be making calls to that method until you analyze the whole text buffer.

And what's the text buffer we have to scan, you ask? Well, that's stored in an internal protected variable in the Syntax class. In fact we have some other juicy variables there. Let's take a look at what these variables mean:

So that's it. Building a custom Syntax object is not that difficult after all: it requires some some attention.

If you want to see real-life examples of custom Syntax objects you may want to take a look at NetBeans' SQLSyntax, the JavaSyntax or at the NetBeans Syntax Highlighting Tutorial Syntax Example.

But, hold down just a moment, there're some juicy tricks you may use to ease building custom syntax objects!!

If you take a detailed look at SQLSyntax you'll see a helper method that recognizes keywords. This helper method creates a new String from the text buffer (using "new String") and that then compares that String against a constant list of SQL keywords.

Of course they're creating new things, so this is not as space-efficient at the JavaSyntax, for instance. Anyway the current garbage-collection technology in recent JVMs has improved so much that creating these short-lived objects is very, very efficient nowadays.

In fact you can even create more interesting short-lived objects: you can create "Matchers" and use regular expressions to match keywords. As you probably know, regular expressions are strongly related to finite state authomata, and so may help us build our custom Syntax state machine. You could, for instance, define the following Pattern that is used to detect Scheme constructs:

private static Pattern CONSTRUCT  = Pattern.compile("and|begin|case|cond" +
    "|define|define-syntax|delay|do|else|if|lambda|let|let\\*|letrec" +
    "|letrec-syntax|let-syntax|or|quasiquote|set!|syntax-rules" );

And then use a helper function to detect if a part of the buffer is a Scheme construct or not, like this:

public boolean isConstruct()
{
  SegmentCharSequence chars = 
    new SegmentCharSequence( buffer, tokenOffset, offset - tokenOffset );
  return CONSTRUCT.matcher( chars ).matches();
}

And we would only be creating a lightweight "SegmentCharSequence"^[9] and a Matcher object per iteration. Again note we're trading space-efficiency by ease of development. And that seems fair to me.

Using regular expressions is most handy if you want to parse complex tokens, such as Scheme numbers. Scheme number syntax is somewhat involved, because you have rational numbers, complex numbers, and exact numbers, for instance. Choosing your regular expressions wisely may boost your productivity when building custom syntax objects.

Remember, again, that if you want to do things really efficiently then you may want to experiment with the lexer module: the new way to do syntax highlighting on top of NetBeans.

The second trick I suggest for building custom Syntax objects is to fully exploit unit testing. Building unit tests in the NetBeans IDE is just a piece of cake (just right-click on your custom Syntax.java and choose Tools/Create unit tests), and building unit-tests for your syntax is not that difficult either.

The NetBeans Syntax Highlighting Module Tutorial contains a good example for building your Syntax unit tests.