On constant databases

The problem: handling NetBeans autocompletion

As you may know I keep on working on an IDE for Scheme.

Of course I want to provide code completion for it, so I've been through the R5RS Scheme standard looking for Scheme keywords and their documentation.

And I ended up with a huge amount of information. For each keyword I have all the documentation for it, including examples, etc.

So in order to provide code completion I was faced with the problem of seeking all the documentation for each keyword.

I could, of course, store all the documentation in a HashMap, using the Scheme keyword as a key, and all the docs and examples as the value.

So I decided to store all that information, in the for of (key,value) pairs, in persistent storage (in the disk). That'd save memory.

And I needed to be able to retrieve it fast, because autocompletion needs to be fast, right?

The solution: JDBM

There're specific databases for doing this sort of thing: just seeking information quickly in persistent storage.

If you're a Unix user then CDB is probably the best option there, because it can seek info in between one million registers in less than one second (see this benchmark).

But I'm using Java, so I had to resort to either JDBM or to SG-CDB

I have been using JDBM in the past, so the choice was obvious. I created a ".db" file with JDBM, including all the R5RS keywords and documentation in it.

And I bundled JDBM inside a Library Wrapper Module, and the database inside one of my modules.

And you know what? The result is impressive: code completion works very fast in the resulting IDE.

Lessons learned: bundling files with your NetBeans modules

Of course I had to learn how to do code completion in NetBeans which is not very complex.

The problem was with my database file. JDBM needs to know what the File is, because it uses a RandomAccessFile to access the database. So I could not use "Class.getResourceAsStream" and bundle the file within my module classes. I had to get access to the real file.

So the problem is this: how do you bundle a file within your NetBeans module and then how to you get access to it after NetBeans is installed?

The solution was not very difficult. Here we go with the recipe:

  • Select the "Files" tab in NetBeans, and create a "release/modules/ext" folder. Place your files there.
  • In your module properties, search "InstalledFileLocator" and add a dependency to the module containing it.
  • In your code, whenever you want to access the file, use the following snippet:
  File myFile = InstalledFileLocator.getDefault()
    .locate("modules/ext/r5rs-completion.db.db",  // A
    "net.antonioshome.scheme.completion", // B


  • "A" is the name of your file (note the missing "release" directory in the path)
  • "B" is the name of your module (this isn't strictly neccessary, but makes finding the file much faster).

And it's that simple. I can now bundle constant databases within my NetBeans modules, and that makes Scheme autocompletion very fast.

Keep tuned for the first release of the Scheme IDE, that is expected by the end of October (2008!).

Happy netbeans-ing, Antonio

blog comments powered by Disqus