caBIO iPhone Application 1.1

The 1.1 update for the caBIO iPhone App is now live on the App Store. This release fixes a bug with the cached "brca1" search which is loaded when the app first starts. In addition, the interface has been improved with result caching, a refresh button, and new toolbar icons.

caBIO iPhone Application 1.0

I've been working on an iPhone App for the caBIO data service and just submitted it to the App Store for review. This site will serve as the "support site" which Apple requires each app to have.

Update (7/18/2009): The application is now live on the App Store. Search for "cabio" to find and install it.

Struts 2: Putting i18n messages in the page context

Struts 2 has very sparse documentation when compared with Struts 1, so this took me longer than necessary to figure out, and I thought I'd document it here. Let's say we have a property called my.property in a properties file called messages.properties. We want to display a substring of it on a JSP. The first thing to do is modify struts.properties to load the properties:

struts.custom.i18n.resources=messages

This will load messages.properties into Struts. There's a similar setting available in struts.xml, but beware of a bug which renders it useless in versions prior to 2.0.4. The project I'm working on happened to be using Struts 2.0.1, so I had to add the struts.properties file for this one setting. You can just drop this file into the classpath (i.e. WEB-INF/classes).

Now if we just want to display this property in a JSP, it's actually quite easy. First make sure to load the Struts 2.x Tag Library:

<%@taglib prefix="s" uri="/struts-tags"%>

And then just display the property:

<s:text name="my.property"/>

But now lets say we want to do some mangling on the value which would be difficult or impossible with OGNL alone. We need to get the value into the page context:

<s:set name="myprop" scope="page" value="%{getText('my.property')}"/>
<%
    String url = (String)pageContext.getAttribute("url");
    // do stuff to myprop
%>
<%=myprop%>

Now, I realize that modern JSP theory frowns on introducing any Java code directly into the page, but there are cases where this is necessary (for example if you don't have access to anything else!). Figuring out how to get a value from the property bundle and set it into the page context is not really documented in any Struts documentation and involves some tricky syntax.

Auto-complete: the server side

Last time we looked at developing a auto-complete implementation, but we stopped short of implementing the server side. We do have a servlet that returns a list of strings, but those strings are hard-coded. In this post, we'll look at creating a more useful servlet which returns relevant suggestions.
First, we note that the jquery.suggest plugin always sends a parameter to the server called q which is the text currently typed into the search box. We should implement our servlet in such a way that it returns ~10 results relevant to the query string q. What is "relevant" is obviously very application dependent. It's important that the search return as quickly as possible, because the result list must be updated upon each keystroke.
The servlet can be very simple, depending on your needs. If your text box is for a disease name, just have the script search the list of diseases for something starting with q and you're done! But suppose you want to auto-complete against a full domain model. We may have several relevant objects in our database, like Gene, Disease, Drug, Organ, and so forth. How do we build our phrase list, and more importantly, how do we order it so that the most useful results are presented first?
Let's create the database table of keywords first.

    CREATE TABLE KEYWORD ( 
        ID NUMBER NOT NULL,
        SCORE NUMBER NOT NULL,
        VALUE VARCHAR2(512) NOT NULL
    );
    ALTER TABLE KEYWORD ADD CONSTRAINT PK_KEYWORD PRIMARY KEY (ID);

We can very easily dump words and phrases into here from any class, but what about the score? When the user types "b" we want to return the 10 best results starting with "b", but we may have 10000 of them in the KEYWORD table, so how do we score them to pick the top 10?
Generally, it is most useful to sort results by popularity. If you are Google and have an abundance of users, then you can simply keep track of past search results and put popular searches first. But in the life sciences, it's far more likely that you're creating a smaller application, with a limited user base. In that case, one way to judge the popularity of a phrase is to count how many times it is referenced by other objects. For example, suppose you have a Gene object with an association to an ExpressionArrayReporter. For each Gene, count how many reporters it is referenced by. In this way, you get a rough estimate of how popular that Gene is.
Of course, when you estimate popularity like this, the average scores will vary wildly across classes. To normalize the data, we can order the keywords in each class and then take the order as our real score. This guarantees that each class is equally represented.
After you populate the KEYWORD table you will need to query it from the servlet. I will assume you're using Hibernate, and I'll skip over the ORM which can be easily generated. Presumably you now have a Keyword class which maps to your KEYWORD table in the database. Let's write a method to fetch the word list for a given query string. With auto-complete, it's usually most natural to only match at the beginning of words. The following Hibernate code does just that.

    public List getKeywords(String s) throws ApplicationException {
        List results = null;
        Session session = null;
        String lwrs = s.toLowerCase();
        try {
            session = sessionFactory.openSession();
            Criteria c = session.createCriteria(Keyword.class);
            c = c.add(Restrictions.or(
                new EscapingLikeExpression("value",escape(lwrs)+"%"),
                new EscapingLikeExpression("value","% "+escape(lwrs)+"%")));
            c = c.addOrder(Order.desc("score"));
            c.setMaxResults(10);
            results = c.list();
        }
        finally {
            session.close();
        }
        return results;
    }

    private String escape(String s) {
        return s.replace("\\", "\\\\").replace("_", "\\_").replace("%", "\\%");
    }
    
    private class EscapingLikeExpression extends LikeExpression {
        EscapingLikeExpression(String propertyName, String value) {
            super(propertyName, value, '\\', false);
        }
    }

Note that escaping special characters is not that straight-forward with Hibernate's Criteria objects. In any case, you can now call getKeywords in your servlet and print out all the Keyword values returned.

public class AutoCompletionService extends HttpServlet {

    public void doGet(HttpServletRequest request, HttpServletResponse response) 
            throws ServletException, IOException {

        String q = request.getParameter("q");
        PrintWriter pw = new PrintWriter(response.getOutputStream());
        
        response.setContentType("text/plain");
        
        try {
            for(Keyword k : getKeywords(q)) {
                pw.println(k.getValue());
            }
        }
        catch (ApplicationException e) {
            log.error("AutoCompletion error",e);
        }
        
        pw.close();
    }
}

You should have a working auto-complete, minus error handling. You can tweak the population algorithm for the KEYWORD table to heart's delight. You might expect different words to come up when you type in a particular string. Tuning that is more of an art than a science.

Auto-complete: the client side

Adding an "auto-complete" or "suggestion" feature for your search is an easy way to greatly enrich your users' experience. This is especially true in applications related to the life sciences, where difficult, lengthy terms are commonplace. Try typing "mechl" into Google, and it immediately suggests "mechlorethamine." The implementation of such a feature is naturally divided along the client/server chasm. In this post I will describe the very simple client-side implementation. Next time, I'll describe the server-side, which may become complicated if you have a very large database without user search statistics.
In the case of a web application, the client will be a simple Javascript-driven text box. Popular Ajax frameworks like jQuery have auto-completion implementations out of the box. For example, jQuery has a number of suggestion plugins. The best for my needs ended up being jquery.suggest, an extremely simple plugin, that nonetheless features all the important basics such as client-side caching. Implementing the client-side was a matter of importing a few files, and enabling the listener on my input box:

<link rel="stylesheet" type="text/css" href="css/jquery.suggest.css"/>

<script src="js/jquery-1.2.3.min.js" type="text/javascript"></script>
<script src="js/jquery.bgiframe.js" type="text/javascript"></script>
<script src="js/jquery.dimensions.min.js" type="text/javascript"></script>
<script src="js/jquery.suggest.js" type="text/javascript"></script>

<input name="searchString" id="searchString" value="">
<script type="text/javascript">
    jQuery(function() {
        jQuery("#searchString").suggest("suggest",{minchars:1});
    });
</script>

That's it! I passed in minchars:1 because the default is 2, and I wanted the user to get immediate feedback when they were typing. To have this work well, I also had to make a slight tweak to jquery.suggest.js to line 100:

} else if (($input.val().length != prevLength)||(prevLength === 0)) {

There are other customizations you can do in the CSS, and by passing additional options to the suggest() function. At this point your server-side should just be a dummy servlet located at /suggest, which returns a newline-delimited list of strings. When you type in the input box, the list returned by your script should appear below. In the next post, I will describe how to make that script return relevant results for what the user is typing.

Highlight occurrences with Javascript

Just a simple function today which takes a list of words (strings) and highlights all occurrences of each word in some text. This is useful for presenting search results from an Ajax query for example, where the highlighting is not done on the server side.

function highlight(label, wordList) {
    var h = label;
    for(var i=0; i<wordList.length; i++) {
        h = h.replace(new RegExp("("+wordList[i]+")","gi"), "<b>$1</b>");
    }
    return h;
}

Named parameters in Javascript

Many Javascript programmers are aware of the benefits of using hashes for parameters to constructors and other complicated functions. Using a hash effectively gives you named parameters:

  function MyWidget(opt_opts) {
    this.background = opt_opts.background  || "white";
    this.border = opt_opts.name || "1px solid black";
  } 

Named parameters are often used for optional parameters that have good defaults. If the programmer is happy with the defaults, they can instantiate a class very easily, passing only the parameters they want to change.

  var widget = new MyWidget({background:"red"});

Establishing the defaults can be very easy with the || trick, as above. If no parameter is given, the || operator yields its second operand, the default value for the parameter. However, one has to be careful with boolean parameters. Here's some code I noticed recently in the production version of a popular Javascript library:

  function MyWidget(opt_opts) {
    this.editable = opt_opts.editable || true;
  } 

See the problem? The editable variable will always be set to true, no matter what the user passes in! It's easy to get carried away with the || trick, and start thinking of it as a "default value operator" when in fact it has boolean semantics. In fixing this, you end up writing something like:

  function MyWidget(opt_opts) {
    this.editable = (opt_opts.editable!==undefined) ? opt_opts.editable : true;
  } 

Not quite as pretty, is it? One alternative method is to define the defaults up front, with a hash literal, and then copy in the user options as "overrides":

  function MyWidget(opt_opts) {
    this.options = {
      background: "white",
      border: "1px solid black"
    };
    for (var s in opt_opts) {
      this.options[s]=opt_opts[s];
    }
  }

Of course, you could always take the fool-proof, Java-inspired approach taken by markertracker.js. Each option is meticulously defined with a default "constant", and then the user option is only copied if it's defined:

  this.iconScale_ = MarkerTracker.DEFAULT_ICON_SCALE_;
  if (opts.iconScale != undefined ) {
    this.iconScale_ = opts.iconScale;
  }

It's not pithy, but I suppose it works. Personally, I'll stick with the first approach. It reminds me of my Perl days.