How to get autocomplete suggestions from the database for Vaadin’s ComboBox

With class com.vaadin.ui.ComboBox the Vaadin Toolkit provides a drop-down selection component that allows you to select an item from a set of values by offering a list of suggestions based on your current input. This works out of the box without having to implement the suggestion logic yourself. You only need to provide a list of selection items that you’ll put into the ComboBox as its data model and then specify the desired filtering mode.

This has one caveat, though. If the list of selectable items contained in the ComboBox is very large, you cannot use this component in a memory-friendly way. That’s because all of the available items in the ComboBox need to be kept in the user session for the suggestion logic to work. You cannot dynamically load suggestions from some external storage and feed them to the drop-down list on demand.

In this article, I’m going to describe a way how you can tweak Vaadin’s ComboBox in such a way that loading suggestions from an external service is possible after all.

Note: You can download a working demo project for this article from GitHub.

If you need functionality that Vaadin does not offer out of the box you typically implement your own custom component. This includes writing a client-side widget with GWT and its server-side Vaadin counterpart. In case you need a drop-down component that provides suggestions from a large data set, you could take that approach and write your own component. However, this has several drawbacks, the most important of which are that you basically duplicate functionality that’s already there (in component ComboBox) and that this approach is relatively complex and time-consuming, let alone error-prone.

This should be easier to accomplish. It would be best if we could reuse the existing ComboBox in some way so that we can profit from its existing features. The problem with Vaadin’s ComboBox, however, is that it does not offer any API for plugging in a different suggestion engine. The filtering algorithm is virtually baked in the implementation of ComboBox and its container data model.

It is still possible to circumvent this restriction with a simple trick (some might call it a hack), though. Let’s investigate how that works.

Here’s an outline of what we need to do:

  1. Subclass com.vaadin.ui.ComboBox and overwrite its protected method ComboBox#buildFilter() with our own implementation.
  2. Implement interface com.vaadin.data.Container.Filter with very restricted functionality: our Filter only needs to transport the current user input.
  3. Write an implementation of com.vaadin.data.Container that performs the actual filter logic.

We’ll go through these steps in detail shortly. I have written a simple demo application that implements this solution. You can download this demo from GitHub to better follow along the following explanations. When you run that application with mvn package jetty:run, you can try out the new ComboBox that fetches its item suggestions from a (mock-)database connection.

In the example, I want to load a list of country names from the database and serve these names as item suggestions in a ComboBox. As this list could potentially grow very large in similar scenarios, I don’t want to load all values from the database all at once, but rather only a subset that matches the current user input. In other words, the suggestion algorithm should run in the database and not in the client code. For instance, I could implement this using a SQL query using a LIKE predicate.

For the sake of simplicity the suggestion algorithm in the demo application is implemented in a straight and naïve fashion by simply using String#startsWith() as filter predicate.

How filtering works in a ComboBox

Before we begin with our custom suggesting ComboBox, we’ll first have a brief look at how Vaadin’s ComboBox handles item filtering internally. This will help us understand the approach used in the demo application better.

Vaadin’s item filtering algorithm is applied on the set of items contained in a select component’s internal container data model. That is, only items contained in a com.vaadin.data.Container can be filtered and sorted. That’s problematic for us, since we don’t want to keep all items in a container (we don’t want to load the contents of an entire database table into the user session, right?). Instead we only want to load a subset of items (those matching a filter) from an external storage and put only these into the component’s data model.

What happens when a user enters text into the input field of a ComboBox is that this input is fed as variable filterstring into the protected method ComboBox#buildFilter() which returns an implementation of interface com.vaadin.data.Container.Filter. Depending on the current value of the com.vaadin.shared.ui.combobox.FilteringMode this filter is a prefix filter (FilteringMode#STARTSWITH), an infix filter (FilteringMode#CONTAINS), or is switched off altogether (FilteringMode#OFF). The filtering mode can be set with ComboBox#setFilteringMode().

The filter returned by ComboBox#buildFilter() is then set as new filter on the container data source of the ComboBox (com.vaadin.data.Container.Filterable#addContainerFilter()) which then does all the remaining filtering magic for the ComboBox.

We need to change this behavior. We want to define our own filtering process, effectively bypassing the whole container filter concept. We have no other choice than to abstain from using Vaadin’s Filter interface since internally a filter is applied to all items in a container. That’s useless for us, since our item data still resides in the database. We cannot change this implementation.

That’s why we need to write our own container implementation for the ComboBox suggestion list. This custom container will externalize the filter algorithm to some service interface. We’re not making use of the Filter interface here.

Implementation

Now let’s delve into the code. Our data model is the following CountryBean JavaBean:

public class CountryBean {
  private Long id;
  private String name;

  // constructor and accessors omitted for brevity
}

This will be the bean type that we’ll put into the data model of the ComboBox. Now, we write an own subclass of ComboBox:

import com.vaadin.shared.ui.combobox.FilteringMode;
import com.vaadin.ui.ComboBox;
import de.oio.vaadin.SuggestingContainer.SuggestionFilter;

public class SuggestingComboBox extends ComboBox {

  public SuggestingComboBox() {
    setItemCaptionMode(ItemCaptionMode.PROPERTY);
    setItemCaptionPropertyId("name");
  }

  @Override
  protected Filter buildFilter(String filterString, 
                               FilteringMode filteringMode) {
    return new SuggestingContainer.SuggestionFilter(filterString);
  }
}

There are only three things to do here. Firstly, we must set the item caption mode to ItemCaptionMode.PROPERTY, and we define the caption property ID as name, i.e. we use the name of a country as item label. Note, that our custom filtering will only work when we use ItemCaptionMode.PROPERTY for the item caption mode. Otherwise, the implementation of ComboBox will deactivate item filtering altogether.

Secondly, we overwrite method buildFilter(). That’s exactly the spot where we apply our little trick. Instead of implementing a sophisticated filter factory method, we simply return a new instance of our custom Filter implementation named SuggestionFilter. Here it is important to notice that we pass the current filterString (this is the String the user entered into the input field) into the constructor of SuggestionFilter.

Looking at the implementation of SuggestionFilter we see that this class only has one purpose: providing a means to intercept the filterString and transport that to our own container implementation. We define SuggestionFilter as an inner class of SuggestingContainer.

// in the class definition of SuggestingContainer
public static class SuggestionFilter implements Container.Filter {

    private String filterString;

    public SuggestionFilter(String filterString) {
      this.filterString = filterString;
    }

    public String getFilterString() {
      return filterString;
    }

    @Override
    public boolean passesFilter(Object itemId, Item item) 
        throws UnsupportedOperationException {
      return false;
    }

    @Override
    public boolean appliesToProperty(Object propertyId) {
      return false;
    }
  }

As you can see, this filter does nothing more than transport the filter String. The two filter methods passesFilter() and appliesToProperty() always return false – they will never be used.

Next we have our own container implementation. Since in this example we work with CountryBean JavaBeans as item type for the ComboBox, we choose BeanItemContainer<CountryBean> as the superclass of our container implementation.

public class SuggestingContainer extends BeanItemContainer<CountryBean> {

  private DatabaseAccessService service;

  public SuggestingContainer(DatabaseAccessService service) 
       throws IllegalArgumentException {
    super(CountryBean.class);
    this.service = service;
  }

  @Override
  protected void addFilter(Filter filter) 
       throws UnsupportedFilterException {
    SuggestionFilter suggestionFilter = (SuggestionFilter) filter;
    filterItems(suggestionFilter.getFilterString());
  }

  private void filterItems(String filterString) {
    removeAllItems();
    List<CountryBean> countries = service.filterCountryTableInDatabase(filterString);
    addAll(countries);
  }

  public static class SuggestionFilter implements Container.Filter {
      // implementation see above
  }
}

In the container’s constructor we pass a data access service class into the object. This service interface opens the door to our external item storage. In this simple example, this service interface defines only one aptly named method:

public interface DatabaseAccessService {
  List<CountryBean> filterCountryTableInDatabase(String filterPrefix);
}

Implementations of this interface are responsible to filter the set of country items with the given prefix String and return a list of matching CountryBeans. Here we see that we outsource the filter algorithm to some external service.

We only need to overwrite one method of the container superclass BeanItemContainer, namely addFilter(). This method is called by the ComboBox each time the user-entered String has changed. That effectively kicks off the filter process. So here we need to step in.

This method is passed an instance of the Filter interface. At this point, we know for sure that the type of the passed instance is our SuggestingContainer.SuggestionFilter, since this method will only be called by our SuggestingComboBox. Therefore, we can safely cast this object into this class. After that, we can grab the current filter String from the filter object (we remember that the sole purpose of this filter implementation is to transport the filter String here) and perform the filtering in the private method filterItems().

We execute the following three steps when filterItems is called:

  1. Clear the container with removeAllItems(),
  2. fetch the filtered items from our external service,
  3. and add the filter result to the item set of the container.

In other words, we replace all items in our container with the result of the last filter process. With that we have managed to shift the filter process from some concrete com.vaadin.data.Container.Filter implementation to an external service.

Conclusion

What have we achieved by that? The filter process is now entirely under our control. The total amount of items from which we derive suggestions based on the user’s input can virtually be unlimited provided that

  1. the filtering algorithm in the external service is fast enough and that
  2. the filter result itself is not too large as this has to be transferred to the client each time the user input changes.

The latter point can be enforced if we only fetch item suggestions when the expected amount of the suggested items is reasonably small. For instance, it would be trivial to adapt the example in such a way that filtering is only triggered after the user has entered at least three characters:

private void filterItems(String filterString) {
  removeAllItems();
  if (filterString.length() >= 3) {
    List<CountryBean> countries = service.filterCountryTableInDatabase(filterString);
    addAll(countries);
  }
}
Short URL for this post: http://wp.me/p4nxik-2iS
Roland Krüger

About Roland Krüger

Software Engineer at Orientation in Objects GmbH. Find me on Google+, follow me on Twitter.
This entry was posted in Java Web Frameworks, Web as a Platform and tagged , , . Bookmark the permalink.

17 Responses to How to get autocomplete suggestions from the database for Vaadin’s ComboBox

  1. S. Hobeck says:

    Hi Roland – thank you very much for posting your solution. It works great!

    I have encountered one issue. I would like to programmatically set a default Country in the ComboBox. I have tried the standard methods: .select(country) and .setValue(country) but unfortunately I get the same result. The ComboBox does not show the country that is set. I am certain that the object is in the container. Any recommendations?

    • Roland Krüger Roland Krüger says:

      Hi, thanks for pointing that out. You’re right, in this simple setup there are still some issues to be handled. Since my solution is a bit hacky, there are some things a Vaadin ComboBox usually does for you but which you have to take care of yourself in my setup. I extended the demo code on GitHub a bit so that the issues you mentioned are fixed. I added an option to define a default value, and the selected value is now shown by the ComboBox. It’s not the cleanest of all solutions, but I think you’ll get the picture. On the downside of this suggesting ComboBox is that you have to explicitly code things like setting the selected value on the container to prevent that from disappearing from the ComboBox. But that’s the price you have to pay when you need to hack around the limitations of ComboBox. I hope that helps.

  2. Hi, Excellent article and very important topic! I’ll feature this in upcoming Vaadin Community Spotlight.

    When I spotted this article, I was so shamed I didn’t fix that mess in ComboBox when that was my job. I did notice this years ago in V6 era, but there was always “something more important” :-( As I was last week working with related topic (simple service layer binding to Table), I wanted to try adapting that to combobox. Got it somewhat working, and at least with Java 8, it is really developer friendly, but I wouldn’t be surprised if there are still some issues. Check it out and tell me what you think, a usage example is in this test case:

    https://github.com/viritin/viritin/blob/master/src/test/java/org/vaadin/viritin/it/LazyComboBoxUsageSample.java

    The solution sure wasn’t easy due to architectural problems in the ComboBox component (and Container interface). Probably the best and cleanest method to solve this would be a completely new custom component, a Field (vaadin interface) but the list of entities should come from somewhere else but form a container.

    • Roland Krüger Roland Krüger says:

      Hi Matti,
      thanks for your feedback! It’s good to hear that this topic will be addressed by you Vaadin guys soon. I see that Vaadin is constantly improving so I’m quite positive that this issue will be tackled by you soon. I’m really looking forward to see a clean and official solution for this, as my approach is just a hack with issues of its own after all. I agree that the cleanest solution would be to write a proper custom component. But then again, that’s quite a task to stem, so using this custom SuggestingContainer of mine can be a quick win. I’ll definitely have a look at your upcoming solution. I don’t think that the container is the problem here, though. I’m quite fond of Vaadin’s container-based approach for data binding. The problem with the current ComboBox implementation (or rather container filtering in general) is that the filtering mechanism is hard coded, i.e. all Items have to be available for the filter to work. It would be better to have a filter that delegates the concrete filtering algorithm to an abstract method. This in tandem with a lazy loading container should do. But I shall now have a look at your approach :)

  3. Pingback: More on lazy data bindinghttps://github.com/viritin/viritin - Blog - vaadin.com

  4. Arthur says:

    Hi Roland
    Thx for this tutorial. I played little bit with the code you provided. And tried to implement Lazyloading for the search in the SuggestingContainer. (Still dont work)
    Can you give me a advise

  5. Roland Krüger Roland Krüger says:

    Sure. Which problems are you experiencing exactly? Can you provide a link to your source code?

  6. Rob says:

    Roland, Thanks for your post. This was exactly what I was looking for even though I am yet to fully get it working. I was initially trying to get the concept to work with an IndexedContainer but didn’t manage that and have reverted to using a BeanItemContainer like you. Although it was doing the filtering it was crashing when I actually selected a value. It was looking for a converter between the Bean and String. Anyhow, just wanted to thank you for the work you put in!! Rob

  7. David Marko says:

    Thanks for such a nice tutorial. I have a one problem with this however. Combo works for item selection and storage (saved in database) but displaying item with already defined value doesnt work for me. It simply display nothing. It mean I can create form item with ComboBox value selected but after save and display again, there is no value in field while value is in database. Any idea what is wrong here?

  8. Manu L says:

    thanks a lot for this tuto, It is very helpful.

    I have (another) one question : Can you specify to wait for the end of user input before doing the search (before calling the filterItems function)?
    In my case, I defined that the filter string has to bee at least 3 characters long, so when I enter a 6 char string, a first search is launched for the 4 first chars and a second for the full 6 chars, so I get latency when running it.
    Example of my logs :
    Filter length:0

    Filter length:4
    Process time: 1234ms
    Filter length:6
    Process time: 101ms

    An useless search has been performed when my inputed string was 4 chars and creates a latency of more than 1s.

    Do you have any recommendations?

    Thanks

  9. Arnie says:

    Hi Roland. Great tutorial. I want to be able to make the autocomplete combobox allow users to enter in their own values without it adding to the list of dropdown result values. Would you have any ideas on how to do this?

  10. KP says:

    Vaadin 7.6.8, getting a No such property exception when calling setItemCaptionMode(ItemCaptionMode.PROPERTY). Suggestions welcome!

  11. Hauglum says:

    Hmm.. the SuggestingContainer is never used so the addFilter will never run.
    (the filter in SuggestingContainer is used but never SuggestingContainer it self)

    Vaadin 7.7.8

Leave a Reply