jQueryUI and Koha

There’s another project I’ve been working on for almost as long as I haven’t posted to this blog: Incorporating the jQueryUI JavaScript library into Koha. This change would replace the YUI Library which currently powers such interface widgets as buttons, menus, and autocomplete.

When I helped develop the new templates for Koha 3.0 I evaluated several different options for creating some application-like interactions in the Koha staff client including YUI , Ext JS, and similar jQuery plugins. The goal was to have options for creating cross-platform, cross-browser interface widgets. After much testing I settled on YUI. Though I found its syntax difficult to work with, the results worked well and seemed robust. The most visible example of YUI is in the staff client interface, where “toolbars” appear on most pages for displaying actions:

 YUI also drives some autocomplete form fields in the staff client and menus in the OPAC.

In addition to YUI, Koha uses the jQuery JavaScript library and some jQuery plugins like Tablesorter and a pre-jQueryUI version of  Tabs. Since the release of Koha 3.0 over three years ago a lot has changed with these projects. The YUI library released version 3 with enough syntax changes to make an upgrade mean totally rewriting our existing YUI code. At the same time the jQueryUI library made great advances, adding a lot of the same functionality for which we depended on the YUI library.

In November 2010 BibLibre founder and CEO Paul Poulain proposed getting rid of YUI in favor of jQueryUI and the response on the Koha developer’s list was positive. Shortly after I filed a bug, Replace YUI JS libraries with Jquery UI and began working on a branch to do just that.

Since then I’ve gotten two major components migrated from YUI to jQueryUI: tabs and autocomplete. Here’s an example of jQueryUI tabs in the OPAC:

Here are the search tabs in the staff client:

And here’s autocomplete in action in the staff client:

As part of this project I have also replaced the calendar widget we have been using with Koha. Since that widget was added to Koha it stopped being open source, so it makes sense to abandon it in favor of functionality built into jQueryUI.

Calendar pop-up on the check-out screen

Also on my list of things to implement is a replacement of the modal window system used in some places in Koha, for which we’re currently using a standalone JS plugin. This can be eliminated in favor of the jQueryUI-native Dialog widget.

Unfortunately we are still waiting for one element for which we depend on YUI which jQueryUI lacks: A menu widget. Their menu widget has been in alpha stage for almost as long as I’ve been working on this jQueryUI branch. Without it we can’t reproduce the toolbars we build now in YUI. For this reason my work has been stalled for a while. I’m keeping my branch up to date and in sync with the master branch, and I’m keeping my eye on the status of the menu widget.

I’m frustrated with the wait, but I’m also excited about the improvements we’ll be able to make to the Koha interface once jQueryUI is our standard go-to library.

An update on Recent Comments

It’s been far too long since my last post. I’m happy to say that my Recent Comments feature didn’t take quite so long to get incorporated into Koha, although it wasn’t until recently that it was obvious it was there. The page itself made it into Koha in a commit in December 2010, and was in Koha 3.2.3. What it lacked was a link in the interface pointing to the page, so you had to know where it was to get to it. This problem was fixed in a commit in October of this year which adds a new system preference, , which controls whether a link to the recent comments view should appear in the OPAC’s search bar. The option will be available to users of Koha 3.6.

A long an roundabout path, but hopefully a useful feature.

Proposed feature: A recent comments view in the OPAC

Koha’s OPAC has a feature which lets logged-in users leave comments about records in your catalog. This feature was originally branded as “Reviews,” but was changed to “Comments” in order to align it with the familiar feature allowing users to comment on content in blogs, on YouTube, etc. If you haven’t seen it before, here’s what it looks like to a logged-in user:

A catalog record with comments

If you visit the Athens County Public Library catalog you’ll find that comments are enabled, but few users have taken advantage of the feature. I’ve often wondered what we could do to encourage people to add their comments, and one of the ideas I had was to feature recent comments–perhaps on the OPAC or library web site home page.

I’m not an experienced Perl programmer. Most of the changes I make to Koha’s Perl code are minor or cut-and-paste. Adding a feature as unformed as a recent comments view was a little daunting, but I think it turned out well, and I hope I did it right! I’m quite pleased with the result:

Recent comments

The page shows the latest comments added to any title in the catalog in descending order by date. The page also offers an RSS feed of recent comments. This offers two benefits: First, patron can subscribe to recent comments if they want to find out what others are talking about; and second, the library can use that feed to pull content into their own web site. If my library site’s content management system can process an RSS feed, I can add those recent comments on my library’s web site too.

Of course it’s too late in the development cycle for this feature to make it into 3.2, but I hope it will arrive with 3.4.

A collaborative process

There’s one more thing I wanted to share about the development of this little feature: When I was creating the RSS feed I was testing it with the W3C’s feed validator to make sure I had constructed the markup for the feed correctly. The only error I didn’t know how to fix was one with date-formatting. The validator told me I needed to  format my date according to RFC 822, but Koha didn’t include a way to format dates that way. I put out a plea on the Koha IRC channel which was answered by Chris Nighswonger. He very generously added the code I needed and I was able to include a valid RSS feed.

This anecdote shows the very best kind of interaction possible when you work with an open community of developers. I’m not suggesting that in the Koha world all one has to do is ask and free code will be provided. But sometimes you catch someone at the right moment, or the challenge appeals to them, and they pitch in. Many thanks to Chris for doing so when I needed help.

Showing item type counts on the checkout screen

Does your library limit your patrons to a certain number of checkouts for certain item types? At the Athens County Public Libraries, for instance, we limit patrons to 10 audio books, 10 music CDs, 10 videos, and 5 DVDs at a time. If a patron tries to check out more than 5 DVDs, Koha will show a warning. But what if you want to be able to tell at a glance how many a patron has?

This functionality was available to us in our 2.x installation, but when we upgraded to 3.0 our support company at the time told us it wasn’t a customization they would support. This wasn’t a feature we were willing to give up, so I set out to duplicate it using the tools available to me: system preferences and JavaScript.

Information there for the taking

All the information we need can be found on the patron checkout screen, we just need to figure out how to get it. The page lists all the items checked out to the patron, and it shows the item type for each:

List of items checked out on the checkout screen

With this available to us we can use jQuery to count each instance of each item type. We need to build a count for each item type in our system, so the script isn’t very portable. It looks for table cells (“<td>“) containing the description of each of our item types:


var itypes = {'circ': 0, 'avid': 0, 'avbk': 0, 'avmu': 0, 'advd':0 };
$("#issuest td:contains('Circulating')").each(function(){
itypes["circ"]++;
});
$("#issuest td:contains('Videos')").each(function(){
itypes["avid"]++;
});
$("#issuest td:contains('DVDs')").each(function(){
itypes["advd"]++;
});
$("#issuest td:contains('Audio Books')").each(function(){
itypes["avbk"]++;
});
$("#issuest td:contains('Music CDs')").each(function(){
itypes["avmu"]++;
});

The script starts by setting up an array of all my item types (“circ,” “avid,” etc.) and giving each a value of zero. Then the script looks for instances of each item type description on the page,  “Circulating,” “Videos,” etc., using jQuery’s :contains selector. Each time it finds an instance of one of those text strings the script increments the count for that item type. At the end of the process the script will have the count for each item type.

Displaying the counts on the page

In order to show the item type counts on the page we need to lay some groundwork by adding some markup. I want to add the count information right after the “Checking out to…” heading, so I’ll find that element’s ID using FireBug and jQuery’s after() function:


$("#circ_circulation_issue label[for='barcode']").after( ... );

The HTML I’m going to add is the default state, so it shows a zero count for everything:


<p style="margin-top:1em">
<span id="avmuout">0</span> Music CDs out, <span id="avmuok">10</span> More Allowed
</p>
<p>
<span id="avbkout">0</span> Audio Books out, <span id="avbkok">10</span> More Allowed
</p>
<p>
<span id="avidout">0</span> Videos out, <span id="avidok">10</span> More Allowed
</p>
<p style="margin-bottom:1em">
<span id="advdout">0</span> DVDs out, <span id="advdok">5</span> More Allowed
</p>

I’ve  included unique IDs for the “count” spans so that I can easily update them with my script:


$("#avidout").html(String(itypes["avid"]));
$("#advdout").html(String(itypes["advd"]));
$("#avbkout").html(String(itypes["avbk"]));
$("#avmuout").html(String(itypes["avmu"]));

$("#avidok").html( 10 - itypes["avid"] );
$("#advdok").html( 5 - itypes["advd"] );
$("#avbkok").html( 10 - itypes["avbk"] );
$("#avmuok").html( 10 - itypes["avmu"] );

In the first of the two sections above I take the count I got earlier, itypes['avid'] and set the content of the corresponding <span> using the html() function. I also want to show how many more the patron can check out, so I subtract the count from the limits I’ve set in my Koha installation.

If you have patrons who have exceeded their checkout limit you’ll see a problem: The page will tell you they’re allowed to check out a negative number more items. We can correct the script to accommodate:


$("#avidok").html((10-itypes["avid"] &gt; 0) ? 10-itypes["avid"] : 0);
$("#advdok").html((5-itypes["advd"] &gt; 0) ? 5-itypes["advd"] : 0);
$("#avbkok").html((10-itypes["avbk"] &gt; 0) ? 10-itypes["avbk"] : 0);
$("#avmuok").html((10-itypes["avmu"] &gt; 0) ? 10-itypes["avmu"] : 0);

Final version

Here’s what the results look like:

The final version includes proper escaping of the HTML content and wraps the whole process into a function (“itemTypeCount”). This function will be called on page load only if jQuery finds that the table of checkouts, which has an ID “issuest” is being displayed. The whole script goes into Koha’s intranetuserjs system preference.


function itemTypeCount(){
$("#circ_circulation_issue label[for='barcode']").after("<p style="margin-top:1em" class="icount"><span id="avmuout">0</span> Music CDs out, <span id="avmuok">10</span> More Allowed</p> <p class="icount"><span id="avbkout">0</span> Audio Books out, <span id="avbkok">10</span> More Allowed</p> <p class="icount"><span id="avidout">0</span> Videos out, <span id="avidok">10</span> More Allowed</p> <p style="margin-bottom:1em" class="icount"><span id="advdout">0</span> DVDs out, <span id="advdok">5</span> More Allowed</p>");

var itypes = {'circ': 0, 'avid': 0, 'avbk': 0, 'avmu': 0, 'advd':0 };
$("#issuest td:contains('Circulating')").each(function(){
itypes["circ"]++;
});
$("#issuest td:contains('Videos')").each(function(){
itypes["avid"]++;
});
$("#issuest td:contains('DVD')").each(function(){
itypes["advd"]++;
});
$("#issuest td:contains('Audio Books')").each(function(){
itypes["avbk"]++;
});
$("#issuest td:contains('Music CDs')").each(function(){
itypes["avmu"]++;
});
$("#avidout").html(String(itypes["avid"]));
$("#advdout").html(String(itypes["advd"]));
$("#avbkout").html(String(itypes["avbk"]));
$("#avmuout").html(String(itypes["avmu"]));
$("#avidok").html((10-itypes["avid"] &gt; 0) ? 10-itypes["avid"] : 0);
$("#advdok").html((5-itypes["advd"] &gt; 0) ? 5-itypes["advd"] : 0);
$("#avbkok").html((10-itypes["avbk"] &gt; 0) ? 10-itypes["avbk"] : 0);
$("#avmuok").html((10-itypes["avmu"] &gt; 0) ? 10-itypes["avmu"] : 0);
}
$(document).ready(function(){
if(document.getElementById("issuest")){
itemTypeCount();
}
});

Caveats

This system works very well for my library, but it comes with a few caveats:

It requires that you hard-code, in the script, handling for each of your Koha item types.

Besides being tedious, it also requires that you modify the script each time you change your item types.

It requires that you hard-code the correct item type limits.

Also tedious, and requires that you modify the script each time you change your circulation rules.

It creates a potential collision with both call numbers and titles.

If my item type description is “DVD” and my call number includes the text “DVD” as well I’ll get an inaccurate count. If my item type description is “Audio Books” and a patron has checked out a print book entitled Audio Books for long trips I’ll get an inaccurate count.

For us the disadvantages are not unwieldy and the collision problem has never caused a problem. The advantage we get is being able to tell at a glance whether the patron is going to be able to check out that stack of DVDs or whether we need to ask them to put some back. Better to ask them to pick their favorites up front rather than after we’ve already checked out some of them.

Developing with branches

There was recently some discussion on the Koha mailing list about how “branches” fit into the development process. I replied with a description of how I use branches in my development process, and this post is an expansion of that.

In my discussion of Koha’s bug-reporting process I mentioned that a developer who wants to work on a bug will “accepts” a bug in Bugzilla before beginning work on it. There is a lot more to the developer’s workflow going on behind the scenes. I’ll describe what my process is.

When I first begin to work on fixing a bug, I create a “branch” in the git version control system. When I do so git creates a separate working copy of the main branch, “master.”  This copy is of the very latest version of Koha available. I create a separate branch for each distinct change I want to make to Koha. That change might be defined as a bug fix for a particular bug, or a new feature. The changes to each branch might include modifications to many different files, but the changes should all be related to that one subject.

It’s vitally important that I keep my branches clean and separate from each other because the Release Manager is relying on me to make it easy for him to apply and test my patches. If I submit a patch that covers more than one “subject,” that makes it all the more difficult to test it properly. If I introduce a bug to one aspect of my update, the whole patch may have to be rejected. Keeping things as simple as possible makes it easier for everyone.

To see how this looks in my workflow, here’s a selection of my current working branches:

ip-Bug-3125-IE-selects-2010-05-18
ip-Bug-4173-restricted-status-2010-03-18
ip-Bug-4901-Zotero-subtitles-2010-07-09
ip-hold-interface-test-2010-07-19
* master
ps-Bug-2307-calendar-i8n-2010-07-13
ps-Bug-2704-440-Display-2010-07-13
ps-Bug-2981-alternate-solution-2010-05-18
ps-Bug-3459-topissues-2010-06-29

When I first sit down to work I create a branch with a prefix “ip-” for “in progress,” give it a short title and bug number if I have one, and add the date I started working on it. You can see I might have quite a view branches in progress at a time. This might be because I’m working on a big update which is time-consuming, or it might mean I’m stumped about what the solution is.

Each time I sit down to do more work on a bug I switch to that branch, download the latest updates which have been submitted and approved on the master branch, and tell git to “rebase” my branch. The rebase process merges my changes with the latest updates. If there are any problems with the merge git will warn me and ask me to make manual changes. Usually the process is automatic. I do this every time I sit down to work on an in-progress branch. This ensures that the changes I’m making are compatible with the latest version of Koha.

Once I have finished my work I check it and test it as carefully as I can. I make sure I’ve rebased against the very latest update in git, and I submit a patch. Submitting a patch takes the changes I’ve made and distills them down to a single text file which can be “applied” by the Release Manager or any other Koha developer to their own git installations. I also attach a copy of my patch to the bug report, if there is one, as I described in my bug reporting post. This is another way to get the patch out into the open where others can review it and test it.

After I’ve submitted a patch I rename the branch I was working on with “ps-” for “patch sent” and keep the branch until my patch is approved (fingers-crossed).

All this is on my computer in a virtual machine, so in my case I’m not being open as I might. If I had the resources to work on a real server, or if I kept my work on a service like GitHub I could share all these branches with everyone, from the time I first started working on it to the time my patch was submitted.

If some time has passed and I see that my patch from one of my “patch sent” branches hasn’t been approved I’ll switch to that branch and rebase: I’ll tell git to grab the most recent Koha commits and merge them with the changes in my patch. If not too much time has passed, or the files I changed haven’t been worked on by others the rebase will be successful and I’ll know I’m still on track. If not I might have to manually make changes to my original commit so that it fits in again with the work others have been doing. If manually merging was required, that tells me the Release Manager would have to do the same. I should consider resubmitting a revised version of my patch which can apply cleanly to the latest version.

Keeping all my “patch sent” branches intact until my patch has been approved is important to my workflow because it’s the easiest way for me to keep track of what has been approved and what hasn’t. I can see at a glance which patches of mine have not been approved. If it’s been a while, I might try to find out why not. If there was a problem with the patch I might be able to revise it to the RM’s satisfaction.

I can’t submit a patch and expect it to automatically work a month from now, a week from now, or even a day from now. I’m responsible for keeping up.

I’m not a git expert, and I’ve had lots of help and advice along the way to get me to this workflow. It works really well for me, and I hope it works as well from the point of view of our Release Manager and fellow developers.