Copac Beta trial interface

There is now a Beta version of the Copac interface available for trial at http://beta.copac.ac.uk/.

We’re working towards making Copac a more personalised research tool and we’d love to have your feedback on the current range of developments. Try out the Beta version of Copac, explore the new features, then fill in a brief questionnaire to give us your views about the changes so far. In appreciation we’ll put your name in a hat to win a £35 Amazon voucher. The Beta interface will be available for comment until May 29th.

Note: Access to the Beta version of Copac is currently restricted to members of UK academic institutions, you will need to login with your university/college username (or Athens username). The developments will be made more widely available in the future and there will continue to be a free access version of Copac available to everyone.

Copac Beta trial interface

There is now a Beta version of the Copac interface available for trial at http://beta.copac.ac.uk/.

We’re working towards making Copac a more personalised research tool and we’d love to have your feedback on the current range of developments. Try out the Beta version of Copac, explore the new features, then fill in a brief questionnaire to give us your views about the changes so far. In appreciation we’ll put your name in a hat to win a £35 Amazon voucher. The Beta interface will be available for comment until May 29th.

Note: Access to the Beta version of Copac is currently restricted to members of UK academic institutions, you will need to login with your university/college username (or Athens username). The developments will be made more widely available in the future and there will continue to be a free access version of Copac available to everyone.

Copac Beta tweaks

[Jargon alert] I just noticed that the Shibboleth TargetedIds (read anonymised usernames) we are seeing when people log into Copac Beta are much longer than I expected. Some of them may have become truncated when saved in our user database. So, I’ve just increased the field size in the database. This may mean that some people will have lost their search history and references. Sorry about that. But a Beta test version is all about finding out about such niggles.

Thanks for persevering everyone.

[Added 30/3/2009] Tagging in ‘My references’ currently broken. We’re looking into it and hope to have it fixed soon.

[31/3/2009] Tagging issues now look to be fixed. However, when I was looking through the logs I spotted another problem which may reveal the reason some people are having difficulties searching Beta. Investigations are in progress.

[1/4/2009] It looks like some people are unable to gain access to Copac Beta because their Identity Provider isn’t providing us with an anonymised userid, or to use the jargon, a TargetedID. We do need this so we know which is your Search History and References.

There’s not a lot we can do about this. It is up to your Institution to release the TargetedID to us. However, if you are getting a “Login Failed” message please contact us, telling us which Institution you are from and we’ll try hassling your system admins.

New features of the Copac Beta Interface

With the new Copac interface, we wanted to make the Search History and Marked List (now re-named My References) more useful. Previously, these features were session based — that is, if you re-started your web browser, your search history and saved records were lost. For us to be able to retain that data over multiple sessions, we need to know who our users are. Hence, for Copac Beta we are forcing you to login.

The advantage of logging in is that you can use Copac Beta from multiple machines at different times and still have access to the searches and references you saved yesterday or last week – or even last year.  Unfortunately, log-in is currently restricted to members of UK Access Federation institutions (most UK HE and FE institutions, and some companies), but don’t worry – there will always be a free version of Copac open to everyone, and we will be widening the log-in scope in the future.

You can tag your searches and references and use a tag cloud to see those items tagged with a particular tag. We are automatically tagging your saved searches and references with your search terms, and you can remove these automatic tags, and add your own.  These tags are then added to your tag cloud, so that you can easily navigate your saved records through tags which are meaningful to you.  Why would you want to delete the automatically generated tags?  Well, records are tagged with all of your search terms so, if you limit your search to ‘journals and other periodicals’, the tags for records from that set will include ‘journals’ ‘other’ and ‘periodicals’.  If you find these confusing, you can just delete them, and have only tags that have meaning for you.

You can also add notes to any of your references – perhaps to remind yourself that you have ordered the item through inter-library loan, and when you should go and pick it up, or perhaps to make comments about how useful you found the item.  This ‘My References’ section was developed as part of the JISC-funded project Discovery to Delivery at Edina and Mimas (D2D@E&M) as a Reusable Marked List workpackage.

You can also edit the bibliographic details of the item.  These edited details are only visible to you, so you don’t have to worry about making any changes.  You could use this to correct a typo or misspelling in the record, or add details that are not visible in the short record display, such as information about illustrations or pagination.

The search history feature allows you to re-run any previous search with a single click, from any screen.  This could be especially useful for anyone who is doing demos, as not only do you know that the search will return results, but it saves you from the jelly fingers that haunt the even the most proficient of typists when in front of an audience.  The date and time of previous searches are recorded, so that you can see what you have searched for and when.  This could be useful for tracking the progress of a project over time, or showing at a glance what effect refining a search has on the number of results.

Many journal records now contain the latest table-of-contents.  Clicking on an article title will take you through to the Zetoc record for that article, and from there you can use the Open-URL resolver to link directly to full-text (if your institution has access), or order the article through your institution or directly from the British Library.  The table-of-contents allows you to get an idea of the scope of the journal, and whether it will be of interest to you, without going to another website. This makes it easier to avoid wasted travel or unnecessary inter-library loan requests.

We’d love to know what you think of these new features – and any suggestions you might have for new ones!  Once you’ve used the new features, please fill-in our questionnaire, to help us learn what we’re doing right, and what you’d like to see changed.  As thanks for your feedback, there’s a £35 Amazon voucher up for grabs for one lucky respondent.  The survey has 9 questions, and shouldn’t take more than 10 minutes of your time.  Of course, you can always give us additional feedback through the comments on this blog, by emailing copac@mimas.ac.uk, by phone or post, or Twitter.  But we’d really like you to do the survey as well :)

Atom and Shibboleth

The Search History and My References feeatures of the Copac Beta Test Interface are stored in a database with an Atom Publishing Protocol (APP) Interface. The idea is to make the database open to use by other people and services and so enable re-purposing of the data.

Authentication poses a problem. We need to authenticate so that we can identify the user and show them their records and not someone elses. We didn’t want people to have to register to use Copac and neither did we want to get into developing a mechanism to handle user registration, etc. So, we have used the JISC supported UK Federation (aka Shibboleth) Access Management system. This allows users to login to Copac using their own instiutional username. Registering separately with Copac is not needed to gain access.

The downside is that Shibboleth is designed to work with web browsers. I don’t know the technacalities of it all, but a login with Shibboleth seems to involve multiple browser redirects, possibly a WAYF asking “Where are you From?” and a web page with a bunch of Javascript that the browser has to interpret that redirects the browser yet again. I’ve tried accessing the Shibboleth protected version of our APP Interface with some APP client software and none of it could get past the authentication — however, it is very hard to diagnose where the problems are.

I also tried the command line program “curl” to access the APP Interface and while it can handle the redirects and the username and password I think it fails when it gets to the page with the Javascript. Which is fair enough, “curl” isn’t a web browser, it is just a program that retrieves urls.

So, can we make do without Shibboleth? Well we can, but the options are either not terribly insecure or not practical. The options I can think of are:

  1. We put a token (eg a unique id) in the url. This effectively makes the users collection of records and search history public if the url is published.
  2. We put the token in a cookie. This is still insecure and subject to cookie highjacking, but is more private as the token isn’t in the url. Many high profile web sites seem to use such an cookie for authentication, and if they do, then I don’t see why we shouldn’t? However, I’m not sure how practical it is to get third party APP clinet software to send the cookie — unless the APP client was written as part of a web browser that already has the cookie.

You can try accessing the Shbboleth protected APP server for yourself at the following url:

  • https://copac.ac.uk/atom/

If you’ve already used the Copac Beta then your Search History and My References collections can be found at the following urls in the form of Atom feeds:

  • https://copac.ac.uk/atom/saved-searches/
  • https://copac.ac.uk/atom/my-references/

Please let us know how you get on! I’ve tried the above urls with Firefox and Safari. Firefox gets through the authentication and displays the Atom feeds and Service Documents. Safari seems to put itself into an infinite loop whilst trying to display the feed (maybe this is something to do with the XML in our Atom feed?)

We’d be very interested to hear your thoughts on the above.

Copac Beta : new search urls

As the new Copac beta test interface is now storing users’ search history in a database we needed Copac search urls to be stateless (or RESTful.) If you look at the current Copac urls, you will notice as you navigate through a result set, just how much saved state is encoded in the url. There are references to the session ID and the number of your query within your session.

In the new scheme of things, that is all gone and I believe our search urls are now stateless — that is, all the information needed to display a search result is now encoded in the url. The CGI script serving the url does not have to go delving into a database to work out what to do.

I’ll attempt here to explain the new url scheme and hopefully you will see how it can be used as a machine to machine interface to Copac. I should point out though, that this is describing the beta version and things may change in the future.

So, to perform an author query against the Copac database, all you need is a url like this:

http://beta.copac.ac.uk/search?au=sutter

The above url will perform an author search for “sutter” and will display an HTML rendered page showing the first page of brief records. If you would like the results sorted, then you can add a “sort-order” element to the url as follows:

http://beta.copac.ac.uk/search?au=sutter&sort-order=ti

The above url will sort the query by the record title field. If the result set is too large to sort, then you will be redirected back to the same query without the sort-order.

If you want to view the first full record in a result set, then add an “rn” element to the url:

http://beta.copac.ac.uk/search?au=sutter&rn=1

Similarly, to view the second page of brief records:

http://beta.copac.ac.uk/search?au=sutter&page=2

All the above urls return an HTML display — not what you want for machine to machine communication. So, to get some programmer friendly XML you can add the “format” element to the url:

http://beta.copac.ac.uk/search?au=sutter&page=1&format=XML+-+MODS

The above url returns a page of MODS XML records. A page, by default, is 25 records. If you’d prefer more or less records in a page, then you can set the page size by sending a “Page-size” header with the HTTP request. And, so that you know how large the result is, a “Result-set-size” header is returned with the HTTP response when a “format” is specified in the url.

You can, of course, specify a “sort-order” along with a “format”. You’ll be able to discover the various query fields, sort and format options by delving around the user interface and performing a few queries. I’m not going to document them here and now as it is all still beta and they may change before we go live.

Copac unavailability: update

Due to a hardware fault, many Mimas services, including Copac, have been unavailable over the past few days. This fault has now been resolved, and the services have been restored. However, stability is not assured, and further brief outages may occur. We hope that these will be minor, and not greatly impact on the service.

We apologise for any inconvenience caused by this unavailability, and thank you all for your patience.