Loading/updating to Copac: how easy do you find it?

As we have been using the same processes and documentation to handle the loading and updating of libraries for a while, we decided that it was time to ask for some feedback to ensure that we were making the process as easy as possible for the libraries involved.

We asked 9 of the most recently loaded libraries to respond to a short online survey, asking them about their experience of the load and update process, how useful they found the documentation, and whether they had any suggestions for improvement. We did have to emphasise that we were concerned only with the Copac side of the process; unfortunately we can’t do anything about how easy (or otherwise) libraries find it to extract data from their library management systems, although we do recognise this as a valid concern.

The results were very encouraging! Respondents were asked to rate how easy they found the load und update processes, and the vast majority replied that they found them either ‘easy or ‘very easy’, with only one library anticipating that they would find the update process difficult. Documentation was also considered very good, with one library saying that they found it ‘clear and easy to follow’.

It wasn’t all sunshine and flowers, however, as some libraries did comment that they hadn’t realised how long it would take to get the records loaded onto Copac, or how much time it would take them to extract their data. We realise that we need to do a better job of managing expectations here: while we do try to add catalogues as quickly as possible, it can sometimes take time to complete the process, and perhaps we aren’t clear enough about that.

General comments had the Copac staff blushing, as we were told that ‘support has always been excellent’, and ‘we found the process of having our records loaded easy at our end, and thank Copac staff for their help’. One library said that they were ‘just surprised how simple and straightforward the whole procedure turned out to be.’

Responses were kept anonymous, so we can’t tell who exactly we have to thank for all of this wonderful feedback, but we are very grateful for it all :)

If there are any libraries out there who would like to know more, or comment, please get in touch with us! We’d love to hear from existing members of the Copac community who would like to comment, or from libraries who would like to be a part of the community and would like to know more about the (very easy!) technical processes involved.

Institute of Education reload

Last week we started re-loading the Institute of Education Library records. Due to the number of records involved it will take a little while to complete the operation and as of today approximately half of the records are visible in the Copac interfaces. The rest of the records should be available this time next week.

The re-load was required to enable better access to live circulation information from the Institute’s Library Management System.

Re-structuring the database

We are thinking of changing the database we use to run Copac. The current database software we use is very good at what it does, which is free text searching, but it is proving problematical in other areas. For instance, it doesn’t know about Unicode or XML, which was okay some years ago when 7-bit ASCII was the norm, record displays were simple and there was less interest in inter-operating with other people and services. We have managed to shoehorn Unicode and XML into the database, though it doesn’t sit there easily and some pre- and/or post-processing is needed on the records.

The current database software doesn’t cope well with the number and size of records we are throwing at it. For instance, the limit on record size is too small and the number of records we have means the database has to be structured in such a way as makes updating slower than we would like. We’d also like a something with faster searching.

We haven’t decided what the replacement software is going to be, though we have been thinking about how a new Copac database might be structued…

De-duplication

Some people think we do too much de-duplication of our records, others think we do too little. So, we are thinking of having two levels of de-duplication, one at the the FRBR work level and another level of de-duplication broadly based on edition and format. The two levels would be linked in a 1 to n relationship. I.e. a FRBR level record would link to several edition level records. An edition level record would link back to one FRBR level record and also other edition level records which link to the same FRBR record. This would result in a three level hierarchy with the individual library records at the bottom. How this would translate in to a user interface is yet to be decided.

Holdings statements

We currently store local holdings information in with the main bibliographic record. Doing otherwise in a non-relational database would have been troublesome. The plan is to keep the holdings out of the bibliographic records and only pull it in when it is needed.

Updating

This should enable us to reduce the burden of the vast number of updates we have to perform. For instance, we sometimes receive updates from our contributing libraries of over 100,00 records and updates of over a quarter million records is not unknown. Our larger contributors send updates of around twenty thousand records on a weekly basis. We now have over 50 contributing libraries and that adds up to a lot of records every week that we need to push through the system.

Unfortunately for us, many of these updated records probably only have changes to local data and no changes to the bibliographic data. However, the current system means we have to delete it from the database and then add it back in. If a record was part of a de-duplicated set then that delete and add results in the de-duplicated record being rebuilt twice for probably no overall change to the bibliographic details.

So, the plan for a new system is that when a library updates a record we will immediately update our copy that stores the local data and mark for update the FRBR level and edition level records it is a part of. The updating of these de-duplicated record sets will be done off-line or during the small hours when the systems are less busy. If we can determine that an updated record had no changes to the bibliographic data then there would be no need to update the de-duplicated sets at all.

What now?

We think we know how we are going to do all the above and our next step is to produce a mock-up we can use to test our ideas…