How usage statistics can inform national negotiations and strategies

The paper describes a recent Joint Information Systems Committee (JISC) funded project, the aim of which was to analyse library usage data from selected e-journal deals negotiated by the JISC within the NESLi2 initiative. The project began in May 2004 and examined usage data from four publishers and 17 libraries. The aims, methodology, conclusions and recommendations of the project are included. Owing to confidentiality agreements with publishers and libraries, it is possible to share only a limited amount of usage data.

The principal aim of the study was to provide the JISC Journals Working Group (JWG) with accurate and up-to-date data on the national use of e-journal 'big deals' negotiated by the JISC within the NESLi2 initiative.The study was intended primarily to inform the JISC JWG in future national e-journal negotiations; a further benefit was the development of guidelines and recommendations which could assist individual libraries in assessing the value for money of their e-journal deals.
The project team collected COUNTERcompliant usage data from four publishers and a representative sample of 17 large, medium and small academic libraries for the whole of 2003 and the first six months of 2004.It should be noted that not all of the 17 libraries took all four of the publisher deals.The aim was to present an account of how journals are being used within institutions and how that usage relates to costs, institutional profile and subject coverage.It also considered how usage data shows the extent of use of all titles within a 'big deal' including those titles to which the library had not previously held a subscription in the time before the 'big deal' when only print subscriptions to selected journals were held.
The project was led by staff in the 'evidence base' research group at the University of Central England, with input from staff at Cranfield and Florida State Universities, and the Director of COUNTER.

Methodology
The initial set of COUNTER-compliant usage data was provided by both libraries and publishers; it was reassuring to note that there was no difference between the data provided from these two sources.The full-text article request (COUNTER Journal Report 1) data was used as the unit of measure.
The study analysed data for January-June 2004, using comparative data from 2003 when available.
Subscription cost data was acquired from libraries.The collection of this data proved extremely time-consuming as ensuring accuracy was highly important to the project outcomes.Most libraries were able to provide the e-access costs of their NESLi2 deals, but providing the cost of their print subscriptions was rather more difficult, often because print subscriptions were dealt with in a different section of the library.In order to examine the use of previously subscribed and unsubscribed journals (as defined above) it was also necessary to collect data from both libraries and publishers.Interestingly, this data from publishers and libraries (unlike the COUNTER usage data) did not match, and the project team found that the lists

Based on a paper presented by Simon Bevan at the 28th UKSG Conference, Edinburgh, April 2005
The paper describes a recent Joint Information Systems Committee (JISC) funded project, the aim of which was to analyse library usage data from selected e-journal deals negotiated by the JISC within the NESLi2 initiative.The project began in May 2004 and examined usage data from four publishers and 17 libraries.The aims, methodology, conclusions and recommendations of the project are included.Owing to confidentiality agreements with publishers and libraries, it is possible to share only a limited amount of usage data.

PETE DALTON and ANGELA CONYERS evidence base, University of Central England
differed -quite substantially in some cases.Libraries had to spend some considerable time reconciling their lists with publishers.
Owing to the original project timetable -which was established by the JISC JWG -it was only possible to examine 18 months' worth of usage data: January-December 2003 and January-June 2004.This data was broken down into three sixmonth periods.(Clearly there are shortcomings here as it is well established that demand in libraries fluctuates throughout the academic year.)The project team have now been asked by the JISC to augment the 18 months of usage data with the fourth period of six months (July-December 2004) thereby giving two full calendar years of data.
An important part of the study was to undertake a qualitative study to augment the data itself, so a number of case studies were also undertaken with a subset of the participating libraries.

Increase in use
Large amounts of data were collected to cover the original 18 months of the project.With only a few exceptions, libraries in the study had subscribed to the particular big deals in both 2003 and 2004.Figure 1 below shows the total increase in use for all publishers and all libraries.
The graph shows a 42% rise in use over 18 months, from 1.5 million to 2.2 million full-text downloads.The addition of usage data for July-December 2004 will show whether this rise will continue over a longer period.

Cost of journals in the big deals
In order to make assumptions about the relative value for money of the big deals being examined, it was necessary to establish what the cost of purchasing the journals within the deal would be if they had to be purchased separately.
Figure 2 shows the list prices of all titles across all publishers in the study.
The project broke down subscriptions into cost ranges: low (under £200), medium (£200-£399), high (£400-£999) and very high (£1,000 or over).There were 872 titles (28%) priced in the very high range and 530 titles (17%) priced in the low range.Unsurprisingly, the largest number of titles, 1714, were priced between £200 and £999.It should be noted that this data excludes 19% of titles for which no cost data was available because they had, for example, ceased publication, merged with other titles, or transferred to another publisher.

Subscribed and unsubscribed titles
Figure 3 examines usage of one publisher's deal for both subscribed and unsubscribed titles.Subscribed titles are those which the library held prior to the 'big deal' and to which subscriptions have to be maintained as a condition of the deal.Unsubscribed titles are those additional titles included in the deal to which the library did not previously hold a subscription.Data illustrated compares the average use of subscribed and unsubscribed titles and shows the smallest institution at the top and largest institution at the bottom.Overall, average usage of subscribed titles for this publisher per library is 99 full-text requests.Overall, average Clearly the average use of subscribed titles is consistently higher across all libraries than the average use of unsubscribed titles.This suggests that the collection development decisions made by librarians concerning individual journal titles before the 'big deal' were, on the whole, reasonably well informed and that other titles now included in the 'big deal' were not core titles for the individual libraries.

Nil usage
Generally an issue of interest (and sometimes concern) to librarians when looking at the 'big deal' is the number or percentage of titles within a deal that are not used.Figure 4 shows the amount of nil usage for one publisher and shows a range between 7% and 40% of unused titles.The general pattern is that nil usage is higher for smaller institutions.This is almost certainly explained by the fact that smaller institutions have lower staff and student numbers and, in most cases, are organizations which focus Example ranges of nil usage from two of the other publishers in the study showed from 4% to 29% and 13% to 73% of titles unused.It should be noted that nil use titles include those titles which have, for example, ceased publication, changed title, merged or transferred to another publisher.The study also showed that nil usage titles were predominantly both unsubscribed and low cost titles.

High usage
High usage was defined as titles with 100 full-text downloads or more.The graph for high usage (Figure 5) shows data from one publisher, demonstrating that the larger research-intensive institutions make higher use of journals.The high use range for this publisher is from 0% (where no titles have been used over 100 times) to 19%.Ranges for other publishers were from 4% to 42% and from 2% to 29%.The reasons for high usage of journals are the flip side of the reasons for nil usage.Larger  Figure 6 shows cost per use for all titles in a deal -both subscribed and unsubscribed -and this is calculated using the full NESLi2 subscription cost as well as the e-access fee.The cost ranges from 30p to £1.60.Other publishers had ranges from 58p to £3.80, from 54p to £3.31 and from £1.35 to £5.28.Cost per use is dependent on two factors -size of institution (amount paid, number of users), and the amount of usage.In this example, the largest institution pays 74p per full-text download (high use, high cost subscription) and the smallest institution pays 30p (lower use but lower cost).

Cost per use
When the subscribed and unsubscribed titles are separated out, the cost per use figures are radically altered.
Figure 7 shows cost per use for titles not subscribed by the Library.This figure is calculated using the e-access fee alone and, with a range from 9p to 79p, it shows that in terms of cost and usage, Overall the cost of an unsubscribed request is very low, rarely more than £1.Where there are higher costs, this is related to low use of the particular deal in relation to price paid.
Figure 8 shows cost per use for subscribed titles.This figure is calculated using the full subscription cost, less the e-access fee.Here the range is from 93p to £5.00 per full-text download, while for other publishers the ranges were £1.95 to £7.50, 49p to £2.94 and £2.01 to £27.59.

Yield
The study team came up with the concept of yield per title.This is defined as the value of unsubscribed titles available for £1 of e-access fee.It is calculated by dividing the list prices of all unsubscribed titles by the e-access fee.So in the example in Figure 9, for the largest institution, £1 of e-access fee buys £10.42 of unsubscribed titles.In this example there is a fairly good correlation between yield and size of institution -the smaller the institution, the higher the yield.

Data conclusions
Looking at all of the data it is easy to draw some fairly obvious conclusions: larger institutions have higher numbers of subscribed titles; the smaller the institution, the higher the percentage of unused titles; the number of full-text downloads in all deals increased by 42% between January-June 2003 and January-June 2004; for most libraries in the study, the average cost per download was low (in relation to an ILL) and considerably lower than pay-perview on the current models; a comparatively small percentage of titles generated high usage.
Generally, titles receiving little or no use were either unpriced (i.e. had ceased publication or changed publisher) or from the low price range, although some institutions did have higher numbers of little or no use titles in the higher price ranges which, for some case study examples, caused concern.High use titles were generally in the high and very high cost range.

Case studies
Case study interviews were undertaken after libraries had received their own usage data from the project to allow them to make informed comments on issues of local use of e-journals.
In response to a request for comments on why libraries had taken up NESLi2 deals, librarians mentioned such issues as cost, coverage, ease with which collections could be built up, critical mass and ease of use (on and off campus).
Librarians did make some critical comments on 'big deals' -that deals were becoming more difficult to justify, price caps were above library budget increases, and that it was becoming more difficult to withdraw from deals.However, the general response to the usage data presented to them by the study team was that the deals currently provide value for money.
Case study libraries were questioned about their approach to e-only access.One of the main reasons libraries continued to take print journals was because of the VAT on e-only subscriptions, although some libraries wished to continue with print for other reasons, such as reluctance of users to change.
There were a number of contextual factors affecting use and case study libraries provided examples of these, such as peaks in usage when students are undertaking dissertations; changes in the way journals are linked (e.g.SFX); the fact that individual titles are added to library catalogues; promotional activities; initiatives to restrict access to print journals; the fact that humanities subjects are considered traditionally to make less use of e-journals; users getting more used to accessing titles online; and the development of a larger critical mass of electronic titles.
One of the key issues that emerged from the case studies was how complex many deals are to understand, both in terms of content provided and pricing.Some specific issues were brought up, notably the desire for more information when signing letters of commitment, a desire for clearer explanations of pricing and a concern over long contract lock-ins.There was general dissatisfaction from all case study libraries over the lack of clarity, and a desire for more flexibility in relation to cancellation policies.It was interesting that one library had been forced to limit the number of NESLi2 agreements they signed solely because of cancellation policies.
Comments on the role of the NESLi2 negotiating team were positive -communication was felt to be good, and the team were seen to be working in the best interests of libraries.There were a couple of critical comments specifying a desire for speedier conclusions to deals.payment to publishers on behalf of all UK libraries participating in a particular NESLi2 deal.
Note: Once the set-up of the new JISC company is complete, the JISC will be able to negotiate contracts (something that they cannot currently do) and it is hoped that this will look at how payment terms and methods could be eased.3. The JISC should consider setting up a portal site for NESLi2 publishers to deposit their national NESLi2 COUNTER-compliant usage statistics.
The idea of a portal site for usage stats is to be taken forward and is currently being considered further.4. The JISC should examine the possibility of funding the creation and ongoing support of a usage statistics toolkit for librarians to assist in the management and analysis of pricing and usage data.In terms of toolkits for usage data, there is interest here from both libraries and commercial companies to take this further.5.The JWG to recommend to COUNTER that the

Figure 1 .
Figure 1.Total increase in usage (all libraries, all publishers)

Figure 2 .
Figure 2. Cost range of journal subscriptions (all titles, all publishers)

Figures 6 -
Figures 6-8 show different aspects of cost per use for one publisher.The data covers six months from January-June 2004 and so figures are based on 50% of 2004 subscription cost.Figure6shows cost per use for all titles in a deal -both subscribed and unsubscribed -and this is calculated using the full NESLi2 subscription cost as well as the e-access fee.The cost ranges from

Figure 6 .
Figure 6.Cost per use (subscribed and unsubscribed titles)