Discovery is not the problem that needs to be solved

I admire Eric Lease Morgan's ability to distill ideas into words. In his latest musing, he writes about Next-Generation catalogs and begins with asking this key question: To what degree is it an inventory list or a finding aide?

Next generation catalogues are also called discovery layers and on this matter, I would like to quote my favourite passage of the piece

"Discovery" is not the problem that needs to be solved. People can find more information than they have ever been able to find before. We are still drinking from the proverbial fire hose. What is needed are tools to enable people to use, to integrate, to exploit, to understand the things they find.

Research is not about finding everything but

“Excuse me, but research isn’t about finding everything. That’s not really how it works at all … this idea of a single search that brings back every relevant item is not like the process of research as it is actually practiced. Research involves finding a few, or even one, good thing, then checking what that one thing references, and checking those sources … then branching out rapidly from there. Literature is not made up of a colection of items connected by having common keywords … a literature is composed of a collection of items connected by common ideas and a community of thinkers who are influenced by each other’s works. It’s about following the network of citations, really.”

Crowd : (nodding)

Me: (now all excited) ”So, what search should do is get your foot in the door, not hand you “everything” … Google works because it gets you some stuff, not everything..."

The above is from a post entitled Search and Research from Doug's Mind. I think his epiphany is worth expanding upon.

What if libraries tried to create a something that brought you some of the best starting points to research?

The forcast is getting more cloudy all the time

Lots of different thoughts flooded my brain as I watched the introduction video to Ubiquity for Firefox. This video should be required viewing for librarians because its a proof of concept that illustrates what 'web 2.0' or 'cloud computing' can be: the user makes use of a website's services without even visiting that site.

The obvious question is how do we integrate the local library into this new ecology? I've been mulling it over and on first glance, I think that answer might be that we don't because we can't. I came to this conclusion after trying to envision a user trying to add a book or an article that they wanted to recommend to a friend. The most likely scenario is that this is an item in that user's own collection. To this end, I suspect that the most natural fit for this space is a web-based citation service such as Zotero 2.0 or newly released Mendeley. LibraryThing could work for books.

If we do want libraries to provide this service to our users (and I do), I think at a minimum, our library catalogue has to provide a means for a user to create their own virtual collections of books that include items from their library and from their own shelves . From my understanding, Bibliocommons is the only library catalogue service that can do this.

Because Ideas Matter - Finding Special Collections on the web

Some years ago MPOW made the painful but necessary decision to remove the bibliographies that were are all grouped together in the Reference Collection and to re-catalogue each one so that each bibliography could be found with the subject it was about. This meant that bibliographies on Shakespeare could be found with the other Shakespeare books which would make it more likely to be stumbled upon and more likely to save the user a trip to another set of stacks in the library.

I bring this up because this project came to mind when I was doing some follow up research on a book by and about Jane Jacobs. The book contains material from the Jane Jacobs Papers that are located, not in New York City or Toronto as one would first presume, but in Boston College. Now those papers are from her life from 1916 to 1995. Jane Jacobs passed away in 2006. Where are the papers from her last decade? That's not a rhetorical question - I haven't found out where they are yet.

If the future of academic libraries means currating our unique collections, we have to start doing a better job of letting the world know what we have to offer. This means recognizing that most people search for a subject - not for a bibliography or a directory of library special collections. But how can we achieve that? How can we nestle information about the unique and rich collections in our libraries right beside the subjects that they are about in the online world?

There is a way. Thank you Wikipedia.

Now you go to it! Add your library's collections to The Free Encyclopedia right now!

Using Google and FriendFeed to Create a Collaborative Research Environment

Some months ago, I suggested that one way to measure of a success of a library's discovery system is to compare it to Google. I've been thinking about extending this measuring stick to the rest of the support that libraries provide to their researchers. So, I created a Google account for a fictional Cloud Researcher.

After creating a Gmail account for "Cloud Researcher" I started a Google Reader account for her. It was shockingly easy to add an RSS feed of the most recent issues to the more well known journals such as Nature and Science. Now my researcher can stay up to date in her field.



Google Reader makes it easy to share items by allowing instant publishing to a clippings blog. (This feature will prove important later on)



I then created a customized Google search engine so whenever my researcher needs to refer or research, she can just search this index of just the journals and websites that she trusts. If she wants to cast a wider net, she can always opt to search with Google Scholar or even plain old vanilla Google.



If her affiliated academic library provides authentication by IP address, then our researcher can keep up with her journal reading on campus without ever visiting the library's website even once - even if she's off-campus by using Windows XP Remote Desktop option to access her work computer. And if she has a loving, caring library, she can authenticate and access research using LibX, which also comes in handy when she uses Google Books.

I'm glad I actually went through the exercise of actualizing this example because in doing so, I had a small epiphany which was this: the future of learning online space is going to be an aggregation of online services. I realized this when I started playing with FriendFeed.

I've been using FriendFeed for about a week now and I still find it a strange beast. The way I would describe it is FriendFeed = Twitter + RSS reader. I think FF is an improvement on Twitter because it allows others to comment directly to a post. But FF is not a conventional RSS reader like Bloglines or Google Reader as the emphasis is on following friends - not publications. But what you can do with FriendFeed is create different 'rooms' for your different circle of friends. I created a room called The Cloud Research Lab Room.



What would happen if FriendFeed could share structured bibliographic citations from Zotero?

I think it could be something wonderful.

Search is a combination what you know and who you know

In my first years of academic librarianship, it struck me as really odd that I would have faculty call me up and ask me to teach their undergraduate students how to use the library indexes. Some of them would even confess that they really didn't know how to use these tools themselves.

In time, I came to realize two crucial points. First, faculty keep up with their research by regularly reading their personal journal subscriptions and keeping track of what the most important researchers in their field are doing. When they do use library-provided indexes, they tend to search by author and by following citations. Google Scholar recognizes this; when you search for Hamlet you get an article written by D. Hamlet as the first result returned.

The second point is that the process of learning who the key figures are in a particular field is part of the transformation from beginner to expert. The trouble is, our existing indexes are designed for librarians - not for students who are making this transition from undergraduate to graduate.

Rarely are students told that research is a reiterative process and that it is often necessary to perform a number of literature searches for a paper as one discovers gaps in coverage or realizes that there is something worth dwelling into more detail. But as seasoned librarians and educators know, even if you tell them this hard won nugget of wisdom, most students will still just grab the first set of articles that match their topic (which can be quite random especially if the database sorts results by date) and then try to mash these citations into their paper, often the night before its due.

So then how can we point these students to the best articles on a particular topic ?

One solution is to point beginning researches to sources of review articles. At the reference desk, I try to check Annual Reviews whenever I get a student who can't narrow down their research question to anything beyond their topic. You know the ones: "I need articles on anorexia." Another solution would be for us to take matters in our own hands. Here's my idea: we should buy the ten most popular textbooks for undergraduates in a particular domain, like Biology. Then we should make note of all the citations in the texts and add them into a searchable database.

These two examples still rely on the work of editors and experts to select the research that made a difference and to put such work in context. TopCited tools from Scopus and Web of Knowledge have been developed to automatically quantify the value of articles and researchers by the rate and number of times papers have been cited.

I believe that the adoption of social networking software into the academic sphere will eventually provide another means of establishing who are the experts in a particular field and what pieces of writing and research are the ones that have made the most impact. It very well might replace the traditional way we search and research and keep up with our fields.

Reading blogs is a case in point. You read library blogs on a regular basis rather than searching for the word 'library' in Google on a regular basis, don't you?

Boolean NOT

I have been thinking about social software and libraries and I suspect that the result of this thinking might be a rash of posts not unlike a bunch I wrote when I couldn't stop thinking about Wikipedia.

But before I can write about how I think social software may affect search and research, I feel its necessary to clear some cognitive space first. So before proceeding, I want to make something very clear: LIBRARIANS HAVE TO STOP TEACHING BOOLEAN SEARCHING.

I know this is scary stuff for many academic librarians who I suspect would be at a loss of what to teach if you took away their ((ANDs) and (ORs)). But teaching about Boolean logic has become a crutch and its time to throw it away. I would go so far to say that banning the word Boolean is the number one way to improve information literacy practice in libraries.

You see, most people don't use Boolean searching. And they are still healthy, happy, and successful people. Sure, they may not be the most searching in the most efficient way but their searching is still effective. Users want to apply their energies to their results and not to search grammar.



It is far more efficient to create searching interfaces that try to address these behaviour patterns than try to educate everyone of the masses on "the right way" to search.

We are at a state where the bibliographic databases have become so large that even some of the worst, poorly constructed search terms now bring relevent hits among the debris. Unfortunately, one response I've seen first hand is a library assignment in which the student is forced to use boolean logic by requiring that only 10 hits or less be returned from a search string. Any research assignment does not resemble how a real person conducts research is a poor one, to say the least.

Instead teaching that AND means 'and' and 'OR' means 'or', we should try to teach something beyond the mechanics of search. Instead of teaching about Boolean, we should instruct on the importance of language, on the nature of the publication cycle, and of the research process.

And it goes without saying that we have to get the word Boolean out of our library catalogues.

Ontario Scholars Portal – Yours to a Discovery Layer

(This is a little something I wrote to support this work)

What is a “Discovery Layer”?

To me, a Discovery Layer allows a user to search across a library catalogue (or several), an ebook platform (or several), and a source of articles (or several).

A Discovery Layer could make use of one or more combinations of the following:

• Federated Searching
• a query is distributed to multiple sources, and responses are compiled, de-duped, and returned
• e.g. Sirsi Single Search
• Metasearching
• a single, regularly complied index is created from the collection of metadata from multiple sources
• e.g. Endeca, Google
• Single host environment
• metadata and content are placed on a common server
• e.g. WorldCat, Ontario Scholars Portal

Why a Discovery Layer?

The pursuit of a Discovery Layer seem to be driven by the need to present one, strong and stable user interface over many disparate sources of information. Some benefits of a discovery layer include:

• users only have to learn one interface, instead of many
• users don’t have to choose from lists of dozens of indexes
• users don’t have to repeat searches depending on format (one search for books, then one for dissertations, then one for articles…)
• users expect simple, effective search tools like Google

What’s the problem?
The challenges that face the construction of a discovery layer include:

• many of our research tools are very difficult to extract data from as they make use of a multitude of non-standard formats and protocols
• most of our research tools (especially the library catalogue) generate search results with poor relevance ranking
• some sources will be rich in text and metadata (articles, ebooks) while other sources will only be represented by metadata (print books)

How much can an improved interface improve things?

At the present time, I would say that there are 3 archetypes of Discovery Layer Interfaces.

• The first uses tabs, like PubMed and Illumina
• The second uses facets, like Endeca, WorldCat, and VuFind
• The third uses page partitions, like Google Experimental and UBC’s OneSearch.

How much can an improved interface, improve relevant results?
Coming up with what a user might deem relevant from 2 or 3 keywords is challenging in a regular search environment. Producing consistently relevant results in a federated or metasearch environment is extremely difficult.

Relevance might be improved through one or more of the following:

• by taking into account the user’s previous searching behaviour
• by weighing results by the number of times an item has been bookmarked, printed, or saved
• by using citation information to determine ‘likeness’ (e.g. based on a percentage of shared citations in item’s bibliography)
• by using user-created lists articles to generate similar items of possible interest
• by knowing what courses a users is currently taking/teaching and emphasizing relevant resources accordingly
  

What is a Good Enough Discovery Layer?
Is it realistic to expect a Discovery Layer to serve both the novice researcher and the expert to access a variety of formats in a multitude of disciplines? Can one size fit all? Should we develop several Discovery Layers with one for each discipline? (Arts, Social Sciences, Medicine). Should we develop one interface for undergraduates and one for faculty and graduate students?

How will we know we have reached the Promised Land?
Most discovery layers are still in the earliest stages of their development and by appearances, they seem more alike than unalike. How should we choose what is an acceptable product? One suggestion is to measure the success of a Discovery Layer by comparing its search results to Google.

Access 2007 Wrap Up Part Three - The Importance of Being Relevent

Peter Binkley did a simple and elegant comparison of a number of online library catalogues to begin his presentation, Searching the OPAC - The State of Play. He did performed the same search in all the library catalogues he examined; he did a keyword search for the word canada.

Its a ingenious example. You see, if a user types in a one word search in a catalogue for, say, dogs and gets results in which all the books that are entitled 'dogs' came up first, you would be hard pressed to say that those results weren't relevant.

But if you search for the word, canada, in say in most university library catalogues, you don't get books entitled Canada coming up first. Instead, the first result you get will likely be a document with a long name like Emergency food service : planning for disasters (U of A), Post-war Canadian housing and residential mortgage markets and the role of government (University of Windsor), or Progress report by the United States. Environmental Protection Agency. Great Lakes National Program Office and Environment Canada (University of Toronto).

This is because these library catalogues use a scoring system in which search and item word matches are used to assign points to items in order to determine relevancy. So a book by a Canadian government agency (which will frequently have the word Canada in its name twice - once for the English version and one for the French) about something Canadian, will tend to outscore the single word titled books.

Discovery layers don't correct this problem: La sécurité humaine et les femmes autochtones au Canada is the first result from McMaster's library catalogue; NCSU's first response is Compendium of plant disease and decay fungi in Canada, 1960-1980.

Librarians have largely left the responsibility of the library catalogue's search algorithm to the commercial library vendors who (putting this delicately) don't dedicate as much development time to it as, say, Google. But with the advent of open source indexing software such as Lucene some librarians are beginning to tinker with relevancy rules. Is it just coincidence that library catalogues that are running either the open-source Koha or the open-source Evergreen both pass the canada keyword test?

Another means of improving relevancy is to put the most popular items at the top of the list, like Amazon does. Bibliocommons and Koha use circulation counts in their relevancy rankings while WorldCat Local uses the number of global holdings in theirs.

But relevancy is too difficult a problem that can be solved by the simple tinkering of the numbered weights in the search algorithm.

A fundamental shortcoming of the library catalog is that it doesn't (and as currently designed can't) know the why for any given search. A search for the subject "breast cancer" in PVLD's catalog results in over 40 distinct subject headings listed in alphabetical order from the simple "Breast – Cancer" through "Breast – Cancer – Religious Life" to "Breast – Cancer – United States". The catalog doesn't know that one person is searching for books on this subject because she has to write a term paper, and another because his wife has just been diagnosed with the disease – and therefore it gives no clue as to which of these subject headings is most relevant to each person. [PVDL Director via Everything is Miscellaneous]

One of the means by which Bibliocommons addresses this issue is through the resorting of items so that highly rated items by trusted sources appear on the top of the list. This is one way of informing the library catalogue of who you are, at who you are as defined by who else you find trustworthy. Richard Wallis of Talis suggested during his Access 2007 presentation that if the catalogue knew a bit about you, it could present better search results to you. Using the example of the keyword lotus, an engineering student would get items about the car of the same name while the botanist would get items about the genus Nelumbo. Mark Leggot, using his metaphor of the search box as Lego brick, suggested in his Access 2007 presentation that it shouldn't be difficult to embed a library search box into a course-specific page that was already constrained to the sources that were relevant to both the subject of the course and the course level. Addendum: As Shibboleth allows for some attribute-based authentication to be passed to an online source, there may be a potential to tailor the interface or search results based on the user's subject specialty or readership level [Scholr 2.0: 3.2.2 Shibboleth].

That's the last of my Access 2007 wrap-up blog posts. Sorry for the irrelevant ending.

Access 2007 Wrap Up Part Two - The Importance of Being Open

(This is the second of what I hope will be three wrap-up posts on Access 2007. Rather than a chronological recount of the days' events, I've chosen to highlight three themes and threads that I picked up. The first one is here.)

The Importance of Being Open

Jessymn West began Access 2007 with a talk that reminded the audience of the rural population that libraries serve and some of the challenges one faces in order to serve them with proper technology. As an aside, she said that a state-wide union catalogue would go a long way to ease some of these challenges.

We learned the next day that the province of British Columbia is pursuing such a project. Ben Hyman spoke of the history and the goals of the project. It was decided that the best means to provide a common interface for all the public library branches of BC would be to first provide a common back-end. And so they have decided to use Evergreen, an open-source library catalogue. They made the same conclusion as the panel of Associate University Librarians did in their panel, ILS Options for Academic Libraries: choosing open-source software is no more risky than investing in a commercial software solution. As if on cue, Laurentian University announced at Access that they would making the shift to Evergreen.

Recognizing that only strengths of library catalogue technology are the 'backend' or inventory functions, many librarians are creating their own front-end interfaces to go over top of the library catalogue. If this interface can handle non-book materials like articles, this interface is frequently referred to as a 'discovery layer', perhaps incorrectly as it is still providing a 'search' function. There are still no usability studies that demonstrate these new layers provide any significant improvement to the user experience.

One reason why development in this area is slow is because many commercial library catalogues are difficult to 'skin' in the first place (in computer parlance, this means adding a layer as opposed to removing a layer) and one of the Access "Thundertalks" was dedicated to the extensive workarounds performed in order augment the interface of the library catalogue. The inability to freely access and remix one's own data hosted in commercial software is clearly frustrating to many librarians.

Joshua Ferraro of Liblime, a company that provides service and support of open source software, believes that much of the friction that occurs between librarians and vendors is a result of a larger problem: that software is largely a service industry operating under the persistent but unfounded delusion that it is a manufacturing industry. He suggests that a closer partnership between commercial vendors and libraries based on open tools, standards, and data could improve this relationship and, presumedly, improve the library experiences of our users. I believe that Biblicommons is an example of this next generation enterprise.

Beth Jefferson of Bibliocommons spoke at Access and at its pre-conference and shared some of her research dedicated to improving the improving library catalogues in the public library sphere. Bibliocommons has developed an interface with partners in the public library community, that integrates with a library's existing catalogue software. One way that Bibliocommons is different from the other library catalogue interfaces and 'discovery layers' that I have seen, is that it has made the effort to integrate the tagging and ranking of items at the user account screen (books out, when books are due, books on hold, etc.) since it is at this point when the reader is 'ready' to share what they felt about what they've read.

There are several implications to how they've done this but I just want to highlight one: library catalogue interfaces will have be able to seamlessly communicate with the information at the back-end (such as circulation data and patron information) and this means that library catalogue must have open data channels. The longer the commercial vendors resist opening up access to what is *our* data, the longer the opportunity for open-source library catalogues like Evergreen will have to make into the library marketplace.

A giant now off my shoulders

I have been reading, thinking and writing lots about academic librarianship and information technology as of late but you won't see the results of this work at this here blog.

That's because I, along with my Scholars Portage partner, Stacy Allison-Cassin, have just released a white paper called Scholr 2.0 on its very own blog to take advantage of the commenting goodness from the CommentPress WordPress theme.

While the purpose of the paper is to generate discussion among the librarians in the consortium that we both belong to, the conversation is open to anyone. Please join in.

Librarianship and cosines - Who knew?

Did your mind ever drift while sitting in that mandatory trigonometry class and you wondered, when will I ever need to know this in the real world?

For me, those daydreams just met their match as I read about the Vector Space Model:

Vector space model (or term vector model) is an algebraic model used for information filtering, information retrieval, indexing and relevancy rankings. It represents natural language documents (or any objects, in general) in a formal manner through the use of vectors (of identifiers, such as, for example, index terms) in a multi-dimensional linear space...

Documents are represented as vectors of index terms (keywords). The set of terms is a predefined collection of terms, for example the set of all unique words occurring in the document corpus.

Relevancy rankings of documents in a keyword search can be calculated, using the assumptions of document similarities theory, by comparing the deviation of angles between each document vector and the original query vector where the query is represented as same kind of vector as the documents.

In practice, it is easier to calculate the cosine of the angle between the vectors instead of the angle... A cosine value of zero means that the query and document vector were orthogonal and had no match (i.e. the query term did not exist in the document being considered).

Advanced Search is Bad For Users

A couple recommended reads [via]:

Twenty-five years of end-user searching, Part 1: Research findings
Karen Markey
Journal of the American Society for Information Science and Technology
Volume 58, Issue 8 , Pages 1071 - 1081
http://dx.doi.org/10.1002/asi.20462

This review summarizes quantifiable evidence on end-user searching. Some findings:

- mean number of queries per search session (most between 2 and 4)
- the use of boolean operators (less than 15% use AND; less than 2% use OR)
- most end users accept default values for searching
- when end users use advance search features in their queries, they use them incorrectly about one third of the time
- the vast majority of end-users are satisfied with their searches

Twenty-five years of end-user searching, Part 2: Future Research Directions
Karen Markey
Journal of the American Society for Information Science and Technology
Volume 58, Issue 8 , Pages 1123 - 1130
http://dx.doi.org/10.1002/asi.20601

Discusses research findings about end-user searching in the context of current information retrieval models.

"When researchers analyse end-users' failed searches, the number one problem is their initial choice of search terms (Debowski, [2001], p. 377; Hsieh-Yee, [1993], p. 169; Lucas & Topi, [2002], p. 105; Sewell & Teitelbaum, [1986], p. 241; Wildemuth & Moore, [1995], p. 299). Instead of using a database's controlled vocabulary, users search for the first terms that come to mind. Failing to use the controlled vocabulary has an adverse effect on the precision of their searches and makes it impossible for users to enlist the vocabulary's special search features such as exploding terms, listing subheadings, and displaying term relationships."

This article brings together two personal interests - user interfaces and improved information literacy practice in libraries. They dovetail nicely here: let us create simple user interfaces that suggest better search terms to our users. Let's concentrate on teaching thinking about language and ideas and end the teaching of boolean searching and if we dare, let's get rid of the advanced search screen.

Is Google Dead Without Metasearch?

When Roy Tennant spoke to the 2005 SMUG Annual Meeting, the title of his presentation was Google Scholar: Is Metasearch Dead? And I remember that during his talk, Roy searched for the word hamlet in Google Scholar and the first hit returned was Hamlet and the Holodeck.

Now if you search Google Scholar for hamlet, you get this article: Partition testing does not inspire confidence [program testing]. Why? Its by D. Hamlet.

Search plain-vanilla Google for hamlet and the first entry is not surprisingly, from Wikipedia. What's amusing, although not together unsurprising, is the the Spark Notes for Hamlet outranks the actual play. What was surprising to me is that if you search for Hamlet in Google Book Search, you get only 6 hits returned. The University of Windsor has 95 titles that begin with that word. Search WorldCat for Hamlet and you are offered over 13,000 items.

Far from being dead, it looks like Google needs Metasearch.

We're Information Literacy. Understand? Do not Demonstrate

The W in WILU stands for Workshops and I attended one entitled, Teaching on the Edge of Chaos. In it, Bryan Miyagishima and Robert Hautala put forwards a model for teaching that recognizes three elements in play: the student, the task and the environment. They suggested that since you can't change the student, you should concentrate on changing the task and/or the environment for better learning.

Joel Burkholder started his presentation by asking the audience what exactly are we librarians, for lack of a better word, trying to 'fix' in our students research habits. Some of the answers (in short form) that came back were:

keyword searching
starting with Google / Wikipedia
full text only
browse few
linear process
first hit reliance
only recent articles
reliance on one source
giving up
don't check scope notes
no plan
no or poor evaluation skills
don't ask for help

Joel then asked the audience, how many of these could we 'pin the blame for' on Google? He suggested that the habitual (and largely successful and satisfied) use of search engines in our students' lives are responsible for their searching 'framework' that they take into the classroom lab or library. Since this framework is working for them, they see no need to change it no matter how many times librarians tell them that there are better ways to research. Burkholder believes that asking students to perform mechanical skills that illustrate a new searching framework is also not sufficient to make students change their ways.

Instead, he suggests we should :

1. find ways to make the student confront their beliefs and reveal preconceptions
2. discuss and evaluate these preconceptions
3. create cognitive conflict
4. offer an intelligible, plausible and fruitful alternative framework

At this point, I realized that Joel was teaching the use of a framework for teaching by using that very same framework for teaching. Coincidently, I found it intelligible, plausible and fruitful.

Filtering out the boolean searches

There's something about PubMed's filtering function that I love...



Essentially, registered users of PubMed can create up to five customized filters that immediately narrow down your search to either the type of research you are interested in (randomized control trial, review articles) or by access (free full text, your library) or by some other means you devise.

What appeals to me is that the filtering is that it occurs automatically post-search.

In the days of yore, librarians used to try get students to stop and consider alternative keywords and how to construct a proper Boolean search and so on before they searched. But now, even random words generate something vaguely useful in the Googlesphere or the scholars portal. So rather than encouraging the user to revise the search, I think we should try to build the tools that allow the user to refine results.

With every search in Pubmed, I can immediately see the impact of five different refinements. And of course, now I want more filters.