There have being quite a lot of indirect evidence pointing to this , but Richard Nurse adds further to this data.
In this interesting blog post, he analysed the searches made in the different search tabs on his library's website. This includes "collections" tab (basically a discovery product, I believe Ebsco Discovery Service), "website tab" and a "Catalogue" tab. He also threw in I think searches in the older federated search product.
He found that...
"Looking at the top 20 results for each type of search then we find that about 40% of them are identical across all the search boxes. That goes up to over 50% if you look at 3 out of 4 search boxes."
You should really read his post for more details and interesting thoughts, but like me he wonders "maybe a single search box is the way to go. But does that then open up a presentation issue about how do you show the results? "
NCSU is also really impressive, it pulls results from Summon, library catalogues, journal specific searches, databases titles, Website results etc. And it's not just a straight forward pull of results into each category.
Notice under Library Website it actually differentiates between FAQs and general website results. Under the Books & Media results, it includes browse by format, The article results as additional suggestions "Find more articles in specialised databases" which exposes specialised databases not well covered by Summon.
The More Search Options generates links on the fly that when clicked will search different sources including Institutional repositories, google scholar with the search term you just searched.
This last option is a very low cost option that can be easily applied to most sources.
So you could have a FAQ search box that gives FAQ results followed by a link that says "Search for articles" and "Search website" that when clicked does the same search in these 2 sources. This ensures that the user is not left high and dry if they search for non-faq items in the FAQ search box.
Any other libraries trying to create an all-in-one box? Any unusual creative method of presenting results from diverse sources?
Does anyone have data to share with regards to effectiveness or lack whereof with regards to the search tabs/boxes format commonly seen on library portals? Do users really know they are searching different silos and choose the right search tab for their query?
EDIT 09 April 2011 : I just came across "Why We Are Not Google: Lessons from a Library Web site Usability Study" (subscription needed) . The authors found a surprising result. Putting a tabbed search box on the front page as opposed to currently where the search is on secondary pages did not improve usability.
Some choice quotes "In almost all cases when students needed to click down one level to our research tools page, they did so successfully. We found little evidence that having the gateway to the research section of our site on a secondary page presented an obstacle to participants successfully completing tasks."
Even though "The single search box seemed comparatively effective in helping students find books and articles, but it became an obstacle in helping students locate other pieces of information on our site."
"The search box became an obstacle in other questions where it should not have been used. In some cases, the search box was viewed as an all-encompassing search of the entire site. Several students searched for administrative information, research guides, and podcasts in this box. It is unlikely students would have utilized this search for these tasks if it was on a secondary page. In some cases, the search box seemed to force all other links to blend into the background even though they may have been an obvious choice. "
A true one search box like NCSU's might be the answer here.