Search Analytics for your Site

Site search analytics in the NY TImes

Mon, 28 Apr 2008 07:38:40 -0500

From yesterday's New York Times article on nude vacationing:

SpaFinder.com, an online spa search engine, recently created a separate category for “nudist spa vacations” after noticing an increase in searches for the term. Since November, searches on SpaFinder.com for such trips have averaged about 720 a month—beating out “pet-friendly spas” (284) and “waxing services” (298).

Thanks to Alexandra Fox for the tip!

What kinds of data to log?

Thu, 10 Apr 2008 14:10:10 -0500

Alex, a college student, writes:

...I'm building a web based library catalog for a special collection of books as part of a group software engineering project. My group has been really interested in UX methods related to conceptual models and task based evaluation (so on the qualitative end) but we are also interested in using some more quantitative methods. Since search is such a key feature of our site, and can provide such telling insight in general, I thought I would reach out to you.

Do you have any suggestions for what types of information to log beyond the query (and IP, host, browser info) or any other advice / cautions?

My quick answer for Alex:

Definitely try to log which pages queries originated from (and what those queries were). This will help you understand where navigation might be failing and what people search when it does fail. (Thereby enabling you to improve contextual navigation from those pages.) You should also log the other end of the process: what search results people click through. This will help determine what's popular; you can also compare which documents are clicked through with which ones you want to have clicked through (and thereby test your most frequent queries' performance). Also log the time and date so you can track sessions.

What might you add?

Site search analytics workshop slides

Thu, 27 Mar 2008 21:20:35 -0500

Just posted my presentation (via the wonderful SlideShare) for my Boston/Sunnyvale/Chicago workshops; enjoy!

| View | Upload your own

Site search analytics talks in Philadelphia, Urbana-Champaign

Tue, 25 Mar 2008 11:44:51 -0500

I'll be giving a short talk on site search analytics the evening of April 2, graciously sponsored by PhillyCHI and hosted by MessageFirst. Here are the details:

Time: 6 - 8PM (social time from 6 - 6:30PM)
Location: Messagefirst
230 N 2nd St. Suite 2C
(215) 825-7423
Map: tinyurl.com/yqsvqv
RSVP: phillychi [at] gmail.com

I'll also lead a short (and similar) session at the University of Illinois Webmaster Forum on April 30.

Finally, my full-day site search analytics workshops are set for Boston (April 4), Sunnyvale (April 22—note the date change), and Chicago (May 15).

Hope to see you this spring!

Five reasons UX people need site search analtyics

Mon, 10 Mar 2008 11:30:06 -0500

Balance: UX methodologies/practitioners are often quantitatively weak
Legitimacy: Data-driven analyses make an impression on people hard to impress
Cost: The data is available; the analysis scales well with available time
Fidelity: The data is behavioral, real, and voluminous
Comprehensiveness: Quantitative methods complement and help improve qualitative methods (and vice versa)

Video on site search analytics

Sat, 08 Mar 2008 17:50:47 -0500

Nice 10 minute video (albeit blurry) on how to set up site search analytics for Google Analytics:

Some general questions (and answers) about site search analytics

Fri, 29 Feb 2008 15:23:32 -0500

What are some of the common questions you hear about site search analytics? More importantly, how do you answer them?

We've begun a list of general questions and answers below. What would you add? How would you answer them differently?

We'll follow them up with additional FAQs for managers and IT professionals, respectively, in the coming weeks.

Why bother with site search analytics? After all, we already know what our users want.
SSA might be useful, but I’m already too busy; why bother?
We already use a service that tells us how users find our site when searching the web; isn’t that the same thing as SSA?
If site search analytics is so wonderful, why isn’t everyone doing it?
Our search engine works fine; why mess with perfection?

And now for some answers...

Why bother with site search analytics? After all, we already know what our users want.
Short answer: Trust us, you don’t.
Long answer: Granted, any response prefaced by the words “trust us” is suspect. But if you’re asking this particular question—or struggling to answer it for someone else—then you need to take a crash course on something much broader than (and inclusive of) SSA: user-centered design. UCD is based on the assumption that no amount of experience, intuition, or expertise regarding users’ needs is a sufficient substitute for finding out directly from users what they need. There are simply too many unknowns and too many surprises for designers to safely assume that “they know what’s right”.
UCD practitioners rely upon a host of methods to learn about users’ needs: some are qualitative (such as field studies), some are quantitative (such as all forms of analytics, including SSA) and some have aspects of both (such as task analyses, card sorting, and surveys). SSA is simply one more tool in the toolkit, though it’s still relatively unknown and under-utilized.
So, if you’re already on board with UCD, look to SSA to provide yet another perspective on users’ collective behavior—in this case, one that’s based on loads of real data that you already own. If not, we suggest buying a crateful of Steve Krug’s mirthful masterpiece, Don’t Make Me Think!. Read one and hand the rest out to every important person in your organization.

SSA might be useful, but I’m already too busy; why bother?
Short answer: With minimal investment, SSA will save you time and money, help provide a better user experience for your customers, and make you a happier person.
Long answer: If you're already busy, then you should get down on your knees and kiss the ring of anyone who offers you an easier way to prioritize your work. SSA, as you'll see, is all about helping busy people prioritize their problems: What content should we create? How can we improve our tagging? Can we avoid running focus groups to determine what’s wrong with our search system? And so on, and so on.… By extension, using SSA will give you a better sense of the problems that don’t merit your attention. In effect, a minimal investment in SSA might make you a less busy person.
For example, you might be the primary author of your site’s content, creating prose at the behest of product managers, marketing staff, and other pests. The list of documents that require your attention is longer than you'll get to in any reasonable amount of time. Rather than pulling an all-nighter, you can use SSA to determine which of those documents is going to plug the exposed content gap by examining which frequent searches retrieved zero results. Now you can prioritize your to-do list knowing that you'll be working first on the documents that users’ need—not the stuff that’s on your list because a senior vice president wants it on the site.
Or perhaps you're responsible for your site's search system. Users have been complaining that search is failing them, so you're being pressed to replace the search engine application. That would be a difficult, time-consuming, and expensive undertaking. But your search analysis indicates that a few minor tweaks to the search interface and the implementation of a spell-checker would dramatically reduce the difficulties of searching—and, at one-tenth the cost of a new search engine. Now you’ve saved both time and money!

We already use a service that tells us how users find our site when searching the web; isn’t that the same thing as SSA?
Short answer: No, not really. Well, sort of. There’s a difference between how users search the web to find your site (typically called Search Engine Optimization), and how they search within your site. This book focuses on the latter variety of search analytics. And for lack of a better, snappier term, we call this variety “site search analytics.”
Long answer: Knowing what keywords users enter to search in Google, Yahoo! or other web-wide search engines to reach your site is incredibly valuable. It also enables you to fine-tune a keyword ad campaign; perhaps, a campaign that could be designed specifically to draw visitors to your site accordingly. Once users reach your site, however, their needs often change. For many who search the web, the process unfolds in two stages: First, they figure out which site might have the answer to their query. Second, they search within that site for the answer. Because users’ information needs have changed, the data from these two stages of search will necessarily vary. SSA will help you do a better job with that second stage.
Many would argue that local search is no longer important: Users increasingly search Google to get them deep into a, thereby bypassing both the local site’s navigation and search system. Or, they use a combination of RSS feeds and aggregators to get the content they need without ever visiting that content’s home site, much less using its search system. There is some truth to these points, but local search isn’t going away any time soon for these reasons:
- Web-wide search tools like Google, Yahoo!, and MSN may not be optimized for local site searches, where criteria other than link popularity may be more important in determining how to rank search results.
- Sites with captive or loyal users, such as Amazon.com, can get away with local search that doesn’t match web-wide searching conventions.
- Sites with highly structured content, such as ESPN.com, often require a customized search system to allow searching specialized fields.
- Accessing a site’s deep content via syndication, however useful, supports a very different user scenario than searching. The former is an example of monitoring, while the latter fills a specific ad hoc need.
- Sites with “deep” content, not to mention Intranets and other secure systems, can’t be indexed or searched by web-wide search tools.

If site search analytics is so wonderful, why isn’t everyone doing it?
Short answer: For a variety of reasons, the people responsible for most websites remain ignorant of SSA’s existence and its benefits.
Long answer: It is strange that SSA isn’t more commonplace, especially when you consider how useful and relatively inexpensive it is. It’s even more surprising given the recent trend toward user-centered design. Three reasons seem most likely:
- Most people assume that anything that mentions “search” means search engines. Because search engines are a technology, they are typically the sole property of IT departments. So designers—the people most commonly responsible for user experience—don’t think to take on search experience, or aren’t allowed to by IT, who often have other things on their minds besides user experience.
- The people who should be responsible for the user’s search experience tend to work on challenges (such as improved navigation) are more within their comfort zone and are easier to fix.
- There is precious little guidance or reporting tools out there to help people perform SSA. Just reams of data, which can be difficult to pry from the IT folks, who have other things to worry about.
We hope this book helps break down these barriers by making a case for the value of SSA, and showing you how easy it is to do.

Our search engine works fine; why mess with perfection?
Short answer: Nothing is perfect, especially search. SSA is the best diagnostic tool to help you determine what to fix and how to fix it.
Long answer: Search is a very complex mix of users, behaviors, interfaces, applications, content, and context. There are many possible failure points—and room for improvements—along the way. With so much complexity and so many challenges, you could apply just about any user-centered design method, like task analysis or field studies, to analyze a search system. We’ll bet dollars to donuts that you’ll uncover plenty of dissatisfaction among searchers and many areas that beg for improvement. Anyone involved with designing and maintaining your site and its content will realize this. SSA just happens to be one of the least expensive, and perhaps most powerful, ways of analyzing your site’s performance.. And it’s the method most focused on the search experience.

More site search analytics seminars: Silicon Valley, Boston, Chicago

Thu, 21 Feb 2008 16:08:56 -0500

I'll be headed to Sunnyvale, California March 18 to teach my first workshop on site search analytics, a day-long affair hosted by my friends at Involution Studios' Master Academy. The class will be intimate, as we're limiting attendance to nine participants, and very hands-on. Fee for the day is $699.

I'll follow that up with sessions in Boston (April 4) and Chicago (May 15). These will be coupled with web usability seminars taught by Steve Krug, author of the wonderful Don't Make Me Think! You can register for one or both workshops; there are discounts for the combination and for early registration. The full rate for a single day is $895.

So to recap:

Sunnyvale, CA: March 18

Boston, MA: April 4

Chicago, IL: May 15

You'll find all the details about my workshops right here.

Rich Wiggins to teach search analytics in Sydney, March 2008

Thu, 20 Dec 2007 06:37:58 -0500

Fresh from the Nielsen Norman Group's speaking tour, Rich Wiggins will be headed to Sydney in March to teach his day-long class for the wonderful people at Key Forums. I taught a Key Forums class two years ago; the production was top-notch, and the hosts and attendees were warm and engaging. I'd go back in a heartbeat.

Learn more about the seminar on the Key Forums site; details from the news release follow:

Search Analytics: Harnessing site search analytics to better understand and meet your user and customer needs

Hosted by Rich Wiggins
6th March 2008, Sydney

Rich Wiggins, the leading international Search Analytics expert, is coming to Sydney on 6th March 2008 to host this intensive one-day master class. Rich’s day job is at Michigan State University where he holds the position of Senior Information Technologist. He is currently working on his book, Search Analytics for your Site: Conversations with your customers, which he is writing with the world renowned Lou Rosenfeld (co-author of Information Architecture for the World Wide Web). This is the first time Rich has visited Australia to host such an event and it is not to be missed.

Search Analytics (SA) is a hugely valuable, and typically under-exploited, tool that can enable you to better understand your customers’ needs—in their own words—and then tune your search service, content, metadata and site navigation to meet those needs. SA is a powerful and often neglected method for improving search performance and overall site effectiveness.

To see a full copy of the brochure for this event, please email Helen Lewis (Helen.lewis [at] keyforums.com.au) or call on +61 2 9436 4255. Bookings can be made online at www.keyforums.com.au or via fax to +61 2 9436 2237.

Search analytics slides from An Event Apart

Wed, 19 Dec 2007 17:08:58 -0500

Better late than never! I totally forgot to post my slides from my presentation at the most excellent An Event Apart in Chicago (last August; guess it's been a busy fall!). Here they are (don't let the goofy typeface scare you; it happens some times with SlideShare):

| View | Upload your own

The talk went fine, though I'm not sure the web developer audience there knew or cared much about site search analytics when I started off. Hopefully at least a few more were interested by the time I finished...

Completing the book (and how you can help)

Thu, 06 Dec 2007 10:35:06 -0500

If you've been waiting for this book to be available, we appreciate your patience and feel your pain. Rich and I began writing over two years ago, and, a la Damocles, it's been a sword hanging over our heads ever since. It's been difficult to find the time—both of us have day jobs (and some of us have more than one!). And frankly, the landscape of site search analytics has dramatically shifted under our feet quite a bit during that time (thanks Google!). We've learned quite a bit since beginning work on the book, and are trying our best to keep up and, we hope, push it forward a bit on our own.

The good news is that the writing is probably at least 50% complete. We have a fantastic collection of case materials and data sets from many, many generous sources. And thankfully, we have some time now. (For various reasons, I need to be finished with the bulk of the work by winter's end.)

But we're missing one thing: a data set that includes referrer data and click-through data. We've not been able to find a source that can provide this data, and it's really frustrating, as these represent both ends of the search process, respectively. We really want to analyze a data set that will help us address questions like these in the book:

From what pages were the queries initiated?
For pages where queries began, what were those queries and how successful were they?
What search results were clicked through most frequently for common queries?

If we can't perform these types of analyses, the book will be incomplete. So, if you want some free consulting, have a reasonably interesting site (e.g., not a personal blog), are willing to share data with us, and wouldn't mind lots of acknowledgment and positive publicity, then please: talk to us.

Mondosoft quotes Mark Twain

Thu, 08 Nov 2007 11:31:48 -0500

Recently we grieved the passing of Mondosoft. But according to Mondosoft's Martin Vikkelsø Madsen, "the rumors of my death are greatly exaggerated". Here's the hopeful news, verbatim from Martin:

"It’s true that Mondosoft went through a bankruptcy, but cmswatch really made it sound bad based on poor translation from their Scandinavian source. The bankruptcy step was part of a plan to reorganize the financials – just like many US airlines have done over the years. Although this case was based on Danish law and our equivalent to ‘chapter 11’.

"That said, Mondosoft is still doing business together with the company who bought us (SurfRay). But naturally development has slowed down a bit during all this legal stuff. Hopefully our upcoming product “Ontolica Reporting” will be pushed to market soon. In short it is an improved, up-to-date version of BehaviorTracking with a new engine. It’s actually a stand-alone reporting service, but the first release will come with a hook to integrate it with our Ontolica product. The Reporting tool could in theory be integrated with any system capable of creating log files. But enough product talk for now.

"All that aside, I am really overwhelmed about your concern for me and my old colleagues :-)
Thank you for that."

Martin, it's great to hear this; thanks for the update. Please let us know when Ontolica Reporting is available and we'll get the word out!

Hello Google Analytics, Goodbye Mondosoft

Mon, 05 Nov 2007 09:14:59 -0500

Well, unless you've been vacationing somewhere warm, tropical, and without connectivity, you probably know by now that Google Analytics has finally released its site search analytics functionality. Frankly, I found it pretty difficult to figure out how to turn it on from GA's account administration interface. But it's possible, and this document will help. I can't wait to give those tires a kick.

This is really a Big Deal for our emerging field. Because GA is becoming so ubiquitous, we should see a real spike in the appreciation of and demand for search analytics. And it should spur other analytics and search engine vendors to provide improved reporting.

Speaking of which, there is some rotten news from Denmark: Mondosoft is going away. Mondosoft has always seemed to appreciate the value of site search analytics as well as, if not better than, any other search engine vendor out there. I remain flummoxed that so many of their peers have ceded something so obvious to the analytics industry. Site search analytics is simply the best way to diagnose problems and improve search engine performance in reasonable, iterative, and often inexpensive ways. Oddly and sadly, a search company that really saw the value in closing this "feedback loop" has failed, but I have a sneaking suspicion that its demise had little to do with its focus on search analytics. My regrets and best wishes to Martin Vikkelsø Madsen and his colleagues at Mondosoft, whom I've found wonderfully helpful over the past few years.

Changing your content creation strategy

Fri, 05 Oct 2007 15:15:47 -0500

I'm working on a chapter on using search analytics to improve a site's content. It'll cover some of the stuff you'd expect, like using top null results queries to tune your content offerings. But I'm also hoping to dig a little deeper into some less-than-obvious uses of analyzing query data for improving content.

Here's one that I'm not entirely sure makes sense, and I'd love some input. Without a huge effort, it's possible to categorize queries. We've already posted an example where frequent queries from Michigan State University were categorized (note the color coding). This doesn't take that long to do, and by addressing the short head, you're actually categorizing a large portion of all of the site's queries.

If you tally the numbers associated with each category, you can start determining an overall profile of your searchers' needs. 21% of searchers might be looking for information on navigating the campus, 15% might want information on study abroad, 4% might want health information...

So here's the question: with such a profile in hand, might you revamp your content creation strategy?

If, for example, you found that your only 5% of your content addressed study abroad, would it make sense to try to increase the supply of such content to match the 15% demand?

I'm not sure this makes much sense, as I'm not sure how exactly one would determine the topicality of their content. I'm not certain the math makes all that much sense either.

But I wonder if there's something to this that I'm missing. If nothing else, could we use that breakdown of searcher's needs by percentage as some sort of loose guideline for content creation?

Sample chapter: search analytics and metadata

Mon, 01 Oct 2007 16:33:19 -0500

We're publishing a fairly complete version of our chapter on how you can use internal search analytics to develop and improve metadata (318Kb PDF file). We'd naturally love to hear what you think, especially if we're missing anything obvious, or if you can come up with better examples of concepts for us to use to illustrate some of these concepts. (Examples are always the hardest part!) Many thanks.