ia play

Last week I went to the Search Solutions event, held by BCS in their lovely office in Southampton Street. There were maybe 50 people, 6 or 7 women and seemingly even less laptops (which rather made it stand out from the more web-focused events I usually attend – because of lack of laptops not the male-female ratio).

I didn’t make masses of notes but I did capture a few points and reminders:

Vivian Lin Dufour from Yahoo talked about Search Pad, an attempt to make search more “stateful”.

Richard Russell from Google explained how the auctions for Google Ads work. Always interesting to hear more about the money side of things.

Dave Mountain, a geographer (another example of Nominative Determinism?) talked about geographical aspects of searching. He explained that if the task is “finding the nearest cafe”, then the ‘near’ isn’t a simple statement. There are types of near: as the crow flies, in travel time, in the direction I’m already going. After all you may not be interested in a cafe that’s already 5 miles behind you on the motorway. He had some good slides covering this, so hopefully they’ll be made available.

Tony Russell-Rose discussed Endeca’s impending pattern library. Should be interesting – public version to be available in the new year.

David White of Web Optimiser talked amongst other things about the importance of cross-media optimisation. He asked why don’t more companies, especially b2b ones, have phone numbers in title/description of search results? He also touched on the growth of twitter as a substantial source of referrals (in response to a question about whether Bing was increasing referrals and thus changing optimisation tactics).

Richard Boulton, as well as discussing his efforts with open source search, introduced us to the marvelous concept of dev/fort/.

“Imagine a place of no distractions, no IM, no Twitter — in fact, no internet. Within, a group of a dozen or more developers, designers, thinkers and doers. And a lot of a food.

Now imagine that place is a fort.”

Well marvellous to me but I wanted to get married in a Napoleonic fort so perhaps I’m not typical. He also mentioned searchevent.org, a day dedicated to open source search systems, which will hopefully happen again sometime.

Andrew Maisey talked about a school of thought that search will increasingly become less important on the site. Dynamic user journeys will encourage more browsing.

(Food was pretty good as usual for the venue.  I’m hoping that we’re going back to BCS for our team away-day later in the year and then I can have more of the strawberry tarts.)

Surprisingly this white paper on building multilingual solutions in SharePoints provides a good overview of how the search works, regardless of whether you are interested in the multilingual aspect.

White paper: Plan for building multilingual solutions.

Read page 15, titled “overview of the language features in search” for a description of content crawling and search query extraction. Then 16-18 provide a good overview of individual features and what they are doing.

Word breakers A word breaker is a component used by the query and index engines to break compound words and phrases into individual words or tokens. If there is no word breaker for a specific language, the neutral word breaker is used, in which case word breaking occurs where there are white spaces between the words and phrases. At indexing time, if there is any locale information associated with the document (for example, a Word document contains locale information for each text chunk), the index engine will try to use the word breaker for that locale. If the document does not contain any locale information, the user locale of the computer the indexer is installed on is used instead. At query time, the locale (HTTP_ACCEPT_LANGUAGE) of the browser from which the query was sent is used to perform word breaking on the query. Additional information about the language availability of the word breaker component is available in Appendix B: Search Language Considerations.

Stemming Stemming is a feature of the word breaker component used only by the query engine to determine where the word boundaries are in the stream of characters in the query. A stemmer extracts the root form of a given word. For example, ”running,” ”ran,” and ”runner“ are all variants of the verb ”to run.” In some languages, a stemmer expands the root form of a word to alternate forms. Stemming is turned off by default. Stemmers are available only for languages that have morphological expansion; this means that, for languages where stemmers are not available, turning on this feature in the Search Result Page (CoreResult Web Part) will not have any effect. Additional information about language availability for the Stemmer feature is available in Appendix B: Search Language Considerations.

Noise words dictionary Noise words are words that do not add value to a query, such as ”and,” ”the,” and ”a.” The indexing engine filters them to save index space and to increase performance. Noise word files are customizable, language-specific text files. These files are a simple list of words, one per line. If a noise word file is changed, you must perform a full update of the index to incorporate the changes. Additional information about the noise words dictionary and how to customize it is available at www.microsoft.com.

Custom dictionary The custom dictionary file contains values that the search server must include at index and query times. Custom dictionary lists are customizable, language-specific text files. These files are used by Search in both the index and query processes to identify exceptions to the noise word dictionaries. A word such as “AT&T,” for example, will never be indexed by default because the word breaker breaks it into single noise words. To avoid this, the user can add ”AT&T” to the custom dictionary file; as result, this word will be treated as an exception by the word breaker and will be indexed and queried. These files contain a simple list of words, one per line. If the custom dictionary file is changed, you must perform a full update of the index to incorporate the changes. By default, no custom dictionary file is installed during Office SharePoint Server 2007 Setup. Additional information about the custom dictionary file and how to customize it is available at www.microsoft.com.

Thesaurus There is a configurable thesaurus file for each language that Search supports. Using the thesaurus, you can specify synonyms for words and also automatically replace words in a query with other words that you specify. The thesaurus used will always be in the language of the query, not necessarily the server’s user locale. If a language-specific thesaurus is not available, a neutral thesaurus (tseneu.xml) is used. Additional information about the thesaurus file and how to customize it is available at www.microsoft.com.

Language Auto Detection The Language Auto Detection (LAD) feature generates a best guess about the language of a text chunk based on the Unicode range and other language patterns. Basically, it’s used for relevance calculation by the index engine and in queries sent from the Advanced Search Web Part, where the user is able to specify constraints on the language of the documents returned by a query.

Did You Mean? The Did You Mean? feature is used by the query engine to catch possible spelling errors and to provide suggestions for queries. The Did You Mean? feature builds suggestions by using three components:

· Query log Information tracked in the query log includes the query terms used, when the search results were returned for search queries, and the pages that were viewed from search results. This search usage data helps you understand how people are using search and what information they are seeking. You can use this data to help determine how to improve the search experience for users.

· Dictionary lexicon A dictionary of most-used lexicons provided at installation time.

· Custom lexicon A collection of the most frequently occurring words in the corpus, built at query time by the query engine from indexed information.

The Did You Mean? suggestions are available only for English, French, German, and Spanish.

Definition Extraction The Definition Extraction feature finds definitions for candidate terms and identifies acronyms and their expansions by examining the grammatical structure of sentences that have been indexed (for example, NASA, radar, modem, and so on). It is only available for English.

Web accessibility is a reasonably familiar topic for IAs but document accessibility is also important. Here’s some considerations for your typical Word documents.

To support screen magnification and other adjustments:

• don’t set the text to black. choose automatic (if you set the text to black and the person reading has the colours reversed for ease of reading then all of your text will disappear)
• use a simple clear font e.g. Ariel
• avoid italics
• use left aligned text including headings (screen magnification users often don’t realise there is content that is centred or right aligned)
• don’t use other colours for fonts (the RNIB training specifically asks us not to use fancy colours like purple. I don’t think it was particularly aimed at me)
• use 14 point text as the standard font size (this seems huge to me, but this is our recommended standard as meeting the needs of most readers)

Screen readers with speech output

• use the correct Word styles
• use heading hierarchies to communicate the structure of the document

You’ll note the advice is less detailed for screen readers. This mirrors my experience with web design in the RNIB. Outside the RNIB most accessibility conversations I heard focused on the challenge of designing for screenreaders but the challenges are much greater in designing for both magnification users and fully sighted users at the same time.

I’m going to “Innovations in Web and Enterprise Search” at BCS next week

Search Solutions is a special one-day event dedicated to the latest innovations in web and enterprise search. In contrast to other major industry events, Search Solutions aims to be highly interactive and collegial, with attendance limited to 60-80 delegates.

Provisional programme

09:30 – 10:00 Registration and coffee

Session 1: (Chair: Tony Russell-Rose)

* 10:00 Introduction – Alan Pollard, BCS President

* 10:10 “Enterprising Search” – Mike Taylor, Microsoft

* 10:35 Accessing Digital Memory: Yahoo! Search Pad – Vivian Lin Dufour, Yahoo

* 11:00 “How Google Ads Work” – Richard Russell, Google

11:25 – 11:45 COFFEE BREAK

Session 2: (Chair: Andy MacFarlane)

* 11:45 “Location-based services: Positioning, Geocontent and Location-aware Applications” – Dave Mountain, Placr

* 12:10 “Librarians, metadata, and search” – Alan Oliver, Ex Libris

* 12:35 “UI Design Patterns for Search & Information Discovery”- Tony Russell-Rose, Endeca

13:00 – 14:15 LUNCH

Session 3: (Chair: Leif Azzopardi)

* 14:15 “Search-Based Applications: the Maturation of Search” – Greg Grefenstette, Exalead

* 14:40 “How and why you need to calculate the true value of page 1 natural search engine positions” – Gary Jennings, WebOptimiser

* 15:05 “Search as a service with Xapian” – Richard Boulton, Lemur Consulting

15:30 – 16:00 TEA BREAK

Session 4: (Chair: Alex Bailey)

* 16:00 “The Benefits of Taxonomy in Content Management”, Andrew Maisey, Unified Solutions

* 16:25 Panel: “Interactive Information Retrieval” – details to follow

17:00 – 19:00 DRINKS RECEPTION

via BCS IRSG – Search Solutions 2009.

A recent conversation with a friend generated shock (and even a little scorn) that I’d been producing wireframes. I was firmly entreated to sketch instead. Around the same time a recruiter approached me with information on a job that would require detailed annotated UI specs of around 40 pages every fortnight.

The profession is still judged, by and large, by the quality of our documentation. Most recruiters and hiring managers seem more interested in the quality of annotation than the quality of thinking.

I’m rather inconsistent in my approach to documentation. Mostly the medium is picked for the context. Is the project agile? How good are the developers? Is there a remote team? Do lots of people need to be consulted? What are their reading preferences?

Whilst I’m happier with pen and paper  than computer, I think it is far to say that I doodle a good deal more than I sketch.  Now there’s always a way to get chickens into a blog post… this little trio were sketched during a conference presentation, presumably a scintilating one and probably about something 2.0 related given the labelling of the fowl.

In fact, it appears I doodle most when irritated by the speaker. In this case , rather than asking an insightful question to highlight the cliched and superficial nature of the argument, I wrote “blog, wisdom of the crowds, whatever”. That told him, I’m sure. I do still want this mug though:

None of this is what my friend had in mind though. She’d like this more: part user journeys, part concept map, but mostly not very pretty. Not really for sharing (apart from with you lot, of course) but it could be re-jigged into something more respectable.

I do these little pages all the time but again they aren’t for collaborative purposes. This one was so I could sanity check we had all the functionality we’d need on the product backlog before the supplier drew up the drawbridge.

Then of course, there’s cheating. Those search forms I shared recently were created in Visio but with the sketchy stencil:

I very rarely do this kind of documentation anymore. My business stakeholders are bored by them and the developers are best told what to do by pointing over their shoulders.

I do still do content models. This kind of specification still gets traction with the developers:

But, horror of horrors, a lot of my documentation these days is actually reasonably high-fidelity mock-ups. These are really aimed at the business stakeholders. Colours and fonts are pretty much fixed by our visibility requirements, so the business units know better than to ask for their favourite shade of puce.  And they worry less if they don’t have to try and visualise from wireframes. It doesn’t take me any longer as I’ve got a colour stencil and the choices are pretty limited.

Is this ironic? I’m working for an organisation of and for blind people and I’m producing the most colourful deliverables ever.  But then you should see the colour of the office floors.

Some of my colleagues use screen readers with braille display output. I’d never come across this particular form of access tech and it wasn’t immediately obvious why the braille displays are necessary…or indeed how they worked. They look a bit space-age, or rather 1960s Sci-Fi movies idea of space age.

According to Wikipedia:

“The mechanism which raises the dots uses the piezo effect of some crystals, where they expand when a voltage is applied to them. Such a crystal is connected to a lever, which in turn raises the dot. There has to be a crystal for each dot of the display, i.e. eight per character.”

An RNIB training video made the why clear.  The braille output is often used in combination with speech output and it is particularly useful for punctuation, spelling and codes. These can’t be easily heard in the speech output, at least not without seriously compromising your ability to listen to the speech comfortably. You can ask the screenreader to speak all the punctuation and spell out words but you wouldn’t always want it to be doing that. And the braille display is much more like reading, as opposed to listening which could make it easier for precision work and for remembering. The video featured a computer programmer explaining how valuable the braille display is for her when reading computer code.

They’re not cheap though. The Braille display available from the RNIB shop is £1,195.00 (Ex. VAT).

I’ve been thinking about the search functionality for our online shop this week. I’ll write up our approach to search properly at a later date but for now I thought I share the variety of search forms I’ve seen on other online shops.

Some things of note:

• The longer search boxes were mostly on book sites.
• 3 sites also offered “suggestions as you type” (Amazon, Borders, Ocado)
• Only 1 site had an obvious link to an advanced search
• All sites handled scopes with a dropdown

(Visio stencil is from GUUUI)

There’s a post on the CMS Watch Blog about the challenges of achieving a metadata-driven publishing model:

“The content needs metadata for this to work. Many will tell you that “people won’t tag.” No, seriously, they won’t tag content with the right labels, add the right metadata, or correctly categorize, “even if threatened with being fired.” And even if they do tag, it will be haphazard and inconsistent.

This is a very real problem. But at the same time it’s complete nonsense. Because if this were the case, why would people meticulously tag and file their holiday snapshots on Flickr and Facebook? Somehow, in their spare time, they do identify the people in a picture, add keywords to a shot, give it a meaningful title, and actually describe it. Without having to be threatened with being fired, or even having to be beaten with a stick.

Partly this is because they get the feedback that makes it worth their while to do so. If you identify your friends in a picture on Facebook, they (and then their friends) will immediately find it and start commenting, which creates a positive feedback loop to tag some more. More importantly though, it’s really easy.”

via Trends: Tagging your web content.

Really good stuff in this month’s FUMSI article by Ian Davis:

“Image indexing gets especially tricky, and really parts company from the world of document indexing, with the ‘aboutness’ access to images. By their nature images convey a myriad of messages to any number of people. Few images are not ‘about’ some type of abstract concept and few images users make no use of this important access point to image content”

via FUMSI – Image Findability: Improving through Tags.

I really like the fact that Ian both addresses the genuine challenges in describing ‘aboutness’ but also highlights that this is exactly what the users of image retrieval systems want.

A lot of commentators, mentioning no names,  often present cataloguing and classification as librarians imposing their view of the world on the rest of us, conveniently glossing over both the usual librarian motivation of just wanting to help and the existence of a mass users who want help and not an ontological debate.

Related articles:

• Automatic classification resources
• Metadata fundamentals article

So good friends in the UX community have expressed concerns about some of my slightly less than impressed references to “user experience”.  There was the whole penguin thing. And the using your clients language one. And the getting a bit snippy in an innocuous post about content management resources.  It is, admittedly, a bit of a bee in my bonnet (and no, I don’t own any bees before any of you ask).

But I can’t say “I’m an user experience designer”.  In much the same way that I couldn’t say “I’m a neo-conceptual artist” with a straight face.  I wasn’t raised that way.

I’m a bit embarassed to say “I’m an information architect”.  And as I said before I tend to avoid that at work.

(I’ve got a biological taxonomy metaphor I can use here but the whole Lakoff’s penguin thing went down so well…I think I better save that for the pub)

It would be a bit like me telling you my husband is a “craftsman”. It is completely accurate. It has grandeur and a philosophical sweep. It gives his career a wide scope and avoids him being boxed in by his job title.  But it doesn’t help anyone of you realise that he could make you a rather nice spoon, a lovely rustic fence or even some really good charcoal. And that he might not be the best person to ask for an earthenware pot or an Aran sweater.

What’s the IA equivalent of  “makes beautiful, useful things with wood”?