My main motivation for talking at Essex (and at City University a couple of weeks later) will be to inspire students to consider working in the world of open source software and more specifically the commercial applications of what academics call information retrieval – search engines.  It’s an interesting field to work in – we have clients in a huge variety of sectors including e-commerce, law, publishing and government; we deal with both small startups and multinational businesses and help build systems indexing a few thousand to several billion items. It’s constantly changing as new requirements, ideas and innovations appear. It’s taken our staff around the world (Singapore, Malaysia, the USA, Denmark as a small sample from the last couple of years) and led to us gaining a global reputation and becoming part of a select group of independent search specialists. From being somewhat of a curiosity when we started, open source search engines have now gained huge acceptance and have changed the search market beyond recognition – no longer can vendors charge six or seven figures for mysterious black boxes (and more to make them actually do something useful).

However our sector needs more people – not just developers, but business-focused search managers who understand how to build search engines that truly deliver value to employees and customers. As I’ll say to the students next week there’s a skill shortage, plainly illustrated by the plaintive slide that ends nearly all search conference and Meetup presentations – “We’re Hiring!”. Time to learn to code, download Lucene/Solr or Elasticsearch, try out the examples, read our book and look forward to a great career in search!

The post Inspiring students to work in Open Source Search appeared first on Flax.

Udo Kruschwitz was next with a brief treatment of his ongoing work on automatically extracting domain knowledge and using this to improve search results (for example see the ‘Suggestions’ on the University of Essex website) – he showed us some of the various methods his team have tried to analyze query logs, including Ant Colony Optimisation (modelling ‘trails’ of queries that can be reinforced by repeat visits, or ‘fade’ over time as they are less used). I liked the concept of developing a ‘community’ search profile where individual search profiles are hard to obtain – and how this could be simply subdivided (so for example searchers from inside a university might have a different profile to those outside). The key idea here is that all these techniques are automatic, so the system is continually evolving to give better search suggestions and hints. Udo and his team are soon to release an open source adaptive search framework to be called “Sunny Aberdeen” which we look forward to hearing about.

The evening ended with networking and a pint or two in traditional fashion – thanks to both our speakers and to all who came, from as far afield as Milton Keynes, Essex and Luton. The group now has 70 members and we’re building an active and friendly local community of search enthusiasts.

The post Cambridge Search Meetup review – Two different kinds of university search appeared first on Flax.