We use search engines daily. But are they really bringing us back what we need? Or do we end up spending more time searching through them than finding the results we need?
Here are some tips on which search engines to use when, and how to use them effectively for background research and due diligence.
The best search engines
Let’s face it, Google is great. But its commercial search engine results page (SERP) features such as Ads, Knowledge Graph, Snippets and Map Pack can crowd out relevant results, making it tricky to find the information you need.
It’s also Western-focused and might not be the best platform when dealing with international subjects.
Here are some of the other search engines you need to know about:
Yandex
Best for Russia and CIS coverage. Google is still used in Russia, but Yandex may well find you that crucial piece of information you are missing.
Baidu
Best for China coverage. Baidu has a better understanding of Chinese language and culture than other search engines. It takes into account the context in which words are used to generate the most relevant results. Other search engines have struggled with this in China.
Baidu’s search engine generates results in Simplified Chinese.
Bing
Best for image search. Bing is able to understand if a search is about a person or thing and then show relevant images. This is helpful to verify information and get a better sense of your subject.
Yandex (above) also has a good reverse image search function. It’s definitely worth checking if your image could have been taken in Russia.
Duckduckgo
Best for privacy and a clean perspective.
Google gives a personalised search experience, based on the data it collects. DuckDuckGo doesn’t collect your data, and as a result, the search results will be the same whoever is searching. This can help when looking to get a sense of what someone else might see if they decided to go digging on you or you subject.
Challengers
The landscape is changing no matter where you are. New, disruptive search engines such as Ecosia, Neeva and Mojeek are challenging Google’s primacy. Ecosia offers an eco-friendly search, promising to use all revenues to plant trees. Subscription-based Neeva promises fewer ads and greater privacy. Whereas Neeva, DuckDuckGo and Ecosia license their web index from Microsoft’s Bing, Mojeek wants to build its own index, to become truly independent.
Top tip
When people are concerned about their reputations, they’ll likely have a word with Google to take down certain pages. But they’re unlikely to think about the others. Consider using a range of different engines to get the full picture.
Insights, not information
The biggest challenge with search engines is trawling through the millions of results to find the information you need. To help, organisations often set limits on the number of pages to visit when conducting due diligence and background research. But what if that critical piece of information was on page 15, and not page 10?
Someone else might find that nugget and it might come back to bite you.
Finding insight out of information isn’t just about sheer tenacity. It’s about clever searching.
The key is to refine your result set to get a manageable number, whilst not missing what you need. At different stages in your research you might need to narrow, or expand your search. When you start out you want to explore your options and see what might or might not be relevant to your subject to build your understanding. But then as you learn, you’ll want to narrow in on specific areas.
-
Draw up a list of initial search ideas. Brainstorm with colleagues to build a diverse and comprehensive search.
-
As you find new information, expand by using ‘or’ and alternative search terms
-
Narrow by using:
– Date range filter
– Phrase searching
– Search operators
-14.png)
Smart searching
Whilst Google is the main supporter of Boolean searching, a number of the other platforms use similar structures. This includes social media.
We all know about:
-
And/+ (expand)
-
Or (expand)
-
Not/- (exclude)
Here are some more essential tools for researchers:
Quote marks and the minus sign
Quotation marks (“”) and the minus sign (-) are the two most important operators for running queries.
Put something into quotation marks and the results will include exactly what you wrote. For example: “Homer Simpson” will not return the Greek poet Homer.
Put something with a minus sign in front of it to exclude it from the search. For example: Homer -Greece -Greek -Poet -Iliad -Odyssey.
Wildcard
The wildcard operator, which is an asterisk (* or shift + 8), substitutes an unknown variable for your search.
This operator can represent a single character, a word, or a whole phrase. It can be used if you don’t know a subject’s middle, first or last name, but you do have other information which will help you search for them. It can also be used for partial dates of birth, if you experiment with the right date format.
Filetype
The filetype: function will return back results that are for this exact filetype, plus the search term or site you list, such as filetype:pdf, XLS, XLSX, Doc etc. This can help you to find official lists and records on otherwise confusing sites.
Site
Site: will find only information from a specific site. This can be really helpful for sites that don’t include their own search function. For example, if I want to find out about risk on Xapien’s site, I can put in site:www.xapien.com “risk” and I can jump straight to the pages I’m interested in.
Verification
Finding ‘golden nuggets’ of information is all very well but unverified insights can easily lead you astray. It seems obvious but a critical mindset is essential. Even the most logical, intelligent thinkers can be convinced by the conspiracies and rumours that abound the internet. The best way to safeguard against this is by applying process to your verification.
How can you tell whether the claims made in blogs, articles and other sites are true?
-
Triangulate: Use more than one source when others are available to get different perspectives.
-
Cross-check: Confirm key information points from other authoritative sources.
-
Use original sources: When available, use original sources, don’t rely on secondary sources. If a media article refers to a corporate insolvency, can you check this on the original record?
-
Owners, motivations, limitations: Identify who is behind the source and whether their reporting is restricted or biased in any way. Understand the story the source is presenting and whether it is attempting to persuade or influence. This can skew results. Attempt to identify any signs of bias, whether obvious or subtle, in how the source presents its information.
The bottom line
The web is a vast bank of resources, but information overload can obscure critical insight. Search engines help to process and organise online data, but the buck doesn’t stop there.
A wide array of sources, clever searching, and process-led verification can help with any background research. But tenacity and time remain essential.
Automated solutions like Xapien can help. Xapien’s powerful natural language processing and disambiguation engines can triangulate insights from thousands of sources in a few minutes.
Make online data work for you.
-01%201.png)