How to Conduct an In-depth
Keyword Research

What is Keyword Research

Keyword research is the process of identifying which terms or phrases your target audience use to search for your goods, services or content in search engines and how frequently these terms or phrases are used. This user behaviour analysis helps you identify which keywords to target and evaluate if your content is satisfying the searcher’s intent.

Why Do You Need Keyword Research?

1. Identify what key terms, phrases your target audience uses to search for your products

Keyword research is often the best place to start your SEO campaign as it enables you to identify what terms your website should be targeting and what your competition is. This is usually the first step in any SEO campaign.

Sometimes clients come to us with a very brand-centric view of their users (we see this very often with high-end brands). They have a vision – different personas of who their users will be and naming conventions for products or services are crafted with this person in mind. This isn’t necessarily the wrong approach as every company should have a unique selling point. However, there is often a disparity in how they believe their customers will search for their products (they often believe search would be done in a very linear way; from the home page down to products) and how their customers’ actually search. Keyword research provides you with the data to challenge this belief and helps you find a balance between brand positioning and search demand.

“Ranking isn’t always the goal, your site can rank well but if nobody searches for it, that’s not really useful. From that, great content alone, even if it’s world-class, does not mean you’ll get great traffic….”

John Muller

2. Analyse and optimise your contents relevancy for frequently used key terms or phrases

The industry has come a long way from keyword stuffing/frequency counts.  Google is better at understanding natural language and penalizing bad spammy content. However, most pages that rank for a keyword/phrase often have that keyword or phrase included in the content because it is very difficult to write about a keyword without mentioning said keyword. Also, we have been conditioned to search for information with head terms and click on the first results that include these terms.  

In this digital age, our attention span is very low. Instant answers have also changed searchers expectation. Users expect answers quick and fast. They do not want to scan long articles/ product pages to find information. Including the keywords your users are searching for high up on your page (above the fold) signpost that they are in the right place and your page/content will answer/fulfil their demand. It also helps search engines understand what your page is about.

3. Discover new opportunity that you might be missing out on

  • Are there other similar or related products you could be targeting?
  • Should your filtered pages be optimized and indexable?
  • Are your consumers researching your product?
  • Do they need additional information to buy?
  • Keywords research will help you answer these questions.

4. Identify your online competition and get/set a budget

Clients often come to us with a long list of who their direct competitors are. They, however, have very little understanding of their search competition. They wrongly assume the competition will only be from the business of a similar size and revenue. Keyword research helps you identify who their search competition is so they can understand how difficult/easy it is going to be to target major generic keywords and what budget they need to put behind the campaign to make an impact in the market.  

Sometimes you might need to go after a less competitive keyword to have any chance at gaining visibility in a highly competitive market. Keyword research helps you understand the search landscape and make these important decisions.

5. Create the optimum Information Architecture (IA) for your site

When building an SEO friendly site architecture, you should plug in information from your keyword research in your decision-making process.  Your site’s architecture informs search engines on how to priorities your site’ pages.  Keyword research identifies which pages should be closest, to the homepage, which category/subcategory or filter pages you should create and how to organise the rest of your content.

6. Identify keyword cannibalisation

Keyword cannibalization is when you target the same/similar keywords across multiple pages. This creates internal competition and search engines struggle to decide what page to rank for what term.

Keyword research helps you identify these pages. From your analysis, you can decide which page to de-optimise and what to consolidate for better authority and relevancy.

How to Conduct Keyword Research

1. The first thing to do when starting your keyword research is to get a list of search terms, phrases and questions that your target audience uses to find you.  You can get these from:

  • Frequently Asked Questions

If you are not building a new site, your current customers are an invaluable resource for keyword research. Find out how they discovered your site, what terms they used if they came from a search engine. Also, questions frequently asked your customer service team is a great resource for creating additional content on your site.

We are often approached by businesses with this particular challenge. They want to:

  1. Reduce the number of calls to customer service.
  2. Increase the number of customers completing tasks online.
  3. Journey to task completion online to be simple and quick. 

Finding out the pitfall costumers experience directly from them is the first and often most useful place to start.

  • Site Search

Do you have a search function on your site? Do your visitors use this often?  Your site search function can provide useful data about visitors to your site.  If your site visitors rely heavily on your site search it is likely an indication that some of your products/services are difficult to find via your main navigation. Harvesting this information is useful to determine keywords visitors associate with your site and information they really want to find (they are patient enough to keep searching for that information on your site and not go to a competitor) so you can priorities this information in your IA or create new landing pages if required.

To ensure you can tap into this wealth of information track your site search function in Analytics (GA). To do this go to the Admin tab in GA, select the property and the view that you want to set up site search for, scroll down to site setting and turn site search tracking on. Don’t forget to add the query parameters of your site search URL to the query parameter field.

  • Competitor Keywords

Find out what keywords your search competitors (especially the big players) already rank for. In all probability, they have done some research and are targeting some of the juicy keywords. No need trying to reinvent the wheel here when you can save some time. Clean up the list by removing products and services that your business doesn’t offer.

Competitor keywords are especially useful for a site that is just starting without any keyword history. Other sites can also use competitor research to identify further opportunity.

  • Google Search Console (GSC)

GSC is a valuable resource for uncovering how Google sees your site. It is free so why not.  Your ranking data in GSC will help you identify underperforming keywords. You can also discover keywords that Google see your site as relevant for that you might have missed.

  • Current Rankings

Even if you haven’t optimised your site or conducted any keyword research in the past, you could still have picked up a couple of rankings so make sure you include these in your research as you don’t want to de-optimise your site for keywords that are already relevant or performing well.

  • Your Website

Get the product/service descriptions on your website. The purpose of this research is to determine if you are targeting the right keywords and uncover further opportunity so you will need these keywords too.

  • Answer The Public

I use this Answer the public when doing content research for featured snippets and FAQ pages.  I find it useful for generating question and long-tail queries.

2. Expand your list using Google Keyword Planner Tool or any bulk keyword tool (to get other related keywords).

For this next step, you will need a bulk keyword suggestion and volume tool depending on how many keywords you already have. Unfortunately, most of them are paid tools.

I use Google Keyword Planner Tool. Technically it is free (I say technically because you need an active Google AdWords account to get average monthly search data.  Without an active AdWords account, you only get shown a range). This is this useful for assessing how competitive a keyword is.

Google Keyword Planner Tool - Showing Search Volume Range and Competition
Google keyword planner – search volume range and competition

Example of Google's keyword Planner Tools Interface
Google keyword planner interface

I also like this tool because you get keyword search volume and suggested keywords (discover new keywords) at the same time.

Google's Keyword Planner Tool -Discover new keywords
Google keyword planner – discover new keywords

If you do not already have an extensive list. You can start building your list here. Type your main head term and enter a domain to use as a filter and you get a suggested list of keywords. You can also start with the URL of a website (no keywords) and based on the target keyword of the website discover new keywords.

Google's keyword planner tool - Get search volume and forecast
Google keyword planner – get search volume

Alternatively, if you have been through the previous steps and already have a massive list, you can jump straight in to get keyword volume and forecast. Copy and paste your keywords (limit of 800) or upload a CSV file.

Keyword shitter is also great for expanding your list. With the free version, you type in a few keywords and it literarily shits out loads more, however, this list does not include the search volume.  

3. Using excel to clean up your data (remove keywords you don’t care about, incidental keywords etc)

The next step is cleaning the data. I use excel for this. Dump all your keywords into excel and dedupe. Remove any suggested keyword that you don’t care about and any incidental keywords that you are ranking for but aren’t related to your product.

I know some people still target misspelling but with auto-suggest, Google gives you results for the correct spelling anyway, so I take misspellings out. Also, mistakes in your copy reduce the credibility and EAT of your site so I do not include mistakes in my copy.  

4. Categories your keywords

Put your keywords into buckets and groups. It is impossible to target every keyword variation. Google has gotten better at understanding natural language (BERT) so you don’t need to create content or optimisation for every synonym of a keyword.  Categories also help you understand trends, patterns and priorities.

The categories you assign to your keywords depends on what you want to do with the outcome of your research. When onboarding a new client my categories tend to be very granular because the document is used to inform the site’s IA, content opportunities, optimisations project and depending on the site (usually large ecommerce sites with dynamically generated sitemaps) what URLs to prioritise for sitemap inclusion and indexation etc.  I know it can be time-consuming and a bit overwhelming, however, you do not have to action all the recommendations from the research at once. This should be written into your strategy and planned out over a couple of months.

You may, however, want to include broader categories like informational query, navigational query and transactional query. This classification helps when deciding what type of content to create.

This research can also be referenced as a benchmark when evaluating how much the client’s visibility has grown when presenting a performance report. Sometimes we forget to refer to the campaign start documents focusing on month to month growth which might not seem that substantial. However, looking at the benchmark document gives you a better view of how momentous your efforts have been.

When performing keyword research for the large site (mostly ecommerce sites) I like to break up the categories into different excel tabs. This makes sorting the keywords and writing recommendations easier.

5. Map keywords to landing pages

If the website you are performing the keyword research for already has some visibility in the SERP, map the landing pages to the keywords in your sheet.  Download your ranking from any of the tracking tools you have available to you. I use a VLOOKUP to map the keywords to URLs

6. Identify targets, underperforming landing pages, keyword groups and further opportunity

Example of sorted keyword categories
Example of Keyword Opportunity by category

This is when the fun beings. Sort your data. This largely depends on what categories you have assigned your keywords.  I recently did some research for a natural health site, my categories were; condition, method of delivery (powder, tablets, syrup), gender & age (man, woman, baby, kid). With these, I was able to identify category page opportunity, what conditions we needed to write support content for, pages that needed information content but were being targeted with transactional content,  categories we should be highlighting in the navigation etc. 

If you are conducting this research for your website this is the time to create an implementation plan and begin your work. However, if this is for a client, you need to present these recommendations to the C-suite or someone who has little understanding of SEO, I strongly recommend not sending the excel document with all the keywords and categories.

We do not send the keywords research documents to clients even though it has a lot of useful information except for when they ask for it. It is often too difficult for the client to go through it and understand what actions they should take. Recommendations are presented in PowerPoint or Word highlighting the trends, issues, recommendations, and next steps.

7. Select keyword groups, you care about and monitor

Select a few keywords from each category or keyword cluster to monitor. I usually track at least 100 keywords (depending on the size of your site). It is not a good idea to only select the keywords with the highest search volumes, this doesn’t give you an overview of the entire site.  

Finally, whilst it is good practice to monitor keywords you care about. You should do so cautiously.  There are so many factors that affect your rankings; personalisation, local search, the location of your tool’s crawler, the searcher’s language, privacy settings etc.  Focus on trends and not position fluctuations. Always overlay your ranking data with your site visitor and revenue trends before you panic.

Now that you know which keywords to target and have uncovered lots of content opportunities, use our free Google Analytics Audit Template to audit your GA account so you can effectively monitor your progress.

Like what you see? Lets talk. Get In Touch