Adding suggestions to a 404 page with pagefind

So I’ve gone into how I implemented search¹ on this site, now I want to explain how I’m also using pagefind² to dig up article suggestions on the 404 page (i.e. when someone visits a URL on this site that doesn’t exist). This isn’t super specific to Jekyll I don’t think, it should work anywhere you are using pagefind.

First of all I must seek forgiveness from the internet gods - over the years my URLs have changed and links have broken over time. I set up redirects but maintaining a huge list of them just isn’t something I can keep on top of forever. Plus if I changed webserver I have to port all the redirects over. Yes I am aware this makes me a bad person.

There is the next-best-thing though which is what we are doing here - using our little search engine to give suggestions of possible correct pages.

This is actually something that’s nice to do regardless of my URL changing sins - it helps people out who might have made a typo in the URL (people type URLs out manually… right? Anyone?).

Include the page slug in the search index

In this context the ‘slug’ is the bit at the end of the URL that sort of mimics the title of the article. e.g. on this page the slug is 404search. It’s important that we can search using this slug, but unless you write it in every post, pagefind doesn’t know what it is by default.

So the first step is to add the slug to every post, and hide it. In jekyll it’s

<div class="hidden">404search</div>

Where ‘hidden’ is a class I have set up that hides the div using css. Make sure it is inside an element that is inside your pagefind indexed stuff, like <main data-pagefind-body>. Or you can just add data-pagefind-body to the hidden div and it will add it to the index along with your main content.

Now you can search for posts using the slug, as well as normal search terms. Handy!

Set up your 404 page

I’m just gonna dump the code for my 404 page at the bottom of this article but in short we are going to use some javascript now to:

1. Take the slug from the URL
2. Send it to pagefind using their API
3. Render a list of search results that include the slug
4. If there is only one result, redirect to that page

The hit-rate for this seems to be pretty good!

• Articles where the URL format changed always work, so https://www.bfoliver.com/film/2017/11/20/theedgeofseventeen/ brings up https://www.bfoliver.com/2017/theedgeofseventeen.
• Articles where the slug changed still have a good chance of showing up (although I don’t think I’ve ever changed a slug).

Some notes and caveats

• Articles with the same slug will return multiple results. This could be fixed by including the date in the search index but I’m not keen on over-complicating it.
• It’s not ‘perfect’ for finding typos in URLs, but it’s better than nothing and to be honest I suspect very few people are typing URLs out now.
• You still need javascript enabled
• Auto-redirecting is probably an iffy choice I may or may not get rid of. It’s fine for searching slugs (where the slug is unique, otherwise there is a decent fallback) but it can also turn up weird results if you type just single words into the address bar (for example https://www.bfoliver.com/fwfff seems to lead to Nosferatu: A Symphony of Horror³ but that’s ok)

The 404 page code

---
layout: page
permalink: /404
title: "Error 404"
---

<div id="suggestions" class="hidden">
  <h2>You might have been looking for:</h2>
</div>
<script>
    document.addEventListener('DOMContentLoaded', async () => {
      const pagefind = await import('/pagefind/pagefind.js');
      await pagefind.init(); // Initialize Pagefind
      suggestPages(pagefind); // Call the suggestion function
    });
    async function suggestPages(pagefind) {
      // Extract the page slug from the pathname
      const pathSegments = window.location.pathname.split('/');
      const slug = pathSegments.filter(segment => segment.trim() !== '').pop() || '';
      // Perform the search using the slug as the search term
      const search = await pagefind.search(slug);
      // Load data for a subset of results
      const maxSuggestions = 5;
      const results = await Promise.all(search.results.slice(0, maxSuggestions).map(r => r.data()));
      if (results.length === 1) {
        window.location.href = results[0].meta && results[0].meta['url'] ? results[0].meta['url'] : results[0].url;
      } else {
        displaySuggestions(results);
      }
    }
    function displaySuggestions(results) {
      const suggestionContainer = document.getElementById('suggestions');
      // Remove the 'hidden' class to allow visibility
      suggestionContainer.classList.remove('hidden');
      if (results.length === 0) {
        suggestionContainer.innerHTML = '<p>No suggestions available.</p>';
        return;
      }
      suggestionContainer.innerHTML += ''; // Clear previous content
      results.forEach(result => {
        // Use metadata if available
        const url = result.meta && result.meta['url'] ? result.meta['url'] : result.url;
        const title = result.meta && result.meta['title'] ? result.meta['title'] : 'Untitled';
        const date = result.meta && result.meta['date'] ? result.meta['date'] : 'No date available';
        const suggestion = document.createElement('div');
        suggestion.innerHTML = `
          <h3><a href="${url}" data-pagefind-meta="url[href]">${title}</a></h3>
          <p>${result.excerpt}</p>
          <p class="small-caps lowercase">${date}</p>
        `;
        suggestionContainer.appendChild(suggestion);
      });
    }
  </script>