Setting up Faceted Apache Solr search in Drupal 8

Note: Extra special thanks to Doug Vann for providing motivation to finally post this blog post!

Early in 2016, when the Search API and Solr-related modules for Drupal 8 were in early alpha status, I wrote the blog post Set up a faceted Apache Solr search page on Drupal 8 with Search API Solr and Facets.

That post was helpful during the painful months when Solr search in Drupal 8 was still in a very rough state, but a lot has changed since then, and Solr-based search in Drupal 8 is in a much more stable (and easy-to-configure) place today, so I thought I'd finally write a new post to show how simple it is to build faceted Solr search in Drupal 8, almost a year later.

Build a local development environment with Apache Solr

These days I always build and maintain Drupal sites locally using Drupal VM; doing this allows me to set up a development environment exactly how I like it, and doing things like adding Apache Solr is trivial. So for this walkthrough, I'll start from square one, and show you how to start with absolutely nothing, and build faceted search on a new Drupal 8 site, using Drupal VM and a Composer file:

Download Drupal VM and follow the Quick Start Guide, then add the following config.yml inside the Drupal VM folder to ensure Apache Solr is installed:

---
# Make sure Apache Solr is installed inside the VM.
installed_extras:
  - drush
  - mailhog
  - solr

# Configure Solr for the Search API module.
post_provision_scripts:
  - "../examples/scripts/configure-solr.sh"

# Use custom drupal.composer.json to build and install site.
drupal_build_composer: true
drupal_build_composer_project: false
drupal_core_path: "{{ drupal_composer_install_dir }}/docroot"
drupal_composer_dependencies:
  - "drupal/devel:1.x-dev"
  - "drupal/search_api:^1.0"
  - "drupal/search_api_solr:^1.0"
  - "drupal/facets:^1.0"

We're going to use Drupal VM's Composer integration to generate a Drupal site that has a Composer-based Drupal install in the synced folder path (by default, in a drupal subdirectory inside the Drupal VM folder).

The drupal_composer_dependencies variable tells Drupal VM to composer require the modules necessary to get Search API Solr and Facets. The post_provision_scripts script is included with Drupal VM, and will configure the version of Apache Solr installed with Drupal VM appropriately for use with the Search API Solr module.

Copy the example.drupal.composer.json to drupal.composer.json, then run vagrant up to build the local development environment and download all the Drupal code required to get started with search.

Note: If you are setting search up on an existing site or don't want to use Drupal VM, download and install the Search API, Search API Solr, and Facets modules manually, and make sure you have Apache Solr running and a search core configured with the latest Search API Solr module version's configuration.

Install the modules

If you want to install the modules via Drupal's UI:

  1. Go to http://drupalvm.dev/, then log in as the administrator (default username and password is admin/admin).
  2. Go to the Extend page (/admin/modules), and enable "Search API", "Facets", "Solr search", and "Solr Search Defaults".

Enable Search API Solr and Facet modules in Drupal 8 UI

If you want to install the modules via Drush:

  1. Run drush @drupalvm.drupalvm.dev en -y facets search_api search_api_so lr search_api_solr_defaults

You should also uninstall the core 'Search' module if it's installed—it is not required for Search API or Facets, and will continue to store extra junk in your site's database if it is installed.

Configure the Solr server

Visit the Search API configuration page, and edit the default Solr Server, making the following changes:

  • Change 'Solr core' to collection1 (default is d8).

Note: The latest versions of Drupal VM no longer require this step, as the default Solr core's name is set to d8 automatically (see this PR).

Search API Solr search core configuration

At this point, on the server's status page (/admin/config/search/search-api/server/default_solr_server), you should see a message "The Solr server could be reached" and "The Solr core could be accessed":

Apache Solr server connection details in Search API configuration

Once this is done, uninstall the Solr Search Defaults module (drush @drupalvm.drupalvm.dev pmu -y search_api_solr_defaults); this module is no longer required after initial install, as the configuration for your Solr server is now part of the site's active configuration store and won't be deleted.

Configure the Solr index

The Solr Search Defaults module creates a default content index containing all published nodes on the website. In our case, that means all Basic pages and Articles would be included.

You don't need to change anything in the index configuration, but if you want to have a poke around and see how it's set up and what options you have in terms of the data source, fields, or processors, visit /admin/config/search/search-api/index/default_solr_index/edit.

Add default content (if you don't have any)

Assuming you built this site using Drupal VM, it's likely the site is barren, with no content whatsoever to be indexed. To fix that, you can use the Devel module's handy companion, Devel generate:

  1. Enable Devel generate: drush @drupalvm.drupalvm.dev en -y devel_generate
  2. Generate dummy content: drush @drupalvm.drupalvm.dev generate-content 100
    • Note: At the time of this writing, the Drush command didn't result in generated content. Use the UI at /admin/config/development/generate/content if the Drush command isn't generating content.

Now you have a bunch of nodes you can index and search!

Confirm Search indexing is working

It's best to let your production Solr servers wait a couple minutes before freshly-indexed content are made available to search; this way searches are a little more performant as Solr can batch its update operations. But for local development it's nice to have the index be up-to-date as quickly as possible for testing purposes, so Drupal VM's configuration tells Solr to update it's search index immediately after Drupal sends any content.

So, if you generated content with Devel generate, then visit the Index status page for the default search index (/admin/config/search/search-api/index/default_solr_index), you should see all the content on the site indexed:

100 percent of content indexed in Search API

If you're working on an existing site, or if all the content isn't yet indexed for some reason, you can manually index all content by clicking the 'Index now' button and waiting for the operation to complete.

Note that indexing speed can vary depending on the complexity of your site. If you have a site with many complex node types and hundreds of thousands or millions of nodes, you'll need to use more efficient methods for indexing, or else you'll be waiting months for all your content to be searchable!

Make a Faceted Solr Search View

The Solr Search Defaults module creates an example Views-based search page, which you can access at /solr-search/content. It should already be functional, since your content is indexed in Solr (try it out!):

Function Search Content page with Search API Solr in Drupal 8

For many sites, this kind of general keyword-based site search is all that you'd ever need. But we'll spruce it up a bit and make it more functional by changing the path and adding a Content Type Facet.

First, modify the view by visiting /admin/structure/views/view/solr_search_content:

  1. Change the Title to 'Search' (instead of 'Search Content').
  2. Change the Path to '/search' (instead of '/solr-search/content').
  3. Click 'Save'.

Second, create a Content Type facet by visiting /admin/config/search/facets:

  1. Click 'Add facet'.
  2. Choose the 'View Solr search content, display Page' Facet source (this is the view you just edited).
  3. Select 'Content type (type)' for the Facet's Field.
  4. Name the facet 'Search Facet - Content type' (this will help with placing a block later).
  5. Click 'Save'.
  6. On the Facet edit page:
    1. Check the box to 'Show the amount of results'.
    2. Check the 'List item label' checkbox (this will make the facet show 'Basic page' instead of 'page'—the label instead of the machine name for each item).
    3. Click 'Save'.

The facet is now ready to be placed in your theme so it will appear when the Search view is rendered. Visit the Block layout configuration page (/admin/structure/block), and click 'Place block' in the region where you want the facet to appear. In my theme, I chose the 'Sidebar first' region.

Find 'Search Facet - Content type' (the Facet you just created) and click 'Place block'. Then set the block title to something like 'Filter by Type', and click 'Save block'. You don't need to set specific visibility constraints for the block because the Facet is set to not display at all if there aren't search results on the page that require it to be shown.

Click 'Save blocks' on the Block layout page, and then visit your sitewide search page at /search:

Solr Search page with a basic preconfigured Facet in Drupal 8

If you perform a search, then you'll notice the facet's result counts will adjust accordingly:

Facets with result count for Drupal 8 Search API keyword search page

At this point, you should have a fully operational Faceted Solr search on your Drupal 8 site. From here, you can customize the search page further, work on outputting different results (maybe a content teaser instead of the full rendered content?), and add more facets (date, author, taxonomy term, etc.) to make the search work exactly as you'd like!

Next steps

If your hosting provider doesn't provide an Apache Solr search core for your site to use, you might want to consider using Hosted Apache Solr to host your site's Solr search core; it uses a similar setup to what's used in Drupal VM, and I can vouch for it, since I run the service :)

Note that the Search API modules are still in beta as of this blog post; minor details may result in differences from the screenshots and instructions above.

Comments

Thanks for you comprehensive instructions. I am been working my way through it and have gotten a little stuck on the schema. I have installed vagrant, virtual box and got the drupal site up and running with solr. But when I look at the solr server page in drupal, my schema is example-data-driven-schema and not the drupal-4.5-solr-5.x as in your website. I also have a notice on top of my solr server page in drupal saying "You are using an incompatible schema.xml configuration file. Please follow the instructions in the INSTALL.txt file for setting up Solr." But I can't see any reference to schema in the INSTALL.txt the mention. I have tried to see where the schema is listed in the vagrant config files, but can't see anything. Can you guide me to where I have to go to change.

I am running this on a windows10 machine. i tried with Solr 5.5 as in your vagrant file, and also tried Solr 6.3

Thanks for any help in advance.
gh

For a follow up, I solved above problem by deleting all the files in /var/solr/data/collection1/conf/ and copied all the files from /var/www/drupalvm/drupal/web/modules/contrib/search_api_solr/solr-conf/6.x/ to prior mentioned directory. This is alluded to in file at modules/contrib/search_api_solr/INSTALL.txt in the area where they talk about "Before starting solr". I did this after starting solr and just refreshed.

Thanks
gh

Hi,
Awesome article.

I have some questions for following use case.

In my earlier Drupal 7 site,
I have a [Author Name] & multiple [Pin codes] field, i have to create a test field (Autocomplete) by which user can search Author by name or by Pin code and i have to use Apache solr for that.

For above use case first i Implement hook_apachesolr_index_document_build(), here i add and attache author name & pin code value with a custom field [tm_title_pincode] with the help of $document->addField().

And with Apache solr query i use following function
$query = apachesolr_drupal_query("apachesolr");
$query->addParam('q', $keys . '*');
$query->addParam('fl', 'entity_id, label, tm_title_pincode');
$query->addParam('qf', 'tm_title_pincode');
$resp = $query->search();

it works for me , How can we create same thing with in Drupal 8. Please guide me.

Thanks in advance
Samit K

It would be helpful if you could elaborate more on what the Solr Search Defaults actually set up. The issue is currently that module requires you to have created an article and page content type. If you are using the minimal install it won't work. So I'm currently digging through the defaults module to find out what I'm missing!

Hi, thank you for the great video!
I have installed everything like in video, the only problem I have it doesn't index new content:
the difference I see in configuration:
Schema example-data-driven-schema

When I try to run index I see the errors:
olarium\Exception\HttpException while indexing: Solr HTTP error: OK (500) .... "msg":"Unable to invoke function processAdd in script: update-script.js: Can't unambiguously select between fixed arity signatures [(java.lang.String, java.lang.String), (java.lang.String, java.io.Reader)] of the method org.apache.solr.analysis.

And
Drupal\search_api\SearchApiException while trying to index items on index Default Solr content index: Solr HTTP error: OK (500) ...Can't unambiguously select between fixed arity signatures [(java.lang.String, java.lang.String), (java.lang.String, java.io.Reader)] of the method org.apache.solr.analysis.TokenizerChain.tokenStream for argument types [java.lang.String, null]","trace":"org.apache.solr.common.SolrException: Unable to invoke function processAdd in script: update-script.js:

How can I fix them ?
Thanks!

As always, what a great job Jeff!

Just want to post how I resolved a strange issue with the Solr core.

I had "An error occurred while trying to retrieve additional information from the Solr server: Solr endpoint http://localhost:8983/solr/d8/ not found." in admin/config/search/search-api/server/default_solr_server

To resolve this issue:

1. Go to edit the Search Server in admin/config/search/search-api/server/default_solr_server/edit
2. Under Configure Solr Backend, and there under Configure Standard Solr connector, change the value of the field: "Solr core" form "d8" -> "collection1"
(this is normally the default core of Solr)
3. Save

That's it. This worked for me.

Once again, thx Jeff for the great job!

Karim,

Thanks for the blog post, the tip of naming the facet block correctly (Search Facet - xxx) is important as I was not able to find it with a less-specific name.

It would be helpful to have a post like this but for those who do not use a local dev environment or drupal vm.

For those coming to this after 2017-03-09: Drupal VM version 4.3.0 changed build_composer to drupal_build_composer and build_composer_project to drupal_build_composer_project.

Thank you for this great video Jeff. I am new to Drupal 8. I have developed on Drupal 6 years ago and had let Drupal as my companie prefered to work with Symfony. Now I'm back to Drupal. Would you know if it is possible to create a faceted Apache Solr Search having paragraphs module fields as facets ?

Does anyone know, how to create a facet based on boolean field?

This is great. Thanks. In Drupal 7 I would customize the layout of my search page using Display Suite Search pages. This is not in D8. Is there a way you recommend to be able to edit the layout of the SOLR search page in display suite manage display page or some other way. I see the Search API Pages module but it looks like it is in alpha. Thanks. [email protected]

this looked promising.... I installed everything, but at the end I got this error:

fatal: [drupalvm]: FAILED! => {"changed": false, "content": "", "elapsed": 0, "msg": "Status code was -1 and not [200]: Request failed: ", "redirected": false, "status": -1, "url": "http://localhost:8983/solr/admin/cores"}
RUNNING HANDLER [geerlingguy.apache : restart apache] **************************

RUNNING HANDLER [geerlingguy.mysql : restart mysql] ****************************

RUNNING HANDLER [geerlingguy.solr : restart solr] ******************************

PLAY RECAP *********************************************************************
drupalvm : ok=177 changed=11 unreachable=0 failed=1 skipped=293 rescued=0 ignored=1

Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

As a followup to my previous comment, it does appear that installing without SOLR works fine... it's just some problem with installing and/or configuring SOLR.