Elasticsearch in Vapor: Getting Started
In this tutorial, you’ll set up a Vapor server to interact with an Elasticsearch server running locally with Docker to store and retrieve recipe documents. By Christian Weinberger.
Sign up/Sign in
With a free Kodeco account you can download source code, track your progress, bookmark, personalise your learner profile and more!
Create accountAlready a member of Kodeco? Sign in
Sign up/Sign in
With a free Kodeco account you can download source code, track your progress, bookmark, personalise your learner profile and more!
Create accountAlready a member of Kodeco? Sign in
Sign up/Sign in
With a free Kodeco account you can download source code, track your progress, bookmark, personalise your learner profile and more!
Create accountAlready a member of Kodeco? Sign in
Contents
Elasticsearch in Vapor: Getting Started
30 mins
- Getting Started
- Running Elasticsearch in Docker
- Using Elasticsearch
- Understanding Indexes and Documents
- Comparing Elasticsearch to Relational Databases
- Implementing CRUD
- Creating a Recipe
- Connecting Elasticsearch to Your API
- Reading a Recipe
- Updating a Recipe
- Getting All the Recipes
- Deleting a Recipe
- Searching Recipes
- Testing Search
- Adding Weighted Search
- Cleaning Up
- Where to Go From Here?
Updating a Recipe
Updating a recipe requires a mixture of what you’ve learned so far. You have to specify both the id
of the document you want to update in the URL and the updated recipe in the request body. To update a resource, use the PUT
HTTP method.
id
doesn’t exist, Elasticsearch will create the document instead.
Now, in ElasticsearchClient.swift, replace the fatalError()
line in updateDocument(_:id:in:)
with the following:
let url = baseURL(path: "/\(indexName)/_doc/\(id)")
var request = try ClientRequest(method: .PUT, url: url, body: document)
request.headers.contentType = .json
return try sendRequest(request)
Next, in RecipeFinderController.swift, add this new handler below the previous one:
func updateHandler(_ req: Request) throws -> EventLoopFuture<Recipe> {
let recipe = try req.content.decode(Recipe.self)
guard let id: String = req.parameters.get("id") else { throw Abort(.notFound) }
return try req.esClient.updateDocument(recipe, id: id, in: "recipes").map { response in
return recipe
}
}
Finally, add the new route to boot(routes:)
:
routes.put(":id", use: updateHandler)
You’re now ready to update a recipe with the id
. Replace the id
25TxfXEBLVx2dJNCgFQB
with your recipe’s id
value, then run this command in the terminal:
curl -X "PUT" "http://localhost:8080/api/recipes/25TxfXEBLVx2dJNCgFQB" \
-H 'Content-Type: application/json; charset=utf-8' \
-d $'{
"instructions": "Mix the two apples with one cake.",
"name": "Apple Cake",
"ingredients": [
"Apple",
"Cake"
],
"description": "As easy as apple pie."
}'
Here, you’re changing the instructions
to use two apples. Vapor displays the updated recipe:
{"ingredients":["Apple","Cake"],"name":"Apple Cake","instructions":"Mix the two apples with one cake.","description":"As easy as apple pie."}
Next, you’ll find out how to see all of the documents you keep in an Elasticsearch index.
Getting All the Recipes
To get all the documents of an index in Elasticsearch, use the _search
endpoint with no search query parameters. You’ll try this next.
In ElasticsearchClient.swift, look at the getAllDocuments(from:)
method signature:
public func getAllDocuments<Document: Decodable>(
from indexName: String
) throws -> EventLoopFuture<ESGetMultipleDocumentsResponse<Document>>
This time, the response is type ESGetMultipleDocumentsResponse
, which contains a list of hits, each representing a ESGetSingleDocumentResponse
.
First, replace the fatalError()
line in getAllDocuments(from:)
with the following:
let url = baseURL(path: "/\(indexName)/_search")
let request = ClientRequest(method: .GET, url: url)
return try sendRequest(request)
This code is similar to what you did earlier for create except you use GET
for searching the documents.
You know the drill by now: Add the handler below your previous one in RecipeFinderController.swift:
func getAllHandler(_ req: Request) throws -> EventLoopFuture<[Recipe]> {
return try req.esClient
.getAllDocuments(from: "recipes")
.map { (response: ESGetMultipleDocumentsResponse<Recipe>) in
return response.hits.hits.map { doc in
let recipe = doc.source
recipe.id = doc.id
return recipe
}
}
}
Finally, update boot(routes:)
with the new route:
routes.get(use: getAllHandler)
Build and run, then call GET /api/recipes/
via cURL:
curl "http://localhost:8080/api/recipes"
As a result, you’ll receive an array of all the recipes you’ve stored in Elasticsearch:
[{"ingredients":["Apple","Cake"],"id":"3JT0fXEBLVx2dJNCr1QE","name":"Apple Cake","description":"As easy as apple pie.","instructions":"Mix apple with cake."},
{"ingredients":["Apple","Cake"],"id":"25TxfXEBLVx2dJNCgFQB","name":"Apple Cake","description":"As easy as apple pie.","instructions":"Mix the two apples with one cake."}]
These results include one recipe created with Insomnia and one created and updated with cURL
.
So, what if you want to get rid of a recipe? You’ll find out how to delete unnecessary entries next.
Deleting a Recipe
To delete a recipe, use DELETE
with the same URL you used to update and read a resource.
First, in ElasticsearchClient.swift, replace fatalError()
in deleteDocument(id:from:)
with the following:
let url = baseURL(path: "/\(indexName)/_doc/\(id)")
let request = ClientRequest(method: .DELETE, url: url)
return try sendRequest(request)
This code is similar to what you did earlier to create a document with two differences: This time, you use DELETE
instead of POST
, and the URL includes the id of the document to delete.
Next, in RecipeFinderController.swift, add the handler:
func deleteHandler(_ req: Request) throws -> EventLoopFuture<HTTPStatus> {
guard let id: String = req.parameters.get("id") else { throw Abort(.notFound) }
return try req.esClient.deleteDocument(id: id, from: "recipes").map { response in
return .ok
}
}
Add the route to boot(routes:)
:
routes.delete(":id", use: deleteHandler)
Now, you can delete a recipe with DELETE
:
curl -X "DELETE" "http://localhost:8080/api/recipes/25TxfXEBLVx2dJNCgFQB"
Remember to replace the ID with the ID of your recipe. To be sure it worked, use your get all command to check that the recipe is gone:
curl "http://localhost:8080/api/recipes"
Sure enough, there’s only one recipe now:
[{"ingredients":["Apple","Cake"],"id":"3JT0fXEBLVx2dJNCr1QE","name":"Apple Cake","description":"As easy as apple pie.","instructions":"Mix apple with cake."}]
Congrats! You’ve made it through CRUD. Now, you can focus on the most interesting part of Elasticsearch: search.
Searching Recipes
Elasticsearch provides you with two options to search documents:
Creating a Query String
In this section, you’ll add a simple search to the recipe finder. It will look for the search term in name
, description
and ingredients
. It will also allow partial searches. For example, a search for app or pple will return apple recipes.
Start by going to ElasticsearchClient.swift and replacing fatalError()
in searchDocuments(from:searchTerm:)
with the following:
The endpoint for searching documents is under {indexName}/_search
.
You’ll need to provide a query string to the parameter q
. In the next section, you’ll see how to construct that query string.
The query string parameter has its own syntax, which lets you customize the search and filter field names. It allows you to use wildcards and regular expressions.
Here are some relevant examples and operators:
- URI Search: A simple search using URI query parameters. You’ll focus on this option for this tutorial.
- Query DSL: This option allows for complex and powerful searches using Elasticsearch’s Query DSL syntax. This tutorial doesn’t cover Query DSL because it could easily fill its own tutorial.
In this section, you’ll add a simple search to the recipe finder. It will look for the search term in name
, description
and ingredients
. It will also allow partial searches. For example, a search for app or pple will return apple recipes.
Start by going to ElasticsearchClient.swift and replacing fatalError()
in searchDocuments(from:searchTerm:)
with the following:
let url = baseURL(
path: "/\(indexName)/_search",
queryItems: [URLQueryItem(name: "q", value: searchTerm)]
)
let request = ClientRequest(method: .GET, url: url)
return try sendRequest(request)
The endpoint for searching documents is under {indexName}/_search
.
You’ll need to provide a query string to the parameter q
. In the next section, you’ll see how to construct that query string.
Creating a Query String
The query string parameter has its own syntax, which lets you customize the search and filter field names. It allows you to use wildcards and regular expressions.
getAllDocuments(from:)
.Here are some relevant examples and operators:
getAllDocuments(from:)
.
let url = baseURL(
path: "/\(indexName)/_search",
queryItems: [URLQueryItem(name: "q", value: searchTerm)]
)
let request = ClientRequest(method: .GET, url: url)
return try sendRequest(request)
-
Single term:
name:cake
finds results that have cake in thename
field. -
Multiple terms:
name:(cake OR pie)
finds results that have cake or pie inname
. -
Exact match:
name:"apple cake"
finds results that have apple cake inname
. -
Fuzziness:
name:apple~
finds results that have apple inname
. It works even for misspelled queries, like appel. -
Wildcard:
name:app*
finds results that have words starting with app inname
. -
Multiple fields:
(name:cake AND ingredients:apple)
finds results that have cake inname
and apple iningredients
. -
Boosting:
(name:cake^2 OR ingredients:apple)
does the same as Multiple terms, but hits inname
are boosted and are more relevant.
A full description of the query string syntax is available at Elasticsearch’s query string syntax reference page.
Now that you understand your options, you’re ready to construct your searchTerm
. In RecipeFinderController.swift, add this method below deleteHandler(_:)
:
func searchHandler(_ req: Request) throws -> EventLoopFuture<[Recipe]> {
// 1
guard let term = req.query[String.self, at: "term"] else {
throw Abort(.badRequest, reason: "`term` is mandatory")
}
// 2
let searchTerm = "(name:*\(term)* OR description:*\(term)* OR ingredients:*\(term)*)"
// 3
return try req.esClient
.searchDocuments(from: "recipes", searchTerm: searchTerm)
.map { (response: ESGetMultipleDocumentsResponse<Recipe>) in
return response.hits.hits.map { doc in
let recipe = doc.source
recipe.id = doc.id
return recipe
}
}
}
Here, you:
- Make sure the user provided a
term
. - Compose
searchTerm
using the query string syntax to findsearchTerm
in name, description or ingredients, and allowing for partial matches. - Use the same implementation as in
getAllHandler(_:)
.
Next, add the new route to boot(routes:)
:
routes.get("search", use: searchHandler)
That’s it! Your search should be ready now, but to be sure, you’ll test it in the next section.