14. Natural Language Classification

Written by Alexis Gallagher

Earlier in the book, you learned how to classify images — for example, judging whether they were of cats or dogs. You’ve also classified sequences of sensor data as device motions. Text is just another kind of data, and you can classify it as well. But what does a class of text look like?

Is this email legitimate or spam? Are customer messages praising your great work or demanding action to address complaints? What’s the topic of an article, patent or court document? These are just a few examples of text classification tasks.

There are a wide variety of techniques for extracting useful information from text, all falling under the general term natural language processing (NLP). This chapter focuses on using NLP for classification, specifically using the methods Apple provides as part of its operating systems. You may be familiar with NSLinguisticTagger, which has been available since iOS 5. It supports several NLP tasks and was covered in the “Natural Language Processing” chapter of our iOS 11 by Tutorials book, when Apple rewrote the class to take advantage of Core ML. This chapter does not use that class.

Apple introduced the new Natural Language framework in iOS 12 — and in each of its other device OS revisions that same year — which is meant to improve upon and replace NSLinguisticTagger. That’s the framework you’ll use here, along with Create ML to train your own models.

In this chapter, you’ll build an app to read movie reviews. Along the way, you’ll perform several NLP tasks:

• Language identification
• Named entity recognition
• Lemmatization
• Sentiment analysis

Don’t worry if any of those terms are unfamiliar to you — you’ll get to know them all soon.

A special thanks to Michael Katz and the editorial team of iOS 11 by Tutorials. Michael wrote that book’s “Natural Language Processing” chapter, on which this chapter is heavily based. Specifically, we reuse much of the starter project and general structure from that chapter, but we implement things differently, here. This chapter does cover some additional topics, such as training custom models, so we recommend going through it even if you’ve already read that book.

Getting started

Open the SMDB starter project in Xcode. Build and run to check out the app, which starts out looking like this (pull down on the list to reveal the Search bar):

The Search feature doesn’t work yet, but you’ll fix that soon. The app contains the following four tabs:

• All: Shows a list of every movie review loaded from the “server.” (To keep things simple, SMDB actually loads from a JSON file included with the project.) You’ll add “heart-eyes” and “sad-face” emojis to the positive and negative reviews, respectively.
• By Movie: Lists movie names where users can tap a name to only see reviews for that movie. You’ll eventually include tomato ratings showing each movie’s average review sentiment.
• By Actor: Currently empty, you’ll make it show a list of names automatically discovered from the reviews, along with emoji showing the average sentiment for reviews mentioning each name. Users will be able to tap a name and see all the reviews that mention it.
• By Language: Currently empty, it will soon list languages detected in the reviews. Users will then be able to tap a language to read all the reviews written in it.

You’ll add these missing features inside NLPHelper.swift, so open it now. It includes empty stubs for the functions that you’ll implement. Notice that it also imports the Natural Language framework, giving you access to well-trained machine-learning models for several NLP tasks. The first one you’ll take a look at is language identification.

Language identification

Your first classification task will be identifying the language of a piece of text. This is a common first step with NLP because different languages often need to be handled differently. For example, English and Chinese sentences are not tokenized in the same way.

This is important enough that classes in the Natural Language framework attempt to automatically identify the language of whatever text they encounter before moving forward with their own work, so in many cases you won’t have to bother with this step. However, detecting languages is also a useful task on its own. For example, to direct support requests to the appropriate staff members, or perhaps — as in this app — to organize documents by language. For times like these, Apple provides NLLanguageRecognizer.

Replace getLanguage(text:) in NLPHelper.swift with the following code:

func getLanguage(text: String) -> NLLanguage? {
  NLLanguageRecognizer.dominantLanguage(for: text)
}

This function is only a single line — it takes a String and passes it to NLLanguageRecognizer’s dominantLanguage(for:) function. That call returns an optional NLLanguage object for the language it thinks is most likely in use by the given text. The values are enums with names that match the language they represent, such as .english, .spanish and .german.

In situations wherein portions of the text are in different languages, it returns the language that makes up most of the text. This function returns nil when it can’t determine the language.

Note: You may be aware that many language names can be abbreviated by a two-character ISO 639-1 code. For example, “en”, “es” and “de” for English, Spanish and German, respectively. You can access the two-character code for the language represented by an NLLanguage object via the object’s rawValue property.

Build and run the app. Switch to the By Language tab, which should look like this:

The table lists each language identified in the reviews, along with how many reviews use it. Tapping a row shows a list of reviews written in that language. Using the Natural Language framework, you’ve improved the app’s user experience, because now users only have to scroll through reviews they can actually read.

Additional language identification options

The NLLanguageRecognizer performs just one task: identifying languages used in text. If you need it, then you’ll most often use it as you did here, via its convenience function dominantLanguage(for:). However, there are situations that call for more control, and, in those cases, you’ll need to create an NLLanguageRecognizer object and call some of its other methods.

You can pass it text via its processString function, which has no return value but stores the most likely dominant language in its dominantLanguage property. If you want more fine-grained information, you can get specific probabilities for multiple possible languages via its languageHypotheses(withMaximum:) function. The withMaximum parameter lets you specify how many probabilities you want to see — for example, the top five. Prior to processing a string, you can provide hints in the form of a dictionary containing the likelihood of encountering specific languages via the languageHints property. You can also restrict what language responses are possible via the languageConstraints property.

Finding named entities

Sometimes, you’ll want to find names mentioned in a piece of text. Maybe you want to sort articles based on who they are about, organize restaurant reviews based on the cities they mention, or extract important information from a document, which often includes names of people, places and organizations. This is called named entity recognition (NER), and it’s a common NLP task with many use cases. It’s also a form of text classification.

When you’re looking for a specific word, a simple search is often enough. However, when there are many such words, and especially when you aren’t sure in advance what those words will be, that’s when machine learning can help. The Natural Language framework provides well-trained models capable of finding names of people, places and organizations.

In this section, you’ll give SMDB the ability to sort reviews based on the people’s names they contain. The app doesn’t know in advance what names might exist, so it has to examine the text and classify words as either names or not names. Apple provides a class that can handle this task — and more — called NLTagger.

Replace getPeopleNames in NLPHelper.swift with the following implementation:

func getPeopleNames(text: String, block: (String) -> Void) {
  // 1
  let tagger = NLTagger(tagSchemes: [.nameType])
  tagger.string = text
  // 2
  let options: NLTagger.Options = [
    .omitWhitespace, .omitPunctuation, .omitOther, .joinNames]
  // 3
  tagger.enumerateTags(
    in: text.startIndex..<text.endIndex, unit: .word,
    scheme: .nameType, options: options) { tag, tokenRange in
    // 4
    if tag == .personalName {
      block(String(text[tokenRange]))
    }
    return true
  }
}

The body of this function shows the general pattern that you’ll follow for many NLP tasks. It goes as follows:

1. Create an NLTagger and pass in an array of NLTagScheme objects telling it what to look for in the text. (More on this later.) Then, you set the text for it to parse via its string property.
2. Fine-tune what the tagger returns with an array of NLTagger.Options values. In this case, you’re going to skip whitespace, punctuation and non-linguistic tokens such as symbols. You also pass .joinNames, which tells the tagger to combine multi-part names into a single token. For example, “Jane Smith” instead of “Jane” and “Smith.”
3. Call the tagger’s enumerateTags method to iterate over whatever tokens it can find within the specified range of the text you set earlier, potentially assigning an NLTag to each one. (More on this later.)
4. Provide enumerateTags a code block to call for each token the tagger processes. In this case, you check that the tag is the name of a person — rather than a place or organization — and, if it is, you pass the identified token as a String into the block passed into getPeopleNames.

You’ll use that pattern often: Create an NLTagger, use it to assign classes to tokens, and then process important tokens in some application-specific way.

Here are some more details about NLTagger and the code you just added:

• NLTagger operates on tokens, but what a “token” means depends on the value you pass to enumerateTag‘s unit parameter. It can be any of .word, .sentence, .paragraph or .document. The tagger will consider text in these unit-sized chunks, broken up using the rules it understands for the text’s language. Some tagging schemes only work with specific units — for example, the .nameType you used here only works with words.
• When NLTagger labels a token, it calls the code block you specify with an NLTag object and the range of the tagged token within the source text. The actual value of the NLTag object is based on the tagging scheme — in the case of names, it can be .personalName, .placeName or .organizationName, but there are other possibilities when using different tagging schemes.
• You used the .nameType tagging scheme to initialize the tagger to classify names, but Apple provides several different built-in options. You’ll take a look at another one in the next section.
• NLTagger doesn’t actually do all the work involved with classifying tokens. It’s mostly a wrapper that uses different models based on the particular combination of tagging scheme and token unit you provide. Later in this chapter, you’ll see how to provide custom models to add new types of tagging.
• You can initialize a tagger with more than one scheme to support multiple tasks, but enumerateTags only handles one scheme at time so you’ll need to call it separately for each one you want to apply.
• Apple doesn’t support every tagging scheme for every language. Call NLTagger.availableTagSchemes(for:language:) to get a list of supported schemes.
• Check out NLTagger’s tag(at:unit:scheme:) and tags(in:unit:scheme:options:) functions. They return a tag or tags directly rather than making you iterate over all the tokens with a block.
• Pro tip: Don’t forget to set the string property before calling enumerateTags! The tagger won’t complain if you don’t, but it won’t produce any results, either.

Build and run, again, and take a look at the By Actor tab.

You’ll see a list of names NSTagger thinks it has identified in the reviews. Tapping one leads to a list of reviews containing that name. The results aren’t perfect, though. For example, it misses the name “Faire Playe,” which appears in two reviews, and it identifies “O” as a name even though it was just part of the term “I/O.” The tagger uses a model that has learned what names generally look like and how they are used in sentences, but in the end it still has to guess about each token it encounters. It will give you good results, but it will never be 100% correct.

Adding a search feature

In this next section, you’ll use NLTagger for another task: lemmatization. That’s the process of identifying the root version of a word. For example, consider the sentences, “I am running” and “I was running.” Reducing each term to its root, both sentences become the same: “I be run.” Sure, it no longer reads as correct, but it encapsulates most of the information contained in both sentences.

Historically, it’s been common to preprocess text by lemmatizing it because it reduces the size of the vocabulary necessary to consider. You’ll learn more about vocabulary sizes in the next chapter, but, intuitively, the larger they are the more difficult they are to support. So rather than needing to understand “run,” “runs,” “running” and “ran,” you would just need to handle “run.” However, as you can see in this example, some important contextual information, such as tense, gets lost in the translation. For some tasks, like machine translation, it is now common to use text without first lemmatizing it in order to get more accurate results.

Note: Stemming versus lemmatization. You’ll probably encounter both of these terms, often used seemingly interchangeably. In the case of stemming, the root is called a stem; in the case of lemmatization, it’s called a lemma. These are essentially the same thing, but the process for generating them is different. Stemming involves basic rules like remove “ing” and “s” from the ends of words, which is fast and easy to implement but doesn‘t always produce the best results. On the other hand, lemmatization involves using a specific vocabulary for a language and applying more complex rules. It’s more involved but usually gives better results.

You’ll use lemmas in the SMDB app to support more sophisticated searches. When the user types search terms, the app will find all reviews containing those terms. But rather than only supporting exact matches, you’ll broaden the results by using lemmas. When a user enters a word like “run,” you’ll make sure the app finds reviews using other forms of the word, like “running,” too. Convenient!

Replace the empty getSearchTerms inside NLPHelper.swift with the following:

// 1
func getSearchTerms(text: String, language: String? = nil,
                    block: (String) -> Void) {
  // 2
  let tagger = NLTagger(tagSchemes: [.lemma])
  tagger.string = text
  let options: NLTagger.Options = [
    .omitWhitespace, .omitPunctuation, .omitOther, .joinNames]
  tagger.enumerateTags(
    in: text.startIndex..<text.endIndex, unit: .word,
    scheme: .lemma, options: options) { tag, tokenRange in
    if let tag = tag {
      // 3
      let lemma = tag.rawValue.lowercased()
      block(lemma)
    }
    return true
  }
}

This code looks a lot like getPeopleNames that you added earlier. That’s because it follows the same pattern. Here’s what’s different:

1. The function accepts an additional parameter — an optional language character code. You can ignore this for now.
2. You’re using the .lemma tagging scheme, which tells the tagger you want it to return the lemma for each token it encounters. Just like when searching for names, the .lemma scheme only works for .word token units.
3. If the tagger identifies a lemma — it won’t always be able to — then it’s contained in the NLTag‘s rawValue property. You extract it, ensure it’s lowercased — this app won’t support case-sensitive search — and then pass it to the block that was passed into getSearchTerms.

The app’s starter code already calls getSearchTerms for each review, mapping the review to each term generated by this function. Therefore, you only have to build and run the app to try some searches. With the app open, pull down on the table to reveal a search bar where you can enter terms to find within reviews.

Note: If you’re curious to see how the app maps reviews to search terms, check out populateSearch in ReviewsManager.swift.

Throughout this section, you’ll search for a few specific examples to see how the app performs and what motivates each specific code choice. These also serve to demonstrate a few of the difficulties involved when working with text. Try the following:

• Type sing, and you’ll see three search results, all of which actually contain the word “singing.” However, actually type singing and you get zero results. That’s unsettling.
• Type dance and you’ll get one result, which actually contains the word “dancing.” However, type dancing and you’ll get two different results, each of which seems to contain the same word. Suspicious, no?
• Type bueno and you’ll get one result, which contains the word “buena.” That’s good — it shows lemmatization works for more than just English. However, type the actual word used in that review — buena — and you’ll get no result. What gives?

The problem here stems from how you generated the search terms via their lemmas. See what I did there?

Remember the app maps reviews to terms generated by getSearchTerms. But this function returns lemmas, which may not match the original text in the review. For example, in these reviews the lemma of the word “singing” is “sing,” so that’s the only version of that word users can find via search. That’s not very convenient, but it’s something you should be able to fix. Instead of searching for exactly what the user types, you could search for the lemma of whatever the user types instead.

Note: If you run the app on a hardware device, as opposed to the simulator, you may not get any results for terms in languages other than the device’s native language. For example, if your phone has always been set to use English, you probably won’t get results for the term bueno above. If you temporarily switch your device to another language — Spanish, for this tutorial — and then switch it back (hopefully, you won’t get lost trying to return!), then the app should start finding search results for that language, too. However, the simulator should work fine for all languages iOS supports without you needing to do any extra work.

Switch over to ReviewsTableViewController.swift and replace findMatches with this new version:

func findMatches(_ searchText: String) {
  var matches: Set<Review> = []
  // 1
  getSearchTerms(
    text: searchText,
    language: Locale.current.languageCode) { word in
    // 2
    if let founds = ReviewsManager.instance.searchTerms[word] {
         matches.formUnion(founds)
    }
  }
  reviews = matches.filter { baseReviews.contains($0) }
}

This bit is more application-specific than the other functions you’ve added, but it shows one way to actually use the results of the tagging process.

1. You pass searchText — what the user entered in the search bar — to getSearchTerms in order to reuse the lemmatization code you added earlier. Now, the app lemmatizes the words users search for instead of just the words in the reviews that the app looks at.
2. For each lemma identified by getSearchTerms, you check inside the ReviewsManager’s searchTerms dictionary. If it finds any reviews, it adds them to the results the user gets.

Build and run, and you’ll see the search behavior has changed, but is it for the better? Try those three examples again.

• Typing sing and singing now both give the same results: nothing! Seems like a downgrade.
• Type dance and you’ll now get zero results, while dancing gives you the one result that “dance” used to give you. Downgrade number two.
• Type bueno and you’ll find the same review as before, but now typing buena also gives you that result. Finally, something got better!

These new errors occur because NLTagger sometimes has trouble lemmatizing short texts. You can test this out by typing just the letter “I”, which will produce no results. Now continue typing so you search for “I sing”. You’ll find as soon as you start typing the second word, regardless of what you type, you’ll get all the results that have the word “I” in them.

That’s because now NLTagger sees it as a sentence and has a better guess about “I” being a word. Once you get to “I sing,” you’ll get all the reviews that contain “singing” — even if they do not contain the word “I.”

The primary cause of this difficulty is that NLTagger can’t always determine the language of shorter texts, and lemmatization requires language-specific knowledge. With longer samples, it’s usually no problem, which you saw when you identified the languages for the reviews. But with shorter texts it’s a good idea to help it if you can.

So how do you do that? By telling the tagger what language you’re using prior to asking it to lemmatize the text. Remember that unused language parameter in getSearchTerms? Well, now it’s time to use it.

Back in NLPHelper.swift, add the following lines inside getSearchTerms, just before the let options: ... line:

if let language = language {
  tagger.setLanguage(NLLanguage(rawValue: language),
                     range: text.startIndex..<text.endIndex)
}

This code sets the language on the tagger when a language is available, telling the tagger how to interpret the text stored in its string property. In this case, you’ll have a language’s two-character code, like “en” for English, and you’ll create an NLLanguage object from it. You assign the language for the full range of the text, but you could assign different languages for different sections if necessary.

NLTagger offers another function named setOrthography, which sets even more information about the language, such as its script, but Apple recommends not using it unless you are sure of the value. The tagger will determine the orthography itself from the text, and setting the language — if set correctly — essentially guarantees you’ll end up with the correct orthography anyway.

Note: If your device’s language is not set to English, then your results for the rest of this section may not exactly match what is described in the chapter. If this makes it difficult to follow along, go back to ReviewsTableViewController.swift and change Locale.current.languageCode in findMatches to be just the string "en". This will force the tagger to assume all search terms are English.

Build and run the app, then try out those test searches again. How well do they work?

• Typing either sing or singing produces the same set of all three reviews that include “singing.” Nice!

• Searching for dance or dancing gives the same single result, but we know there are two other reviews that contain the word “dancing.” Better, but not quite right yet.

• Now, typing bueno or buena each give zero results. Uh oh, things are going in the wrong direction again. Coding is hard!

These errors are caused by two different issues, but solving one will solve the other well enough for this chapter’s purposes.

The first problem — the one you won’t fix here — is with the code you wrote earlier in findMatches. It passes the language code for the language currently set on the device. This will not always be correct — for example, when the user’s iPhone is set to use English but they try searching for a Spanish term like “bueno.” Now that we are setting the language directly, the NLTagger no longer determines it automatically, so it doesn’t recognize this as Spanish and can’t lemmatize it correctly.

A better approach would be first letting the NLTagger try to determine the language and only resorting to the user’s default language when that fails. We won’t show that here, but it’s a small addition that readers should be able to make on their own after going through this chapter.

The second problem — the one you’re about to fix — can be demonstrated more clearly with some other searches. Try searching for Kotlin or realz. Those terms appear in reviews, but they produce no search results. Why not?

It’s because the tagger can’t find lemmas for unknown terms like “Kotlin,” but getSearchTerms currently only processes the lemmas it finds. Terms like these are considered out-of-vocabulary, but that doesn’t mean users won’t want to search for them.

In this case, you can fix the problem with a couple lines of code, but you’ll see later in the book that out-of-vocabulary words cause other, more difficult, problems for NLP tasks, too.

Still in getSearchTerms, find the if statement inside the enumerateTags block, and add the following line right above it:

let token = String(text[tokenRange]).lowercased()

This line gets the token from the original text, and ensures it’s lowercase just like how you handled the lemmas earlier.

Next, add an else block to the if:

if let tag = tag {
  ...
} else {
  block(token)
}

This the token to the block that the app passed into getSearchTerms. That means now all lemmas and any words that have no lemmas will get added as search terms.

Build and run with these changes. Repeating those searches gives the following results:

• Both sing and singing still work properly. That’s a good sign!
• There is no change for dance or dancing. OK, at least they aren’t worse, right?
• Now bueno works, but buena still finds nothing. That’s at least some improvement.
• And what about words where the NLTagger could find no lemmas? Searching for these out-of-vocabulary words, like “realz” or “Kotlin,” works properly and returns the appropriate reviews.

At this point, searching for either “dance” or “dancing” finds only one review containing “dancing” — the one when it’s used as a verb. Here’s why: When you search for the word “dancing,” it gets lemmatized as “dance.” But when the reviews were processed for search terms, the noun usages of “dancing” did not produce lemmas because “dancing” is a valid root when used as a noun. So NLTagger lemmatizes some terms differently when it encounters them in the reviews versus when it sees them as user-entered search terms. It’s being clever by trying to give you the most appropriate lemmas for the context, which is usually a good thing. But you want users to be able to find both sets of reviews, so what can you do?

Go back to that same if statement inside the enumerateTags block, and add the following code just after the call to block(lemma):

if lemma != token {
  block(token)
}

This new if statement checks to see when a token and its lemma are not the same word. In that case, it passes the token to the block that the app passed into getSearchTerms. So, in cases where you search for “dancing,” it will process both “dance” and “dancing.”

Build and run and try those test searches one last time. They mostly all work fine, but there’s still a difference between the results for typing dance and dancing — the former finds the one review that uses “dancing” as a verb but misses the two reviews where it’s used as a noun, while the latter finds all three of those reviews.

This is the best you’re going to do without additional preprocessing. One option would be to lemmatize a sentence and attempt to break it up into tokens and lemmatize each token individually. That would give you more possible search terms because it would lemmatize each term both in and out of context. While it would fix the “dancing”-used-as-a-noun issue, you’d still have other problems. For example, spelling mistakes would still break the search, and out-of-vocabulary words still won’t support even basic stemming, so searching for the singular of an unknown word does not find reviews containing usages of that word’s plural.

One last thing: Remember earlier in findMatches when you passed getSearchTerms the device’s current language along with the search term and that broke foreign-language searches? Now typing either bueno or buena works fine, but why? It’s for a subtle reason: When the app lemmatizes the reviews, it correctly lemmatizes “buena” as “bueno” because it recognizes the language as Spanish. But now this new code you just added associates the review with both of those terms rather than just the lemma. Then later, when you try to search for one of them, even if the default language causes NLTagger to fail its lemmatization, it just goes down your other code branch that handles out-of-vocabulary words by looking for exact matches. And sure enough, the search finds what you typed — regardless of whether it was “bueno” or “buena.”

At this point, you’ve got a pretty good search feature. It isn’t industrial strength, for sure, but it’s still surprisingly powerful for writing so little code. And along the way you’ve seen some of the problems you might encounter when tying to work with text in your own apps. Now, it’s time to move away from the Natural Language framework’s built-in support and train some custom models.

Sentiment analysis

Could we really cover machine learning for natural language without mentioning sentiment analysis? Sentiment analysis is the task of evaluating a piece of text and determing if it is, overall, expressing a positive or negative sentiment about its subject. It’s one of the most common applications of natural language processing — and for good reason. Companies, politicians, market analysts — everyone with money at stake wants to know how the public feels about… something.

For this reason it’s no surprise that Apple ships a built-in sentiment analysis model (as of iOS 13). Apple does not reveal how their model works and you cannot configure it or fine tune it for your problem domain, but it is certainly easy to use. You can feed this any piece of text and it will return a score from -1.0 to +1.0, indicating if the text is very negative or very positive.

It relies on a type that will be familiar to you by now, NLTagger, using a new dedicated tag scheme .sentimentScore. The .sentimentScore tag scheme configures a tagger that will return a tag containing a numerical sentiment score. The one quirk in this API is that, although it returns a numerical value, it returns this value as a String, requiring some trivial conversion on your part. Also, while you previously used tag schemes that return tag values at the level of a single word unit, the sentiment scheme returns a value at the level of a sentence or paragraph.

To write a function that does basic sentiment analysis add the following to NLPHelper.swift, just below your definition of getSearchTerms:

// 1
func analyzeSentiment(text: String) -> Double? {
  // 2
  let tagger = NLTagger(tagSchemes: [.sentimentScore])
  tagger.string = text
  // 3
  let (tag, _) = tagger.tag(at: text.startIndex,
                           unit: .paragraph,
                           scheme: .sentimentScore)
  // 4
  guard let sentiment = tag,
     let score = Double(sentiment.rawValue)
     else { return nil }
  return score
}

This is only slightly different from our previous functions:

1. The function is synchronous, taking a String and returning an optional Double.
2. You’re using the .sentimentScore tagging scheme with your tagger, and handing it the text.
3. You’re calling the synchronous NLTagger.tag(at:unit:scheme:) function, which returns an optional NLTag immediately rather than taking a callback.
4. Finally, you unwrap the optional and parse the String in sentiment.rawValue to return a Double, measuring the sentiment.

Later in this chapter we will show how to integrate this function’s output into the user interface. But for now, just print the score to the console, by adding the following line to tableView(_:cellForRowAt:) in ReviewsTableViewController.swift, immediately before the return statement:

print("review text: \(review.text)\nscore: \(String(describing: analyzeSentiment(text: review.text)))\n\n")

If you open the console as you scroll the app, you’ll see reviews rolling by with their associated sentiment scores. “The Sound of MusicKit” rates a solid 1.0, but “The Swift and the Dead” only clocks in at -6.0.

Not bad a for just a few lines of code! But what if you want a bit more control?

Building a sentiment classifier

While it is convenient that Apple provides their own sentiment analysis API, it is instructive to build your own sentiment classifier. Why? Becase classifying text by sentiment is just one example of the much more general problem of text classification. Spam detection, prioritizing support requests, and identifying document topics are all variations of that same problem. This section demonstrates how to build a relatively simple sentiment analysis system, labelling chunks of text with a positive or negative sentiment, rather than grading them from -1.0 to +1.0. Remember, you can use these techniques for all sorts of classification tasks.

Training a text classifier with Create ML

You’ll use Create ML to train an MLTextClassifier model. This class is meant to classify larger chunks of text rather than individual words, although it is technically capable of doing both. You’ll see a different model later in this chapter that is better suited to classifying word tokens.

In previous chapters you’ve used the Create ML GUI application to train models. In this one you’ll train your model in an Xcode playground. With the default model types, training the model in this section shouldn’t take long and we recommend you go through the steps. However, if you’d prefer you can use the pre-trained model found at projects/starter/models/SentimentClassifier.mlmodel in the chapter resources.

Note: As you may have seen, the Create ML GUI app provides a drag and drop interface to Create ML, allowing you to select your training data with a file picker, choose your model type by selecting a radio button, and kick off training by pushing a big “Train” button with the same familiar icon which the Music app uses for playing a tune. When this approach works, it’s great! But it is also worth being familiar with playgrounds. For one thing, training on a playground will work on macOS Mojave (10.14) or macOS Catalina (10.15). In addition, playgrounds are closer to the typical machine learning workflow, since they support easier iteration, experimentation, and tracking of past results, like Jupyter notebooks.

Before dealing with Xcode, you’ll need a dataset. Xcode playgrounds have special access to a specific folder on your Mac, where you’ll store your dataset and output your trained model. If it doesn’t already exist, create a folder named Shared Playground Data inside your Documents folder. This folder must have that exact name and be in that location for your playgrounds to access it.

Note: You can add files directly to your playground’s bundle resources — and you’ll see that done later in this chapter. But when I tried that with the large dataset involved here, Xcode struggled and I spent way too much time staring at spinning beachballs and force-quitting the app. Things performed much better with the data stored outside of the bundle.

Next, create a folder named TextClassification inside Shared Playground Data. You’ll keep everything for this chapter organized there.

Find projects/starter/datasets/MovieReviews.zip in the chapter resources and unzip it into the Shared Playground Data/TextClassification folder you just created. You should now have a subfolder named MovieReviews.

This new folder contains subfolders with 50 thousand movie reviews, half labeled as positive and the other half negative. It’s a slightly paired-down version of the Large Movie Review Dataset, from the 2011 paper, “Learning Word Vectors for Sentiment Analysis,” by Andrew L. Maas et al., published by the Association for Computational Linguistics.

Interested readers can find the paper at www.aclweb.org/anthology/P11-1015. There’s a README file describing the changes we made, primarily to save space by eliminating files which were unrelated to this chapter.

Create a new playground file using any template for macOS. The specific template doesn’t matter, but the operating system does because Create ML is only available on macOS. Or if you’d rather follow along with a completed playground, you can find one at projects/final/playgrounds/MovieSentiment.playground.

If you started from a template, delete whatever starter code Xcode provided and add the following imports:

import CreateML
import PlaygroundSupport

You’ll train your text classifier with Create ML, so you import it here. And you import the Playground Support framework to access the Shared Playground Data folder you set up earlier.

Now, add this next bit of code to access your training and test data:

// 1
let projectDir = "TextClassification/"
let dataDir = "MovieReviews/"
let trainUrl =
  playgroundSharedDataDirectory.appendingPathComponent(
    projectDir + dataDir + "train", isDirectory: true)
let testUrl =
  playgroundSharedDataDirectory.appendingPathComponent(
    projectDir + dataDir + "test", isDirectory: true)
// 2
let trainData =
  MLTextClassifier.DataSource.labeledDirectories(at: trainUrl)
let testData =
  MLTextClassifier.DataSource.labeledDirectories(at: testUrl)

This code creates the datasets that you’ll use for training and testing your model. Here’s what you did:

1. Create URLs pointing to the MovieReviews/train and MovieReviews/test folders containing your dataset.

2. Create MLTextClassifier.DataSources backed by those folders. This lets the model access data samples stored on disk as separate files, one subfolder per label you want your classifier to handle.

As you can see in the following image, your dataset includes two folders: “test” and “train,” and each of those contains two more folders: “neg” and “pos.”

Each of the files stored in these subfolders contains a single review, and the name of its folder is its classification label. So all the reviews in a “pos” folder are classified with positive sentiment, and the ones in a “neg” folder are classified with negative sentiment.

Note: This way of loading data works well when your dataset is spread across files like this. However, you can also train your model with an MLDataTable, which you can create from a JSON or CSV file, or from a Swift dictionary. You can even populate one programmatically if you need to. Use whatever method works best for your dataset. You’ll see an example of loading a JSON file a bit later.

Now, create an MLTextClassifier with the following code:

let sentimentClassifier = try!
  MLTextClassifier(
    trainingData: trainData,
    parameters:
      MLTextClassifier.ModelParameters(language: .english))

This single line not only creates your model, it trains it, too! It even separates a portion of the training data to act as a validation set to ensure the model doesn’t overfit.

Note the ModelParameters object which you pass to initialize the classifier. This object lets you specify what kind of classifier model to use, how to define your validation data, and the language of the text.

In the code above you you’ve only told it to train for English. This is a good idea because later you can query its supported language to ensure you only use it in the proper context. By default, the system will use a maximum entropy classifier with a validation set built from randomly picking less than 10% of the training dataset.

You can consider other settings later. For now, run the playground. Depending on the speed of your machine, this may take a few seconds to a few minutes, but you should get results similar to the following:

You didn’t specify a separate validation set as part of the ModelParameters, so the classifier reserves 5% of the training data for that purpose. It then spends a good bit of time tokenizing the reviews and converting them to training features. After that process completes, it starts training a MaxEnt model (more on that later), performing multiple training iterations until it reports a training accuracy close to 100%.

That’s great performance on the training data, but what really matters is how it performs on data it hasn’t ever seen. Add the following code to your playground to evaluate your model against a real test dataset:

// 1
let metrics = sentimentClassifier.evaluation(on: testData)
// 2
if metrics.isValid {
  print("Error rate (lower is better): \(metrics.classificationError)")
} else if let error = metrics.error {
  print("Error evaluating model: \(error)")
} else {
  print("Unknown error evaluating model")
}

This looks like more code than it really is. Here’s what you’ve added:

1. You pass a data source to the model’s evaluation(on:) function. This compares the model’s predictions for each item in the given test dataset against the correct labels.
2. Then you display the classification error if the model successfully calculated one, and print out any errors otherwise.

Here’s the output for the model supplied in the resources for the finished project:

Notice that this reports the error rate, not accuracy. People commonly report the percentage of incorrect responses rather than the percentage of correct ones. It’s important to keep track of which metric you’re dealing with, otherwise you might not properly compare the performance claims of different models.

Also keep in mind these values are related to each other: Subtract either value from 1.0 to get the other value. So an error rate of 0.1244 is an accuracy of 0.8756 — or 87.56%.

Beyond error rate, the metrics returned by evaluation(on:) also include precision, recall and a confusion matrix describing how the model predicted values for each class. It’s not shown here, but the confusion matrix for this model shows it handles each class equally well with no obvious bias toward one or the other.

While your model’s accuracy of almost 88% is not state-of-the-art on this dataset, it’s still quite reasonable for something you created with essentially a single line of code and no tinkering with parameters. If you really needed better results, you could create a model with one of the many other libraries that support conversion to Core ML.

Now that you have a trained model, add the following code to your playground to save it for use in your app:

// 1 (Optional)
let metadata = MLModelMetadata(
  author: "Your Name:",
  shortDescription:
    "A model trained to classify movie review sentiment",
  version: "1.0")
// 2
try! sentimentClassifier.write(
  to: playgroundSharedDataDirectory.appendingPathComponent(
    projectDir + "SentimentClassifier.mlmodel"),
  metadata: metadata)

These two lines do the following:

1. Specify your model’s metadata. This isn’t a requirement, but here’s how to do it if you want to.
2. Export a Core ML version of your model. Here, you write it out to the playground’s data folder.

Save this final version of your playground in case you ever want to come back to it. Then run it and you’ll end up with a trained model file named SentimentClassifier.mlmodel stored in your Shared Playground Data/TextClassification folder.

Now you can put this model to use. But before doing that, it’s worth experimenting to determine if this is the best model you can make.

Exploring other model types

You initialized MLTextclassifier with default parameters, specifying only that the language was English. But you can and should explore other configurations.

In particular, setting the algorithm property determines what kind of what kind of classifier model is used. The basic choice of model architecture can be regarded as one of the hyperparameters chosen while exploring a problem. Just as the training process searches for the best parameter weights to fit a model to your data, you yourself are searching for the best hyperparameters guided by trial and error and intuition.

In this case, CreateML offers you four possible kinds of classifier models — either a maximum entropy classifier, a conditional random field classifier, or a classifier based on transfer learning. The transfer learning classifier builds on top of a pretrained model shipped with iOS, which knows statistical relationships of words in your language. This is the knowledge that is being “transferred” to your problem. With a transfer learning-based model, you can additionally choose to use either a static or dynamic embedding of words, the latter being more a sophisticated kind of model which takes into account the context rather than just the identity of every word. (We will discuss embeddings in more detail in two chapters.)

Which type of model should you use? Apple does not in fact publish description of the detailed models underlying these choices. And even if they did, it would be hard to anticipate the best one for your data. So you should simply try a few and see which works best. The more sophisticated models, such as the transfer learning-based classifier, will take longer to train. But the more sophisticated models are not guaranteed to perform better.

For instance, using this dataset on a MacBook Pro (a 15-inch from 2016, with a 2.9 GHz Quad-Core Intel Core i7, 16 GB memory, and an Intel HD Graphics 530 graphics card), training the maximum entropy classifier takes about 3 minutes, the conditional random field classifier takes almost four hours, a transfer learning model with static embedding takes almost two hours and forty-five minutes, and, finally, a transfer learning model with a dynamic embedding takes over four hours. However, test accuracy of the the simplest classifier is the best, around 87%, while the accuracy of the fancier transfer learning classifiers is only around 75%. In short, when in doubt, experiment!

For now, proceed withe the model trained using the maximum entropy classifier. It’s time to put it in an app.

Use your text classifier in an app

Open your SMDB project in Xcode. Drag SentimentClassifier.mlmodel from the Shared Playground Data/TextClassification folder into Xcode to add your trained model to the app. Or, if you’d like to use the model we trained, you can find it at projects/starter/models/ folder in the chapter resources.

Then select SentimentClassifier.mlmodel in the Project Navigator to see what Xcode tells you about the model:

You’ve seen quite a few model summaries in Xcode at this point, and this one isn’t much different. Here are some highlights:

• Its type is Text Classifier, which tells you more about what the model is for than what it is. It’s actually a maximum entropy (MaxEnt) classifier, which is a probabilistic model that essentially determines how likely is it for a piece of text to represent a specific class. It generates numerical features from the text and performs a multinomial logistic regression over them. There are many possibilities for what features it could use — word counts, n-gram statistics, syntactical information, to name a few — but Apple doesn’t expose what features Create ML uses.
• The model’s not huge by machine learning standards, but at over 2MB, it’s larger than some you’ve made in this book. Still, it should be fine for use on mobile.
• Both its inputs and outputs are listed as single String values — the input text and the output label, respectively. You’ll give the model some text and it will return one of the labels — “pos” or “neg” — that you trained the model to predict.

Now that you’ve got your model in the project, open NLPHelper.swift and replace getSentimentClassifier with the following:

func getSentimentClassifier() -> NLModel? {
  try! NLModel(mlModel: SentimentClassifier().model)
}

This creates an instance of your model, but it does so a bit differently from other models you’ve created. Here you instantiate a SentimentClassifier, then use its model property to create an NLModel.

NLModel wraps Core ML models for use with the Natural Language framework. Xcode will let you use MLTextClassifier objects directly, like you’ve used models in earlier chapters, but it is essential to wrap them in NLModel first. This ensures the model preprocesses inputs the same way Create ML did during the training process. And as you’ve learned, it’s vital for preprocessing steps to match between training and inference, otherwise your models won’t produce the correct results.

Now replace predictSentiment inside NLPHelper.swift with the following code:

func predictSentiment(
  text: String, sentimentClassifier: NLModel) -> String? {
  sentimentClassifier.predictedLabel(for: text)
}

The SMDB app calls getSentimentClassifier once at startup and then passes the model it returns to predictSentiment for each review whose language matches the one supported by the model. To get a prediction, you call predictedLabel(for:), which classifies the given text and returns the label it predicts with the highest probability. Remember the folders for your data were named “neg” and “pos”, so those are the two possible return values here.

The MLTextClassifier — whether or not it’s wrapped in an NLModel — does not provide access to the actual prediction probabilities it calculates. That makes it different from some other models you’ve worked with elsewhere in this book. It’s a bit less flexible than some models, but what it lacks in flexibility it makes up for with ease of use.

Build and run one last time. You should now see happy faces on the positive reviews and sad faces on the negative ones.

Notice the faces only appear on the English-language reviews. That’s because the app only analyzes the sentiment of reviews whose language matches the one supported by your model. It accomplishes this with the following guard statement inside findSentiment in ReviewsManager.swift:

private func findSentiment(_ review: Review,
                           sentimentClassifier: NLModel?) {
  guard let sentimentClassifier = sentimentClassifier,
    review.language ==
      sentimentClassifier.configuration.language else {
    return
  }
  ...
}

NLModels have a configuration property that gives you access to an NLModelConfiguration object that contains some information about the model. Here, you access its language property to ensure it supports the review’s language.

Note: It’s important that you always verify your model supports an input before using it, like this function does. If you don’t, the model will still return a prediction, but it will be nothing more than a random guess.

The emoji feature uses a single sentiment prediction, but the app also shows how to aggregate sentiment. It converts the predicted labels into numerical values of 1 and 0 for positive and negative reviews, respectively. It then uses those numbers to calculate sentiment across multiple reviews. To see the fruits of that calculation, tap the By Movie tab. Each movie now includes a tomato rating indicating the average sentiment of its (English-language) reviews.

Finally, tap the By Actors tab. The list now lets you find the actors in the most-liked movies by showing an emoji indicating the prevailing sentiment of all the reviews mentioning that actor’s name.

For readers who went through the “Natural Language Processing” chapter in iOS 11 by Tutorials, you’ve now had the experience of using a pre-trained model as well as training one on your own. The one you trained even outperforms the pre-trained model from that book. For example, here’s a review that was scored with a negative sentiment in the original project: “What a great film! Ms. Keras Smith was truly magnificent, and Billy Caffe’s singing and dancing is the stuff of legends. Three thumbs up!” If you check that same review in your app here, which you can find easily by choosing Ms. Keras Smith or Billy Caffe in the By Actor tab, you’ll see it now correctly displays a happy face.

The MLTextClassifier you used in this section works well for larger chunks of text. In the next section, you’ll create a model used to classify individual words within chunks of text instead.

Comparing the analyzers

Before we finish, let’s make one more enhancement to the UI: update it to show the sentiment analysis from Apple’s built-in analyzer, so we can compare the result to our own classifier and provide the user more information.

First, delete the print statement that you temporarily introduced to tableView(_:cellForRowAt:) ReviewsTableViewController.swift.

Instead of that line, update the call to setSentiment to this new form, where you also pass in the numerical sentiment from Apple’s analyzer:

cell.setSentiment(sentiment: review.sentiment, score: analyzeSentiment(text: review.text))

Now, in ReviewTableViewCell.swift, update the function setSentiment(sentiment:) as follows:

func setSentiment(sentiment: Int?, score: Double? = nil) {
  // 1
  let classified: String
  if let sentiment = sentiment {
    classified = sentimentMapping[sentiment] ?? ""
  } else {
    classified = ""
  }
  // 2
  let scored: String
  if let score = score {
    scored = "(: \(String(score)))"
  } else {
    scored = ""
  }
  // 3
  sentimentLabel.text = classified + " " + scored
}

This is fairly straightforward:

1. The first if statement takes your zero or one score and maps it to an emoji or an empty string.

2. The second if statement takes the numerical score, and yields a string with an Apple icon and the number, or else an empty string.

3. Then you save a concatenated string in the cell for display.

Run the app now, and when you browse reviews you will see both ratings.

It’s interesting to notice that in some places, such as is pictured below, your classifier clearly does a better job than the built-in sentiment analysis API, such as some of the reviews of “Xcode Apocalypse” shown above. Not bad!

Custom word classifiers

You’re done with the SMDB app for now, but you’ll come back to it again in the next chapter. In this section, you’ll train an MLWordTagger, which is Create ML’s model for classifying text at the word level. You’ll use it to create a custom tagging scheme for NLTagger.

The model you make here attempts to identify names of Apple products mentioned in text, but you can train a model like this to tag individual words of any type. For example, imagine creating a profanity filter or automatically adding links to domain-specific jargon like legal or medical terms.

Create a new macOS playground and delete any code included from the template. Or, if you’d prefer, you can follow along with the completed playground at projects/final/playgrounds/CustomTokenTagging.playground.

There’s a tiny dataset stored in the chapter resources at projects/starter/datasets/custom_tags.json. Drag that file into the Resources folder in the playground’s Project navigator to add it to the playground.

Note: You could also use the Shared Playground Data folder like you did for the sentiment classifier, but this JSON file is quite small and Xcode should have no problem handling it as part of the playground bundle.

Select custom_tags.json in the Project Navigator to view the training examples. Here is a snippet from that file:

[
 ...
  {
    "tokens": ["The", "Apple", "TV", "is", "great", "for",
               "watching", "TV", "and", "movies", ",",
               "and", "you", "can", "play", "games",
               "on", "it", ",", "too", "!"],
    "tags": ["_", "AppleProduct", "AppleProduct", "_", "_", "_",
             "_", "_", "_", "_", "_",
             "_", "_", "_", "_", "_",
             "_", "_", "_", "_", "_"]
  },

  {
    "tokens": ["Apple", "adding", "Windows", "support", "for",
               "iTunes", "helped", "the", "iPod",
               "succeed", "."],
    "tags": ["_", "_", "_", "_", "_",
             "AppleProduct", "_", "_", "AppleProduct",
             "_", "_"]
  },
 ...
]

This JSON file contains a list, where each element is a dictionary with two keys: tokens and tags. Each dictionary in the list defines a single training example. The tokens key maps to a list of strings for a tokenized text sample, and the tags key maps to the list of tags that correspond to items in the tokens list.

The specific tags used here were chosen somewhat arbitrarily. Each word you’re interested in — the ones that name Apple products — is tagged with “AppleProduct,” whereas the other tokens are all tagged with a simple underscore. You could use a descriptive term if you’d prefer, but I chose this to help the product tags stand out in the list.

Notice the second example includes the word “TV” twice, tagged once with “AppleProduct” and once with an underscore. Learning to assign tags properly involves more than memorizing words; the model has to learn to evaluate tokens in context, otherwise it would not be able to handle cases like this one.

A few notes about the training data:

• The actual key names tokens and tags don’t matter. You can name them anything you want as long as it’s consistent across samples.
• Models are allowed to produce more than one tag. This example happens to assign everything either “AppleProduct” or an underscore, but feel free to include as many tags as necessary for your task.
• The formatting shown here is adjusted slightly from what you’ll see in the actual file to make it easier to read in the book. Specifically, you don’t actually need to split the tokens and tags into multiple lines like this.

Now that you’ve looked at the data, you’ll train a model. To get started, add the following to your playground:

import Foundation
import PlaygroundSupport
import CreateML
import CoreML
import NaturalLanguage

You’re importing several frameworks here because you’re going to train a model and use this playground to simulate the model’s usage in an app. However, they should all appear familiar to you now.

Next, prepare your training data with the following:

let trainUrl =
  Bundle.main.url(
    forResource: "custom_tags", withExtension: "json")!
let trainData = try MLDataTable(contentsOf: trainUrl)

You access the custom_tags.json file from the playground’s resource bundle, and use it to create an MLDataTable. This class stores tabular data for use with Create ML. It populates itself with a new row for each item in the JSON file’s list. It uses the keys it finds in each dictionary as column names, and maps the dictionary values to the corresponding row-column cell.

So for the file you just loaded, the MLDataTable will have 11 rows, each with two columns named tokens and tags.

Next, add the following line to create your model:

let model = try MLWordTagger(
  trainingData: trainData,
  tokenColumn: "tokens", labelColumn: "tags",
  parameters: MLWordTagger.ModelParameters(language: .english))

Here, you create an MLWordTagger, passing in the training data and the names of the columns, which define the tokens and labels. Regardless of how much data your table contains, the model only ever looks at the two columns you specify here. This is why it doesn’t matter how you name the keys in your JSON file — pick whatever you like and then tell MLWordTagger what to look for when you create it.

Once again, you specify the language this model supports as English, to match the training data. You’ll see how NLTagger uses this information a bit later.

In classic Create ML fashion, just creating your model object also handles training it. You could test this model like you would any other Create ML model, but for now just save it out with the following code:

let projectDir = "TextClassification/"

// Optionally add metadata before saving model

let savedModelUrl =
  playgroundSharedDataDirectory.appendingPathComponent(
    projectDir + "AppleProductTagger.mlmodel")

try model.write(to: savedModelUrl)

Here, you export your model in Core ML format to the same Shared Playground Data/TextClassifier folder you used to train your sentiment analysis model.

Next, you’ll need to know how to use a custom word classifier like this one inside an app. But rather than pigeon hole this functionality into the SMDB project, you’ll just use the model right here in the playground. However, to do that you do need to do one special step. Add the following line:

let compiledModelUrl =
  try MLModel.compileModel(at: savedModelUrl)

When you add a Core ML model to Xcode, it actually compiles it into a format that can be used by your app. However, this does not happen automatically in playgrounds. This line loads the model file at the specified URL, compiles it, and writes the results to a temporary folder on your device. It returns the URL of the compiled model.

The rest of this section shows code that you could use inside an app just like you do here. Add the following line to instantiate your model:

let appleProductModel =
  try NLModel(contentsOf: compiledModelUrl)

This is similar to what you did with the sentiment classifier. Here, you wrap your MLWordTagger inside an NLModel to ensure your app tokenizes inputs the same way as Create ML did when you trained the model. You create it with the URL of your compiled model, but in an app you could also create the model directly like you did earlier with SentimentClassifier.

Next, add the following code to configure an NLTagger to use your new model:

// 1
let appleProductTagScheme = NLTagScheme("AppleProducts")
// 2
let appleProductTagger = NLTagger(tagSchemes: [appleProductTagScheme])
// 3
appleProductTagger.setModels(
  [appleProductModel], forTagScheme: appleProductTagScheme)

Here’s how you configure the tagger:

1. Create a new NLTagScheme object to represent your word classifier. You can name it anything you like; it doesn’t need to match the name of your model or the names of any tags it produces.

2. Create an NLTagger like you did before, but give it your new tag scheme. You can provide multiple schemes here, including built-in options and other custom ones.

3. Call setModels on appleProductTagger, passing it your custom model and tag scheme. This tells the tagger to use your custom model when asked to tag with that scheme on a language supported by the model. You can provide more than one model in this list if you’ve trained different ones for different languages, and the tagger will use the correct one based on the language of the text it processes.

And, finally, you’ll this code to test out your model on some sample inputs. First, create some test strings to simulate inputs.

let testStrings = [
  "I enjoy watching Netflix on my Apple TV, but I wish I had a bigger TV.",
  "The Face ID on my new iPhone works really fast!",
  "What's up with the keyboard on my MacBook Pro?",
  "Do you prefer the iPhone or the Pixel?"
]

These include a mix of Apple products that were in your training set, Apple products that were not in the training set, and non-Apple products.

Next, follow follows the same pattern you’ve seen before when using NLTagger to enumerate all Apple products:

let appleProductTag = NLTag("AppleProduct")
let options: NLTagger.Options = [
  .omitWhitespace, .omitPunctuation, .omitOther]

for str in testStrings {
  print("Checking \(str)")
  appleProductTagger.string = str
  appleProductTagger.enumerateTags(
    in: str.startIndex..<str.endIndex,
    unit: .word,
    scheme: appleProductTagScheme,
    options: options) { tag, tokenRange in

    if tag == appleProductTag {
      print("Found Apple product: \(str[tokenRange])")
    }
    return true
  }
}

The only difference here is you create a new NLTag for your custom tag name and check for that while processing the tokens.

Run the playground to train your model and see how it performs on your test cases:

Here’s what you see while training the model, line by line:

• According to the first message, the model doesn’t create a validation set because your dataset has fewer than 50 items. Seems reasonable, but the very next message claims it’s using two samples for validation. These two statements seem to contradict each other, but rest assured — you’ll probably never see this message in real life because you would never train a real model with fewer than 50 samples, right? Right?
• Next, it tokenizes the data just like when you trained your sentiment analysis model. It goes much faster this time, though, mostly because you’re working with a tiny dataset but also because the JSON file already defines each input as a list of tokens.
• It claims to start “CRF training,” but what’s that? It’s just talking about training the model. CRF stands for “conditional random field,” which is the algorithm MLWordTagger uses to classify words. This is another probabilistic model, but one that usually does better than MaxEnt when predicting labels on individual words — MaxEnt works better when classifying larger chunks of text. Its primary advantage is that it considers the tokens as sequences, which MaxEnt does not necessarily do. (It can use some sequential data, like n-gram statistics, but CRF relies on it more heavily.) Once again, Apple does not provide the details of Create ML’s implementation.
• It trains for only one iteration over the dataset. It would likely train more if you ran with more data, but it achieves perfect accuracy on (all two of) the validation samples so it stops training.

And here’s what its output from the tests looks like:

The model does really well, especially considering your dataset only had 11 samples — and the model only trained on nine of them! It managed to correctly label the different versions of “TV” in the first example, and even labeled “Face” and “ID” as Apple products, even though those tokens never appear in the training set.

However, it wasn’t all good. Notice it also attributes “Pixel” to Apple, which I’m sure would surprise Google.

These examples prove the model learns something about the context where these tokens appear, rather than just memorizing the words. Training with a larger dataset will give better results, but just like everything else based on machine learning, it won’t ever be perfect.

One last thing: Notice your model tags multi-word names like “Face ID” and “MacBook Pro” as multiple words. That’s because the NLTagger first tokenizes the input based on its rules for the text’s language, and it doesn’t already know that these words are meant to go together. There’s no way to avoid this, so you’ll need to label them as separate words in your training data, and then write your own logic for recombining them later.

The remaining bits

The Natural Language framework supports a few other things not specifically covered in this chapter. The three you’ll most likely use are gazetteers, part-of-speech tagging, and tokenization.

A gazetteer is a simple concept. It’s essentially just a dictionary: It maps a predefined list of entities to a single tag for each entity. For example, in the last section, you trained a tagger that could consume a text string and tag which words were Apple products. Great! But in order to train that tagger, you needed to provide it training data – a collection of many sentences where you had already tagged the words representing Apple products.

But what if, starting out, you didn’t have a large list of tagged sentences but you did have a plain old list of Apple products? This is exactly where you would need a NLGazetteer, also known by Apple as a text catalog. It holds a fixed list of entities and their tags in a highly efficient representation. Once you’ve set your NLTagger to use a gazetteer, then it can identify the entities you named. So you could define a gazetteer which mapped every known Apple product to a single tag, and use a tagger to find the Apple products. A gazetteer is not a machine learning model at all but it is worth keeping in mind in case it’s what you really need.

Part-of-speech tagging refers to analyzing text for grammatical structure. In code, it requires nothing more than using an NLTagger just like you’ve done elsewhere in this chapter. In this case, you iterate over tokens using either the .lexicalClass or .nameTypeOrLexicalClass tag schemes and the tagger assigns NLTag values indicating how those tokens are used in the text. For example, .noun, .verb or .adjective. Consult the documentation for the possible values.

Tokenization is the process of splitting a piece of text into smaller units. It most often means dividing strings into individual words and punctuation, but it could mean breaking it into other units, like sentences or characters.

The classes you’ve used throughout this chapter all tokenize their inputs automatically, so you haven’t needed to worry about it. However, if you ever need to do it yourself, the Natural Language framework provides NLTokenizer to chunk text by word, sentence, paragraph or document. It uses language-specific rules which are generally good but might not always be exactly what you want. Still, it’s a nice option so you should at least try it the next time you need to tokenize some text.

You’ll use NLTokenizer as a preprocessing step when you implement language translation in the next chapter. In the meantime, you can check out NLExtras.playground in the projects/final/playgrounds folder to see sample code for both part-of-speech tagging and tokenization.

Key points

• Use Apple’s new Natural Language framework to take advantage of fast, well trained machine-learning models for NLP.
• NLLanguageRecognizer can identify the language used in a piece of text.
• NLTagger and NLTagScheme allow you to chunk text into specific, labeled types. There are several built-in tagging schemes available, and you can specify your own.
• NLTokenizer can break up text into documents, paragraphs, sentences or words.
• Use Create ML and MLTextClassifier to train your own models to classify larger chunks of text, like sentences, paragraphs or documents.
• Use Create ML and MLWordTagger to train models to classify text at the word level.
• NLModel wraps Create ML models like MLTextClassifier and MLWordTagger in a way that ensures inputs are preprocessed in your app the same way they were during training. It’s also the required type for custom tagging schemes used with NLTagger.

Where to go from here?

This chapter covered most of what Apple makes easy via the Natural Language framework. You can find a completed version of the project in the chapter resources at projects/final/SMDB. When you’re ready, go on to the next chapter, where you’ll learn how to implement more advanced NLP features that involve creating custom models in Keras. You’ll continue working with this app, adding the ability to translate Spanish-language reviews into English.