4. Getting Started with Python & Turi Create

Written by Audrey Tam & Matthijs Hollemans

Congratulations! If you’ve made it this far, you’ve developed a strong foundation for absorbing machine learning material. However, before we can move forward, we need to address the 10,000 pound snake in the room… Python. Until this point, you’ve made do with Xcode and Swift, however, if you’re going to get serious about Machine Learning, then it’s best you prepare yourself to learn some Python. In this chapter,

• You’ll learn how to set up and use tools from the Python ecosystem for data science and machine learning (ML).
• You’ll install Anaconda, a very popular distribution of Python (and R).
• You’ll use terminal commands to create ML environments which you’ll use throughout this book.
• Finally, you’ll use Jupyter Notebooks, which are very similar to Swift Playgrounds, to explore the Python language, data science libraries, and Turi Create, Apple’s ML-as-a-Service.

Starter folder

The starter folder for this chapter contains:

• A notebook folder: The sample Jupyter Notebook data files.
• .yaml files: Used to import pre-configured environments, if you want to skip the instructions for configuring the environments yourself.

Python

Python is the dominant programming language used for data science and machine learning. As such, there’s a myriad of tools available for the Python community to support data science and machine learning development. These include:

• Data science libraries: Matplotlib, NumPy, Pandas, SciPy and others.
• Machine learning libraries: Caffe, Keras, Microsoft Cognitive Toolkit, PyTorch, TensorFlow, scikit-learn and others.
• ML-as-a-Service: Amazon Machine Learning, Google ML Kit, IBM Watson, Microsoft Azure Machine Learning Studio, Turi Create and others.
• Tools: coremltools, pip, Anaconda, Docker, Jupyter notebooks, Google Colaboratory and others.

If you know the Swift programming language, you’ll find that although Python is quite different, it also shares some similarities with Swift. For instance:

• You import modules similarly to Swift modules.
• It has the similar concepts for primitive types, tuples, lists, dictionaries, operators, loops and conditionals.
• You can create objects, classes and functions.

Of course, there are some differences too. For example:

• Python is interpreted, not compiled.

• You define closures, functions, classes with indentation instead of { ... }.

• Naming conventions tend toward terse abbreviations, similar to C programming.

• Module and function names are snake_case, while class names and exception names are PascalCase.

• Comments start with # instead of //.

• Multi-line comments begin and end with """ instead of /* and */, and the end """ is on its own line. These are similar to the multi-line strings in Swift.

• True/False, not true/false.

• Dynamic types, no support for constants; no let or var.

• Enumerations, but no switch.

After you set up the tools, you’ll try out some Python while learning about the libraries. If you’d like some more practice or information, here are two helpful resources:

• Michael Kennedy’s November 2014 Comparison of Python and Swift Syntax: bit.ly/2AXQ1UF.
• Jason Brownlee’s May 2016 Crash Course in Python for Machine Learning Developers includes NumPy, Matplotlib and Pandas examples: bit.ly/2MqBCWD.

Packages and environments

Python is already installed on macOS. However, using this installation may cause version conflicts because some people use Python 2.7 while others use Python 3.x, which are incompatible branches of the same language. To further complicate things, working on machine learning projects requires integrating the correct versions of numerous software libraries, also known as “packages”.

Note: The Python development team will stop supporting Python 2.7 in 2020 (https://www.python.org/dev/peps/pep-0373/#update), so the major open source Python packages have pledged to drop support for Python 2.7 no later than 2020 (https://python3statement.org).

Most people create environments where they install specific versions of Python and the packages they need. You can have multiple of these environments on the same computer, each with its own Python interpreter and its own set of Python packages.

The most basic toolset includes the environment manager virtualenv and the package manager pip. Aside from setting up the environment, you still have to figure out which versions of which packages you need — a very manual process, with a high probability of frustration.

There is a better way!

Conda

The data science community developed Conda to make life easier. Conda handles Python language versions, Python packages, and associated native libraries. It’s both an environment manager and a package manager. And, if you need a package that Conda doesn’t know about, you can use pip within a conda environment to grab the package.

Conda comes in two distributions:

• Miniconda: Includes only the packages needed to run Conda. (400 MB)
• Anaconda: Includes all of the standard packages needed for machine learning. (2 GB)

You’ll be using Anaconda in this chapter. It doesn’t take long to install, and it’s way more convenient!

Installing Anaconda

In a browser, navigate to https://www.anaconda.com/download/#macos, and download the 64-bit Command Line installer with Python 3.7, as highlighted in the image below:

At the time of writing, the filename downloaded is called Anaconda3-2019.07-MacOSX-x86_64.sh. After downloading is complete, open up a Terminal and navigate to the directory in which you downloaded the installer. You can run the installer by running the following command in the terminal:

sh Anaconda3-2019.07-MacOSX-x86_64.sh

You’ll have accept the licence agreement, and then give the installer a directory to install Anaconda (or accept the default location if that works for you). Once the installation starts, it may take a while.

While you’re waiting for the installation to finish, scroll down to the Get Started links and take a closer look at Anaconda Training:

These are video courses about using Python for machine learning. You can view some parts for free, while others require you to be a subscriber before you can watch.

If you’re asked to run conda init, type yes. Once installation is complete, restart Terminal. Once restarted, you can try to run the following command to check that the installation succeeded.

conda --version

If the above command fails with a command not found message, chances are you’ll need to add the Anaconda install path to to your global path environment variable. This means, you’ll have to edit the .bashrc or .zshrc file in your home directory (usually found /Users/<username>/). If the file doesn’t exist, you’ll have to create based on the shell your terminal is currently using. If you’re running Catalina or later, this mean your currently shell is most likely Zsh.

In either case, open or create a your .zshrc or .bashrc and either find or add a line that resembles the one given below. Assuming you installed Anaconda in your home directory, the line could look:

export PATH="/Users/<username>/anaconda3/bin":"${PATH}"

In this line, you prepend the path to the anaconda installation to your existing path. Two important things to note include the inclusion of the bin directory to the path, and the colon separating the installation path with the existing PATH variable.

Close any existing Terminal windows, and open a new one. Try running the conda --version command again in the new terminal. Opening a new window will pick up any changes to environment in the .zshrc or .bashrc file. You should have a working Anaconda installation at this point.

Using Anaconda Navigator

Anaconda comes with a desktop GUI that you can use to create environments and install packages in an environment. However, in this book, you’ll do everything from the command line. Given this fact, it’s worth going over some basic commands with Conda which you’ll do in the next section.

Useful Conda commands

As mentioned before, Conda is a package and environment management system. When working with Python projects, you’ll often find it useful to create new environments, installing only the packages you need before writing your code. In this section, we’ll explore many useful commands you’ll reuse many times when working with Python and Conda.

Below are the commands used in this chapter, along with some other useful commands.

Note: Some command options use two dashes. One-dash options are often abbreviations of two-dash options, for example, -n is short for --name.

Another Note: Some Conda environment management tasks can be done in two ways: conda env <command> or conda <different command> <options>

Basic workflow

Create a new environment:

conda create -n <env name>

Clone an existing environment to create a new environment:

conda create -n <new env name> --clone <existing env name>

Create a new environment from a YAML file:

conda env create -f <.yaml file>

The first line of the YAML file sets the new environment’s name. The starter folder for this chapter contains YAML files for mlenv and turienv. If you prefer the GUI to the command line, you can also import these into Anaconda Navigator.

Activate an environment:

conda activate <env name>

Once you’ve activated an environment, the Terminal prompt shows the name of the active environment in parenthesis, like so:

(envname) $

That way it’s always obvious what environment you’re currently using.

Install packages in an active environment:

conda install <pkg names>

Install packages in a non-active environment:

conda install -n <env name> <pkg names>

Note: A message from conda about installing multiple packages: It is best to install all packages at once so that all of the dependencies are installed at the same time.

Install non-conda packages or TensorFlow and Keras in an active environment: Use pip install instead of conda install. To install multiple packages, create a requirements.txt file listing the packages, one per line, then run this command:

pip install -r requirements.txt

Start Jupyter from the active environment [in a specific directory]:

jupyter notebook <directory path>

Shutdown Jupyter: Logout in the Jupyter web pages, then press Control-C-C in terminal window where server is running. (That’s not a typo, you have to press C twice.)

Deactivate an environment: Run this command in the terminal window where you activated the environment:

conda deactivate

Remove an environment:

conda remove -n <env name> --all

Or

conda env remove -n <env name>

Listing environments or packages

List the environments you’ve created; the one with the * is the currently active environment:

conda info --envs

Or:

conda env list

List packages or a specific package in the active environment:

(activeenv) $ conda list
(activeenv) $ conda list <package name>

In a non-active environment:

conda list -n <env name>
conda list -n <env name> <package name>

OK, that was a lot of commands to throw at you. However, the more Python you work with, the more these commands will come in handy. Just having them in the back of your mind will help you move more quickly. If you ever need need a quick refresher, checkout this printable Conda cheat sheet: https://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf.

Setting up a base ML environment

In this section, you’ll set up some environments. If you prefer a quicker start, create an environment from myenv.yaml and skip down to the Jupyter Notebooks section. You can do this by importing mlenv.yaml into Anaconda Navigator or by running the following command from a Terminal window:

conda env create -f starter/myenv.yaml

Python libraries for data science

Begin by creating a custom base environment for ML, with NumPy, Pandas, Matplotlib, SciPy and scikit-learn. You’ll be using these data science libraries in this book, but they’re not automatically included in new Conda environments.

Here’s an overview of what each of these libraries are:

• NumPy: Functions for working with multi-dimensional arrays.
• Pandas: Data structures and data analysis tools.
• Matplotlib: 2D plotting library.
• Seaborn: Statistical data visualization library.
• SciPy: Modules for statistics, optimization, integration, linear algebra, Fourier transforms and more, using NumPy arrays.
• scikit-learn: Machine learning library.
• ipython and jupyter: A Swift-like playground for Python.

Once you have the custom base environment for ML, you can clone it to create separate environments for the ML libraries, Keras, TensorFlow and Turi Create.

From the command prompt in Terminal, create a new environment named mlenv, with Python 3.7:

conda create -n mlenv python=3.7

Type y to proceed with installing the base packages of the environment. Next, activate the environment:

conda activate mlenv

You should see (mlenv) in the command propmpt. Finally, install a bunch of packages at once:

conda install numpy pandas matplotlib seaborn scipy scikit-learn scikit-image ipython jupyter

Type y again to proceed and wait for the installation to finish.

Note: It’s possible that when you’re reading this book, the Anaconda download will be for Python version 3.8 or later. However, in all the environments used by this book, you will need to use Python 3.7. This means, when you create your environment, be sure to specify the Python version. If you choose another version of Python, some of the machine learning libraries you’ll need for the book may not work with that version.

An important note about package versions

Technology moves fast, also in the world of Python. Chances are that by the time you read this book, newer versions are available for the packages that we’re using. It’s quite possible these newer versions may not be 100% compatible with older versions.

For example, in this book we use Keras version 2.2.4. But newer versions of Keras may not work with some of the code examples in this book. Even version 2.2.1, which seemed like a minor upgrade from 2.2.0 that shouldn’t have much of an impact, actually broke things.

Here’s a dirty little secret you should be aware of: You do not need to use the latest, greatest version of these packages. Keras 2.2.4 works fine for our purposes and we can’t keep updating the book every time a new version comes out and breaks something.

So, don’t feel compelled to always upgrade to the newest versions. If you’ve set up a Python environment for a machine learning project and it works well, then don’t fix what isn’t broken. It’s not uncommon for people in the industry to use versions of packages that are 6 months to a year old.

Our advice: If your code works fine and you don’t need any of the new features or essential bug fixes, then keep your Python installation stable and only update your packages when you have a good reason.

Jupyter Notebooks

With Jupyter Notebooks, which are a lot like Swift Playgrounds, you can write and run code, and you can write and render markdown to explain the code.

Starting Jupyter

From Terminal, first activate your environment and then start Jupyter:

$ conda activate mlenv
$ jupyter notebook

If you’re using Anaconda Navigator, in the Home tab select mlenv and click the Jupyter Launch button. The following command appears in a new Terminal window, followed by messages about a server starting and how to shut it down:

/anaconda3/envs/mlenv/bin/jupyter_mac.command ; exit;

Keep this Terminal window open!

A web browser window also opens, showing your home directory:

Navigate to the starter folder for this chapter, and open notebook/mlbase.ipynb:

The notebook appears in a new browser tab:

Pandas and Matplotlib

The notebook has a single empty cell. In that cell, type the following lines:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

This imports the NumPy and Pandas modules into the current session. It also imports the pyplot module of Matplotlib, and gives everything their customary abbreviated aliases: np for NumPy, pd for Pandas and plt for Matplotlib. There’s no output for import statements, although you might see a warning about future deprecation of a module.

Press Shift-Enter to run this code, which will also open a new cell below it.

Next, type these lines in the newly created empty cell, then press Shift-Enter again:

data = pd.read_json('corpus.json', orient='records')
data.head()

The starter/notebook folder contains the file corpus.json. The code you just entered loads the data from this JSON file into a DataFrame — the Pandas data container, with rows and columns like a spreadsheet. It has powerful functions for manipulation, which is important for massaging data to get the best input for training a model.

The orient parameter indicates the JSON string format: 'records' means it’s a list of column -> value. You’ll take a look at the documentation for this function in a moment.

The head() function shows the (default) first five rows:

Note: Shift-Enter runs the current cell and, if this is the last cell, opens a new cell below it; this is convenient when you’re testing code as you write it. Control-Enter runs the current cell; you’d do this when you add something to an earlier cell and want to update it. The bracketed numbers in the margin keep track of the order you run the cells, regardless of their order within the notebook.

In the next empty cell, type the following line, then press Shift-Enter:

?data.tail

The question mark shows the documentation for this function, instead of running the function:

Press Esc or the x button to close the documentation.

Replace the code in the cell with a call to tail:

data.tail(3)

Then press Control-Enter or Shift-Enter to run the cell, which will display the last three rows of data:

If you’d like, you can also see documentation in a pop-up box: select pd.read_json in the second cell, then press Shift-Tab-Tab:

The question mark doesn’t work on this line unless you delete data =.

In the next empty cell, type data.d. Then press Tab to see a list of options:

Now, press Enter to select data.describe. Then type (), and press Shift-Enter:

The output includes the column identifiers: author, text, and title. You can use these to sort the data.

Next, Shift-Enter the following line:

data.sort_values(by='title')

You can extract a column into a separate Series object and count how often each value appears:

authors = data.author
freq = authors.value_counts()
freq

As in Swift Playgrounds, an object name (freq) on a line by itself displays that object.

Frequency varies from 6 to 361. You can plot a histogram of this distribution:

plt.hist(freq, bins=100)
plt.show()

Specifying bins=100 divides the range [6, 361] into 100 consecutive, non-overlapping intervals, called bins or buckets. The histogram’s x-axis has 100 bins, between 0 and 361-ish. The y-axis shows the number of authors in each bin.

Note: This example is from our tutorial Natural Language Processing on iOS with Turi Create which you can find here: bit.ly/2NhAEwf. It trains a natural language model with lines from poems by famous authors. The trained model can be used to classify new text. For each author it knows about, it computes the probability that this author wrote the new text. The freq values here should set off alarm bells — there’s way too much bias toward Emily Dickinson, so the model will classify most test texts as written by her.

Differences between Python and Swift

In this section, you’ll spend some time getting familiar with common Python syntax.

A major syntax difference between Python and most other programming languages is the importance of indentation. With Python, indentation replaces {} to define blocks. For example, an if-statement looks like this:

if a == b:
    print('a and b are equal')
    if a > c:
        print('and a is also greater than c')

Python also has a built-in None type to represent “no value”. This is similar to Swift’s nil but Python does not have optionals. To test for a no-value result, you should use is or is not, instead of the == you’d use in Swift.

if authors is None:
    print('authors is None')
else:
    print('authors is not None')

The output is:

authors is not None

Here’s how you define and call a function:

def mysum(x, y):
    result = x + y
    return result

print(mysum(1, 3))

This outputs 4.

Notice the indentation on the lines inside the function. You have to un-indent the line with print, so that Python knowns this line is outside the function. Coding convention says to leave an extra blank line after the function definition, but it’s not a syntax rule, and you may be more comfortable omitting the blank line.

Also notice how you just wrote result = x + y to put the sum into a new variable. There is no need to write let or var in Python.

Here’s an example of how to use a loop and a list:

mylist = [1, 2]
mylist.append(3)
if mylist:
    print('mylist is not empty')

for value in mylist:
    print(value)

print('List length: %d' % len(mylist))

Lists in Python are similar to arrays in Swift. To test whether a list is empty, use its name. for loops are also similar to Swift, but they use the : plus indentation syntax. The len() function works on any Python collection object, and it returns the length of the list, in a similar way to how the .count property in Swift returns the number of items in an array.

Run those commands, and you’ll see this output:

mylist is not empty
1
2
3
List length: 3

To make a point about indentation, go ahead and add a blank line, but indent the last statement to match the print statement in the loop, like so:

for value in mylist:
    print(value)

    print('List length: %d' % len(mylist))

Now, both print statements are considered to be inside the loop, and so the output becomes:

1
List length: 3
2
List length: 3
3
List length: 3

By the way, string literals in Python can use single quotes or double quotes (or even triple quotes for multiline strings). It doesn’t really matter which one you use, just pick a style you like and be consistent with it. Writing 'List length: %d' % len(mylist) is similar to doing String(format: "List length: %d", myList.count) in Swift. Python 3.6 also has string interpolation, just like in Swift, but this isn’t commonly used yet.

Excellent, you survived a session with Python and used a few library functions! Feel free to play around some more until you get the hang of it. This book uses a lot of Python libraries and functions, so it’s good to understand the basic syntax before moving on.

Transfer learning with Turi Create

Despite the difference in programming languages, deep down Turi Create shares a lot with Create ML, including transfer learning. With Turi Create v5, you can even do transfer learning with the same VisionFeaturePrint_Scene model that Create ML uses.

In this section, you’ll create the same HealthySnacks model as the previous chapter, except this time, you’ll use Turi Create. Unlike Create ML, which allowed you to train your model through the playgrounds UI in Xcode, Turi Create needs some coding when compared to Create ML. This means you’ll learn more about working with Python.

Creating a Turi Create environment

First, you need a new environment with the turicreate package installed. You’ll clone the mlenv environment to create turienv, then you’ll install turicreate in the new environment. Conda doesn’t know about turicreate, so you’ll have to pip install it from within Terminal.

Note: Again, if you prefer a quicker start, import turienv.yaml into the Navigator, or run conda env create -f starter/turienv.yaml, and skip down to the section Turi Create Notebook.

While it’s possible to clone mlenv in Anaconda Navigator’s Environments tab, you’ll be using a command line to install turicreate, so it’s just as easy to use a command line to clone, as well.

Note: If you’ve changed Terminal’s default shell to something different from bash, check that your $PATH includes ~/anaconda3/bin (or whatever directory you install anaconda into).

Open a new Terminal window, and enter this command:

conda create -n turienv --clone mlenv

This creates an environment named turienv, which is cloned from mlenv.

Wait a little while until you see the message:

#
# To activate this environment, use:
# > conda activate turienv
#
# To deactivate an active environment, use:
# > conda deactivate
#

Note: If you see a message to update Conda, go ahead and do that.

Time to install Turi Create into this environment. From the same Terminal window, enter the activate command:

conda activate turienv

The command line prompt now starts with (turienv), showing it’s the active environment.

Enter this command to install the turicreate package:

pip install -U turicreate==5.8

This downloads and installs the newest available version of the turicreate package, which lets you use the Vision framework model for transfer learning.

List pip-installed packages

In Terminal, use this command to list all of the packages in the active environment or a specific package:

conda list
conda list coremltools

You need the coremltools package to create Core ML models from Turi Create models. Installing turicreate also installs coremltools.

The output of the second command looks similar to this:

# packages in environment at /Users/amt1/anaconda3/envs/mlenv:
#
# Name                    Version                   Build  Channel
coremltools               3.0                       <pip>

The Build Channel value <pip> shows coremltools was installed with pip, not conda.

Note: If you a quick look at the turienv environment in Navigator; it still shows only 105 packages. That’s because packages installed with pip don’t show up in Navigator.

Turi Create notebook

Note: If you skipped the manual environment setup and imported turienv.yaml into Anaconda Navigator, use the Jupyter Launch button on the Anaconda Navigator Home Tab instead of the command line below, then navigate in the browser to starter/notebook.

This time, you’ll start Jupyter in the folder where the notebooks are stored; locate starter/notebook in Finder.

Note: If you downloaded the snacks dataset for the previous chapter, copy or move it into starter/notebook. Otherwise, double-click starter/notebook/snacks-download-link.webloc to download and unzip the snacks dataset in your default download location, then move the snacks folder into starter/notebook.

In Terminal, enter the following command to start a Jupyter notebook in the turienv environment, starting from this directory:

jupyter notebook <drag the starter/notebook folder in Finder to here>

In the browser, open HealthySnacks-Turi.ipynb. There’s only an empty cell.

Type the following commands in this cell and press Shift-Enter:

import turicreate as tc
import matplotlib.pyplot as plt

You’re importing the Turi Create package and the pyplot module of the Matplotlib package into the current session, with aliases tc and plt. You may get a FutureWarning message, which you can safely ignore.

In the next cell, Shift-Enter this command (put it all on one line):

train_data = tc.image_analysis.load_images("snacks/train",
                                           with_path=True)

This loads all the images from the snacks/train directory into an SFrame, the data container for Turi Create. An SFrame contains rows and columns, like a Pandas DataFrame — in fact, you can create an SFrame from a DataFrame. SFrame has powerful functions for manipulation, similar to DataFrame. It’s also optimized for loading from disk storage, which is important for large data sets that can easily overwhelm the RAM.

Like Create ML’s MLDataTable, an SFrame keeps only the image metadata in memory.

Note: It’s safe to ignore warnings about .DS_Store being an unsupported image format.

This SFrame object contains a row for each image, as well as the path of the folder the images were loaded from. This SFrame should contain 4838 images. Verify this by asking for its length:

len(train_data)

Note: Run each command in its own cell. Remember Shift-Enter runs the current cell and opens a new cell below it. Always wait for the [*] in the margin to turn into a number, indicating the command has finished running.

Next, look at the actual contents of the SFrame:

train_data.head()

The head() function shows the first 10 rows:

Even though the SFrame only shows the image’s height and width in the table, it actually contains the complete image. Run the following command to see the actual images:

train_data.explore()

This opens a new window with image thumbnails (it may take a few seconds to load). Hover over a row to view a larger version of an image.

This interactive visualization can be useful for a quick look at the training data. The explore() command only works with Turi Create on the Mac, not on Linux or from a Docker container.

Enter this command to look at individual images directly inside the notebook, using Matplotlib’s imshow() command:

plt.imshow(train_data[0]["image"].pixel_data)

Here, train_data[0] gets the first row from the SFrame, ["image"] gets the object from the image column for that row, and .pixel_data turns this image object into something that matplotlib can show with the plt.imshow() command.

Your notebook may show a different image than in the illustration, since Turi Create may have loaded your images in another order. Feel free to look at a few images by changing the row index (use any value from 0 to 4,837).

There is one more piece of data to gather before you can start training — the name of the class for each image. The images are stored in subdirectories named after the classes — “apple,” “hot dog,” etc. The SFrame knows the path the image was loaded from, but these paths look something like this:

snacks/train/hot dog/8ace0d8a912ed2f6.jpg

The class for image 8ace0d8a912ed2f6.jpg is “hot dog”, but it’s hidden inside that long path. To make this more obvious, you’ll write some code to extract the class name from the path. Run the following commands to extract the name of the first image’s class folder:

# Grab the full path of the first training example
path = train_data[0]["path"]
print(path)

# Find the class label
import os
os.path.basename(os.path.split(path)[0])

Here, you’re getting the full path of the first image, then using the os.path Python package for dealing with path names. First, os.path.split() chops the path into two pieces: the name of the file (8ace0d8a912ed2f6.jpg) and everything leading up to it. Then os.path.basename() grabs the name of the last folder, which is the one with the class name. Since the first training image is of an apple, you get “apple.”

Note: The # character starts a comment in Python. Note that you first need to import the os package, or else Python won’t know what os.path is.

Getting the class labels

OK, now you know how to extract the class name for a single image, but there are over 4,800 images in the dataset. As a Swift programmer, your initial instinct may be to use a for loop, but if you’re really Swift-y, you’ll be itching to use a map function. SFrame has a handy apply() method that, like Swift’s map or forEach, lets you apply a function to every row in the frame:

train_data["path"].apply(lambda path: ...do something with path...)

In Python, a lambda is similar to a closure in Swift — it’s just a function without a name. train_data["path"].apply() performs this lambda function on every row in the path column. Inside the lambda, put the above code snippet that you used to extract the class name from the full path:

train_data["label"] = train_data["path"].apply(lambda path:
    os.path.basename(os.path.split(path)[0]))

Run the above cell and now the SFrame will have a new column called “label” with the class names. To verify this worked, run train_data.head() again — do this in a new cell, or scroll up to the fourth cell, and press Control-Enter to run it.

You can also use train_data.explore() again for a visual inspection. Run this command to see the summary function:

train_data["label"].summary()

This prints out a few summary statistics about the contents of the SFrame’s label column:

As you can see, each of the classes has roughly the same number of elements. For some reason, summary() only shows the top 10 classes, but we have 20 in total. To see the number of rows for all of the classes, run the following command:

train_data["label"].value_counts().print_rows(num_rows=20)

All right, that’s all you need to do with the data for now. You’ve loaded the images into an SFrame, and you’ve given each image a label, so Turi Create knows which class it belongs to.

Let’s do some training

Once you have your data in an SFrame, training a model with Turi Create takes only a single line of code (OK, it’s three lines, but only because we have to fit it on the page):

model = tc.image_classifier.create(train_data, target="label",
                                   model="VisionFeaturePrint_Scene",
                                   verbose=True, max_iterations=50)

Alternatively, if training takes too long on your Mac, you can just load the Turi Create model from the current folder:

model = tc.load_model("HealthySnacks.model")

This command creates a new image classifier from the train_data SFrame. The target parameter tells Turi Create that the class names are in the SFrame’s label column. By default, Turi Create only does 10 iterations, but you increase this to 50, so the logistic regression will train for up to 50 iterations.

The first time you run this command, Turi Create downloads a pre-trained neural network. The model parameter contains the name of that neural network, in this case VisionFeaturePrint_Scene. This is the model used by Apple’s Vision framework, and is also the default model for Create ML.

At the time of writing, Turi Create supports three model architectures: The other two are ResNet-50 and SqueezeNet version 1.1. ResNet-50 exports a Core ML model ~90MB, which is not really suited for use on mobile devices.

SqueezeNet exports a Core ML model ~4.7MB, so it’s a better option. But VisionFeaturePrint_Scene is built into iOS 12, so it produces a much smaller model — only ~41 KB.

Turi Create, like Create ML, performs feature extraction on the images. This takes about the same amount of time as Create ML — 2m 22s on my MacBook Pro. And then comes the logistic regression:

Validation

After 15 iterations, validation accuracy is close to training accuracy at ~90%. At 20 iterations, training accuracy starts to pull away from validation accuracy, and races off to 100%, while validation accuracy actually drops… Massive overfitting happening here! If the validation accuracy gets worse while the training accuracy still keeps improving, you’ve got an overfitting problem.

It would’ve been better to stop training the model after about 15 iterations. But running the image_classifier.create command with max_iterations=15 will also do the feature extraction all over again! Too bad Turi Create doesn’t let you save the intermediate states of the model, or stop the training when the validation accuracy shows a decreasing trend.

Actually, in the next chapter, you’ll learn how to wrangle the Turi Create code — it’s open source, after all! — to save the extracted features, so you can experiment more with the classifier.

Spoiler alert: Keras, which we’ll talk about in an upcoming chapter, lets you save the best-so-far model while it’s training, so you can always retrieve the results from an earlier iteration in case your model suffers from overfitting. Keras also lets you stop early if validation accuracy doesn’t improve over some given number of iterations (your choice).

Let’s go ahead and evaluate this model on the test dataset.

Testing

Run these commands to load the test dataset and get the class labels:

test_data = tc.image_analysis.load_images("snacks/test", with_path=True)

test_data["label"] = test_data["path"].apply(lambda path:
       os.path.basename(os.path.split(path)[0]))

len(test_data)

The last command is just to confirm you’ve got 952 images.

Next, run this command to evaluate the model and collect metrics:

metrics = model.evaluate(test_data)

Unlike Create ML, the output of this command doesn’t show any accuracy figures — you need to examine metrics. Run these commands in the same cell:

print("Accuracy: ", metrics["accuracy"])
print("Precision: ", metrics["precision"])
print("Recall: ", metrics["recall"])
print("Confusion Matrix:\n", metrics["confusion_matrix"])

Here are my metrics:

Accuracy:  0.8697478991596639
Precision:  0.8753552272362406
Recall:  0.8695450680272108
Confusion Matrix:
+--------------+-----------------+-------+
| target_label | predicted_label | count |
+--------------+-----------------+-------+
|  ice cream   |      candy      |   1   |
|    apple     |      banana     |   3   |
|    orange    |    pineapple    |   2   |
|    apple     |    strawberry   |   1   |
|  pineapple   |      banana     |   1   |
|  strawberry  |      salad      |   2   |
|   popcorn    |      waffle     |   1   |
|    carrot    |      salad      |   2   |
|    orange    |    watermelon   |   1   |
|   popcorn    |     popcorn     |   36  |
+--------------+-----------------+-------+
[107 rows x 3 columns]
Note: Only the head of the SFrame is printed.
You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.

No surprises: Accuracy, precision and recall are all similar to the final validation accuracy of the model. Unlike Create ML, Turi Create gives only overall values for precision and recall, and you need some code to get precision and recall for each class. In the next chapter, you’ll learn how to get recall for each class.

The confusion matrix shows only the first 10 rows: the model mistook one “ice cream” image for “candy,” three “apple” images for “banana,” etc. Presented this way, it doesn’t look much like a matrix.

In the next chapter, you’ll learn how to get this nifty visualization:

This heatmap shows small values as a cool color — black or dark purple — and large values as warm colors — red to orange to white. The larger the value, the brighter it gets. The correct matches are on the diagonal, so the highest values are there. With only 21 correct matches, “pretzel” stands out, but there are only 25 images in the pretzel folder, so 21 is OK. Purple numbers off the diagonal indicate problems. More about this in the next chapter!

Exporting to Core ML

In the next cell, Shift-Enter this command:

model

This displays information about the model.

Class                                    : ImageClassifier

Schema
------
Number of classes                        : 20
Number of feature columns                : 1
Input image shape                        : (3, 299, 299)

Training summary
----------------
Number of examples                       : 4590
Training loss                            : 1.2978
Training time (sec)                      : 174.5081

Now you will save this model so you can load it with Core ML. There are two ways to save models using Turi Create. First:

model.save("HealthySnacks.model")

This saves the model in Turi Create’s own format, which allows you to load it back into the Python notebook later using tc.load_model(). Once you’ve trained a Turi Create model, you can’t modify it afterwards, but you might want to evaluate it on different test data, or examine the metrics more closely.

Run this command to get a Core ML model:

model.export_coreml("HealthySnacks.mlmodel")

You can add the mlmodel to Xcode in the usual way if you want to compare it with the Create ML model. Despite being based on the same pre-trained model, the two custom models aren’t the same: The accuracy of this model is a little lower, and it’s half the size of the Create ML model.

Shutting down Jupyter

To shut down Jupyter, click the Logout button in this browser window and also in the window showing your ML directory.

In the Terminal window from which you ran jupyter notebook — or the one that ran jupyter_mac.command ; exit; if you used Anaconda Navigator to launch Jupyter — press Control-C to stop the server. You may need to press this twice. If the prompt doesn’t return, close this terminal window.

Deactivating the active environment

If you activated turienv at the terminal command line, enter this command to deactivate it:

conda deactivate

This deactivates the turienv environment; the command line prompt loses the (turienv) prefix.

Docker and Colab

There are two other high-level tools for supporting machine learning in Python: Docker and Google Colaboratory. These can be useful for developing machine learning projects, but we’re not covering them in detail in this book.

Docker is a useful tool for creating reproducible environments for running machine learning projects, and is therefore a useful tool when you want to scale up projects. Colaboratory is a Jupyter notebook in the cloud that gives you access to free GPU. But, while you’re working through the Turi Create and Keras examples in this book and trying out your own modifications, it’s more convenient to have the turienv and kerasenv environments, and know how to build or modify them.

Docker

Docker is like a virtual machine but simpler. Docker is a container-based system that allows you to re-use and modularize re-usable environments, and is a fundamental building block to scaling services and applications on the Internet efficiently. Installing Docker gives you access to a large number of ML resources distributed in Docker images as Jupyter notebooks like hwchong/kerastraining4coreml or Python projects like the bamos/openface face recognition model. Our Beginning Machine Learning with Keras & Core ML (bit.ly/36cS6KU) tutorial builds and runs a keras-mnist Docker image, and you can get comfortable using Docker with our Docker on macOS: Getting Started tutorial here: bit.ly/2os0KnY.

Docker images can be useful to share pre-defined environments with colleagues or peers, but at some point they will require an understanding of how to write Docker images (by editing the corresponding Dockerfile), which is beyond the scope of what we’re covering here.

You can download the community edition of Docker for Mac from https://dockr.ly/2hwNOZZ. To search Docker Hub hub.docker.com (a repository for Docker images), click Explore, then search for image classifier:

Google Colaboratory

Google Research’s Colaboratory at colab.research.google.com is a Jupyter Notebook environment that runs in a browser. It comes with many of the machine learning libraries you’ll need, already installed. Its best feature is, you can set the runtime type of a notebook to GPU to use Google’s GPU for free. It even lets you use Google’s TPUs (tensor processing units).

If you don’t have access to a machine learning capable computer, you can certainly follow along with parts of this book using Colab. However, the authors of this book recommend that readers follow along with a local installation of Python. If you choose to use Colab, you’ll have to perform the following set up. Of course, you will need a Google account to in order to continue.

Access your Google Drive drive.google.com and from the side menu, create a new Folder named machine-learning.

Double click the folder, and drag and drop the unzipped snacks dataset into it. This may take a while. While you wait, you’ll need to add Colab as an “app” to Google Drive. Right-click anywhere in the machine-learning folder, select More from the dialog, and select + Select Connect more apps.

From the Connect apps to Drive window that opens up, search for colab in the search field, and select + Connect.

Once it’s been successfully installed, close the window, and Right-click anywhere again, and from the More dialog, select Colaboratory. This will open a new tab or window with something that should look a lot like a Jupyter notebook.

Rename the file to getting-started.ipynb by clicking the ttile and renaming it inline.

From the toolbar, select Runtime > Change Runtime type.

From the Notebook settings dialog that open, change the Hardware accelerator from None to GPU. Save the changes.

In first code cell of the notebook, paste the following code:

from google.colab import drive
drive.mount('/content/drive/')

These two lines will walk your through mounting your google drive folders into the notebook. This requires giving Colab access to your Google Drive folders. Click the tiny play button beside the code cell. Follow the instructions in the output window by opening the link to give Colab the authority to access your Drive. You’ll be given an access code to paste into your notebook.

Once the mounting is complete, add a new code cell, and run the following piece of code:

!ls "/content/drive/My Drive/machine-learning/snacks"

You may notice that the code starts off with an exclamation. This is Juypter-specific syntax that allows you to run system level commands. In this case, you’re trying to list the contents of the directory in which you uploaded the snacks dataset. If all goes well, you should now be able to set this path as your root directory to the snacks dataset.

You’ve completed setting up a Google Colab notebook environment, configured to use the GPU, that you can use for this book. It’s worth reiterating that using Colab is untested with respect to this book, and you may run into issues while using it. However, it offers a compelling alternative for devselopers looking to do machine learning, but don’t have access to a machine powerful enough to run machine learning algorithms.

Key points

• Get familiar with Python. Its widespread adoption with academics in the machine learning field means if you want to keep up to date with machine learning, you’ll have to get on board.
• Get familiar with Conda. It will make working with Python significantly more pleasant. It allows you to try Python libraries in a controlled environment without damaging any existing environment.
• Get familiar with Jupyter notebooks. Like Swift playgrounds, they provide a means to quickly test all things Python especially when used in combination with Conda.

Where to go from here?

You’re all set to continue learning about machine learning for image classification using Python tools. The next chapter shows you a few more Turi Create tricks. After that, you’ll be ready to learn how to create your own deep learning model in Keras.