Instruction

What is a Learning Model?

All ML models are created through a process called training. This process involves feeding a model large amounts of data and adjusting its parameters to minimize errors in predictions. Depending on the size of the data, the result can require many hours and vast amounts of expensive hardware. The result will be a model that you need to deploy to get any benefit. This is done by storing the model in a model file.

An ML model file is a saved representation of the machine learning model. This preserves the knowledge gained during model training for future use. The model file will define the architecture of the model and hold the parameters and values that define the model’s behavior. It may also contain additional information about the model to help with deploying or tuning the model. These file formats often follow from the environment used to train and test the model during development.

Swift may be the language of choice for Apple platforms, but machine learning focuses on the Python language. While you can train and deploy models without knowing Python, you’ll need to engage with Python to use most models that aren’t specifically built using Create ML. Fortunately, you’ll find that’s not a roadblock, as Apple provides tools to convert the most popular third-party model files into CoreML files that you can easily use in your own apps.

First, you’ll look at some popular machine learning platforms and how each defines its model files.

Different ML Model Types

Since Python is the most used programming language in model development, the most common file formats reflect that Python lineage, even if the frameworks support other languages. While building or using third-party models, you’ll find several different frameworks, each with a particular focus and design paradigm. Here’s an overview of three popular platforms you’ll often see when working with third-party models.

Using TensorFlow

TensorFlow is an end-to-end open-source platform for machine learning that provides Python and C++ APIs. It’s an established platform that’s been around for a long time and has a large community. It prioritizes scalability and production readiness in models, and is well-suited for large-scale deployments and complex models. It’s rigid compared to other platforms, but that provides the developer with finer control over training and the resulting models.

Tensorflow supports several model formats, each optimized for a set of use cases:

• SavedModel: The recommended format for storing TensorFlow models due to its flexibility and portability. It stores a directory containing serialized signatures, variables, and assets which include the model’s architecture, weights, and other necessary components.
• TensorFlow Lite (.tflite): This is a compressed format for deploying models on resource-constrained platforms. It contains a flat binary file with the model’s architecture and quantized weights for a smaller model size and faster inference.
• Protocol Buffer (.pb): A binary format used internally by TensorFlow to represent the computational graph. It stores the model architecture and weights, and is efficient for storing and loading a model, but isn’t suitable for direct human inspection. It’s mostly meant for internal use but sometimes shows up in the format of a third-party model.

Using PyTorch

PyTorch is a machine learning platform that emphasizes flexibility and ease of use. It’s often preferred for research and rapid prototyping. It’s also tied to Python more than TensorFlow and many other models. It has a large and active research community, making it a popular choice for academic research. Its flexibility with dynamic sequences and variable input length also make it popular for Natural Language Processing (NLP) tasks.

Most PyTorch model files use the .pth or .pt file extensions. These files contain a serialized Python dictionary known as the state dictionary. This state dictionary maps each layer of the model to its weights and biases. The state dictionary only stores the model’s parameters and not the model architecture. PyTorch supports saving the entire model, but this results in larger files and limits portability by tying the file to a Python environment.

Using Core ML

This is Apple’s preferred format for their platforms (iOS, iPadOS, macOS, watchOS, tvOS.) These model files are optimized for efficient execution on Apple devices and are focused on deployment rather than development or training. You’ll find the models almost useless on other platforms and produced either using Create ML or by converting from another format. Due to the popularity of Apple devices, especially the iPhone, you’ll find that model developers will often provide Core ML model files for their projects, even if using another platform for development.

All Core ML models use the .mlmodel extension and are natively supported in Xcode.

Converting Models

The correct choice of platform and model file will depend on the platform, model size, deployment environment, and resources, plus the need for interaction with other platforms and tools. You’ll likely find that unless your needs focus solely on Apple platforms, you’ll use TensorFlow, PyTorch, or another machine learning platform to build models that you then convert into Core ML for deployment on iOS devices.

Since many models aren’t natively developed in Core ML, Apple provides Core ML Tools to convert these models into the Core ML format. The coremltools Python package is the primary way to convert third-party models to Core ML. You’ll use Core ML Tools to convert trained models from libraries and frameworks such as TensorFlow and PyTorch to the Core ML model package format. The tools also read, write, and optimize Core ML models to use less storage space, reduce power consumption, and reduce latency during inference. The tools also provide a way to verify creation and conversion by making predictions using Core ML in macOS.

You won’t need extensive knowledge of Python for most tasks, but it’ll be helpful to understand how to set up Python in a way that optimizes your experience with Core ML Tools. In the next section, you’ll set up a Python environment on macOS while setting up Core ML Tools and supporting libraries.