Chapter 1

Why Use Deep Learning for Pattern Recognition?

Deep learning is used to develop models that can find patterns in data; however, it isn’t the only method capable of doing this. This chapter explores some of the reasons why you might choose deep learning over another method.

Once a model is trained, unlabeled data can be fed in and the model will apply the most probable label based on the information it learned when training.

Pattern Recognition

Here are two simple examples of pattern recognition: one uses linear regression to determine how sensor data trends over time, the other uses a k-means algorithm to cluster data into similar groups.

The benefits of a lot of these non–deep learning methods are that the logic tends to be more understandable and they use smaller amounts of relatively simpler data. When you train a deep neural network, explaining why it works can be difficult, and the network requires a lot more quality labeled data for training. With that in mind, why would anyone choose deep learning over a different method? One reason is that deep learning is well suited for finding really complex patterns across large solution spaces.

For example, you might want to find and label specific objects in an image for object tracking and localization or you might want to visually inspect hardware for defects and material damage

In these examples, the patterns are specific combinations of pixels that represent things like pedestrians or chipped concrete. These are complex patterns that can be wildly different from instance to instance, or they might vary in subtle ways that are nearly indistinguishable from other patterns. Therefore, not only does the algorithm that you choose need to classify these complex patterns, but it has to do so across a really large solution space.

Complex patterns also exist in non-image data as well. In audio signals, you might need to recognize spoken words or identify a song from a few-second snippet. 

There are also complex patterns in time-series signals like those in predictive maintenance applications and biomedical industries.

In all of these cases, the complexity and range over which the solution exists can make it difficult to come up with rule-based approaches in which a human has to craft and curate the logic themselves. This is where deep learning can be a better approach.

Note: The examples in the ebook focus on classification, in which a discrete label is applied to the data. However, regression, in which the output of the model is a continuous value rather than a discrete label, is also possible with deep learning. See how to train a convolutional neural network (CNN) for regression.

Deep learning uses deep neural networks to model the relationship between the input data and the output classification. The details of neural networks are beyond the scope of this chapter; however, in a simple way you can think of the network as a series of layers that each perform specific operations on the data. Early layers tend to look for smaller features that exist within the larger pattern, and then later layers look at combinations of those features to determine the most probable label for the data.

How Do Features Help Define Patterns?

To understand the benefits of deep learning and how feature identification can help with the larger pattern recognition, imagine you need to develop an algorithm that can recognize high fives from acceleration data.

The plots in the below image show the acceleration pattern for three different arm motions. The middle plot represents a high five. How would you approach developing an algorithm that could distinguish the high-five pattern from all of the other possible arm motions?

If You Know All the Defining Features

You might start by looking for a simple rule-based approach to this problem. One solution could be to use a known high-five profile that you could pattern-match against the incoming continuous stream of acceleration measurements from the sensor. For example, a function could look at the standard deviation between the two signals and could claim there was a high five when the deviation drops below some threshold. 

The difficultly with this approach, however, is that every high five is not the same. Some people might move their hand faster or slower, pause right before the big slap, or just high five in a strange way. Even if you could come up with a representative profile, or set of profiles, an approach like this might not be robust to high-five motions that it hasn’t seen before. Plus, if you add too many profiles or raise the standard deviation threshold to make sure you capture every high five, you run the risk of mislabeling motions as high fives.

A smarter approach might be to realize that instead of looking at the pattern as a whole, which can have wild variations, you can find a set of smaller features that every high five has in common.

If You Don’t Know All the Defining Features

Deep learning can help solve this problem. Features of the pattern, even unintuitive features, are learned automatically without a designer having to explicitly define them. So, if you are trying to classify complex patterns in data and you are unable to define all the rules that distinguish that pattern from others, deep learning might be a good option to explore.

Keep in mind that deep learning has its own set of requirements. You need a lot of quality labeled data, you need the right network architecture, and you need to make sure that the network can be deployed to the target processor and executed in the time allocated for it. The following chapters of this ebook address these requirements.