Sarah Palfreyman, MathWorks
Text Analytics Toolbox™ provides tools for extracting text from documents, preprocessing raw text, visualizing text, and performing machine learning on text data. The typical workflow begins by importing text data from documents, such as PDF and Microsoft® Word® files, and then extracting meaningful words from the data. Once text is preprocessed, you can interact with your data in a number of ways, including converting the text into a numeric representation and visualizing the text with word clouds or scatter plots.
Features created with Text Analytics Toolbox can also be combined with features from other data sources to build machine learning models that take advantage of textual, numeric, audio, and other types of data. You can import pretrained word-embedding models, such as those available in word2vec, FastText, and GloVe formats, to map the words in your dataset to their corresponding word vectors. You can also perform topic modeling and dimensionality reduction with machine learning algorithms such as LDA and LSA.
To get started transforming large sets of text data into meaningful insight, download a free trial of Text Analytics Toolbox.
Text Analytics Toolbox provides tools for extracting text from documents, preprocessing raw text, visualizing text, and performing machine learning on text data.
You can use Text Analytics Toolbox to analyze data from sources like maintenance reports, operations logs, financial documents, web and social media sources.
You can extract raw text from a variety of sources including Microsoft Word, Microsoft Excel, and PDF and use word clouds to view the relative frequency of words and interactive scatter plots to understand the numeric relationships between words.
Text Analytics Toolbox provides functions for pre-processing raw text such as removing common words and punctuation and tokenizing documents into individual words or phrases.
Once text is pre-processed, converting text to numeric representations let you do more analysis and visualizations to understand word frequencies including:
Statistics and machine learning algorithms can be used with text analytics to perform topic modeling to identify themes in documents, classify documents and make predictions.
You can train machine learning models or use pre-trained word embedding models such as word2vec, FastText and GloVe.
In this example, the Latent Dirichlet Allocation algorithm is used to build a topic model with 60 topics in storm reports to identify damage and weather patterns.
You can also use deep learning algorithms to build accurate classifiers when you have large sets of documents and use parallel computing to speed up text processing and training.
For more information about Text Analytics Toolbox, see the product page, or choose a link below.