- We have tested to export models from PyTorch and TensorFlow.
- Pre-trained models for a downstream task are supported. This comprises text-classification (e.g. sentiment, multiclass or multilabel models) as well token-classification (e.g. for named entity recognition NER).
- Models with a different structure then BERT (like Roberta etc.) are not supported.
- Only models that use the word-piece tokenizer are currently supported.
- Install the Mathworks implementation of BERT and add it to the Matlab path (not included).
- Install Python (we have only tested Python 3.9.x)
- Extract the toolbox in a specific folder (e.g. “exportBertToMatlab”) and add it to the Matlab path.
- Generate an environment using the “bert2matlab.yml” provided in our “Python”-folder. This installs PyTorch, TensorFlow, and HuggingFace’s “transformers” libraries, to be able to import the pre-trained Python models. GPU support is not necessary.
- A specific IDE is not necessary to export models, you can use the Python command line interface.
- For example, these commands will export a plain, pre-trained German BERT model from HuggingFace, where the import syntax consists of the HuggingFace model name, the type of model (“none”, “text-classification”, or “token-classification”), and the model format (“tf” or “pt”):
- For the Python syntax to import a model from HugingFace, see the included "MinimalExample.txt" file.
- Use the "readBertFromPython.m" function to load the model into Matlab and use powerful NLP.
- The sbert-embeddings are mostly based on the MPNet-BERT model with relative positional encoding, which would require a modification of the Mathworks BERT-implementation.
- Multilingual sentence embeddings rely on Byte-Pair-Encoding (BPE) for which we don’t have a compatible implementation.
Moritz Scherrmann (2023). export BERT to MATLAB: Load pre-trained BERT models (https://www.mathworks.com/matlabcentral/fileexchange/125305-export-bert-to-matlab-load-pre-trained-bert-models), MATLAB Central File Exchange. Retrieved .
MATLAB Release Compatibility
Platform CompatibilityWindows macOS Linux
Inspired by: Transformer Models
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!Start Hunting!