No changes when using function erasePunctuation to remove digits.
3 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
I'm trying to remove the digits in my document that has been tokenized.
However, using the erasePunctuation function, I didn't see any changes (no digits were removed) to the updated document. I've checked the type, and the tokenizer does recognize these tokens as digits. Please help. Thanks.
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/552387/image.jpeg)
The output:
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/552392/image.jpeg)
0 Kommentare
Antworten (1)
Cris LaPierre
am 16 Mär. 2021
Bearbeitet: Cris LaPierre
am 16 Mär. 2021
erasePunctuation still only erases punctuation, not numbers. The 'digits' specification tells it what type of token to remove punctuation from. See the description here.
You could try to remove digits using the following.
tkD = tokenDetails(cleanDoc);
cleanDoc = removeWords(cleanDoc,tkD{tkD.Type=="digits"});
Siehe auch
Kategorien
Mehr zu Text Data Preparation finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!