So I found a solution.
Appereanly the OCR prefers white background and black charecters (and not the other way around)
so the following code does the trick:
load image
imshow(TEMP)
BW = ~imbinarize(TEMP);
imshow(BW)
ocrOutput = ocr(BW, 'TextLayout', 'word', 'characterSet', '0123456789')
ocrOutput =
ocrText with properties:
Text: '297183↵↵'
CharacterBoundingBoxes: [8×4 double]
CharacterConfidences: [8×1 single]
Words: {'297183'}
WordBoundingBoxes: [11 12 638 165]
WordConfidences: 0.9626
TextLines: {'297183'}
TextLineBoundingBoxes: [11 12 638 165]
TextLineConfidences: 0.9626