Main Content

checkpoint

Class: matlab.compiler.mlspark.RDD
Package: matlab.compiler.mlspark

Mark an RDD for checkpointing

Syntax

checkpoint(obj)

Description

checkpoint(obj) marks an RDD for checkpointing.

Input Arguments

expand all

An input RDD that is to be marked for checkpointing, specified as an RDD object.

Examples

expand all

Use the checkpoint method to save an RDD to a file inside the checkpoint directory.

%% Connect to Spark
sparkProp = containers.Map({'spark.executor.cores'}, {'1'});
conf = matlab.compiler.mlspark.SparkConf('AppName','myApp', ...
                        'Master','local[1]','SparkProperties',sparkProp);
sc = matlab.compiler.mlspark.SparkContext(conf);

%% checkpoint
sc.setCheckpointDir('myDir')
myFile = sc.parallelize({1,2,3});
mapRDD = myFile.map(@(x)({x,1}));
mapRDD.checkpoint();
Introduced in R2016b