Main Content

getNumPartitions

Class: matlab.compiler.mlspark.RDD
Package: matlab.compiler.mlspark

Return the number of partitions in an RDD

Syntax

numPartitions = getNumPartitions(obj)

Description

numPartitions = getNumPartitions(obj) returns the number of partitions in obj.

Input Arguments

expand all

An input RDD, specified as an RDD object.

Output Arguments

expand all

Number of partitions in the input RDD, returned as a scalar value.

Examples

expand all

Use the getNumPartitions method to return the number of partitions in an RDD.

%% Connect to Spark
sparkProp = containers.Map({'spark.executor.cores'}, {'1'});
conf = matlab.compiler.mlspark.SparkConf('AppName','myApp', ...
                        'Master','local[1]','SparkProperties',sparkProp);
sc = matlab.compiler.mlspark.SparkContext(conf);

%% getNumPartitions
inputRDD = sc.parallelize({'A','B','C','A','B'},2);
redRDD= inputRDD.map(@(x)({x,1})).reduceByKey(@(x,y)(x+y),3);
coaRDD = redRDD.coalesce(2); % {{{'B',2}},{{'C',1},{'A',2}}}*
disp(['Number of Partitions: ' num2str(coaRDD.getNumPartitions())]);
Introduced in R2016b