Main Content


Read large collections of data

The datastore function creates a datastore, which is a repository for collections of data that are too large to fit in memory. A datastore allows you to read and process data stored in multiple files on a disk, a remote location, or a database as a single entity. If the data is too large to fit in memory, you can manage the incremental import of data, create a tall array to work with the data, or use the datastore as an input to mapreduce for further processing. For more information, see Getting Started with Datastore.


alle erweitern

datastoreCreate datastore for large collections of data
tabularTextDatastoreDatastore for tabular text files
spreadsheetDatastoreDatastore for spreadsheet files
imageDatastoreDatastore for image data
parquetDatastoreDatastore for collection of Parquet files (Seit R2019a)
fileDatastoreDatastore with custom file reader
arrayDatastoreDatastore for in-memory data (Seit R2020b)
readRead data in datastore
readallRead all data in datastore
previewPreview subset of data in datastore
hasdataDetermine if data is available to read
resetReset datastore to initial state
writeallWrite datastore to files (Seit R2020a)
subsetCreate subset of datastore or FileSet (Seit R2019a)
isSubsettableDetermine whether datastore is subsettable (Seit R2022b)
shuffleShuffle all data in datastore
isShuffleableDetermine whether datastore is shuffleable (Seit R2020a)
numpartitionsNumber of datastore partitions
partitionPartition a datastore
isPartitionableDetermine whether datastore is partitionable (Seit R2020a)


combineCombine data from multiple datastores (Seit R2019a)
transformTransform datastore (Seit R2019a)


CombinedDatastoreDatastore to combine data read from multiple underlying datastores (Seit R2019a)
SequentialDatastoreSequentially read data from multiple underlying datastores (Seit R2022b)
TransformedDatastoreDatastore to transform underlying datastore (Seit R2019a)
KeyValueDatastoreDatastore for key-value pair data for use with mapreduce
TallDatastoreDatastore for checkpointing tall arrays


alle erweitern Base datastore class parallelization support to datastore subset and fine-grained parallelization support to datastore (Seit R2022b) Add Hadoop support to datastore (Seit R2019a) shuffling support to datastore File-set object for collection of files in datastore File-reader object for files in a datastore file writing support to datastore (Seit R2020a) Folder property support to datastore (Seit R2020a) File-set for collection of files in datastore (Seit R2020a) Blocked file-set for collection of blocks within file (Seit R2020a)