pyspark.sql.datasource.DataSourceReader

class pyspark.sql.datasource.DataSourceReader

A base class for data source readers. Data source readers are responsible for outputting data from a data source.

Methods

partitions()	Returns an iterator of partitions for this data source.
pushFilters(filters)	Called with the list of filters that can be pushed down to the data source.
read(partition)	Generates data for a given partition and returns an iterator of tuples or rows.