Streaming iterator read

[xiaodaigh]

One of the most important features is to be able to read a chunk of SAS and then process it because in the wild there are a lot of large SAS datasets which are 50G~100G in size and they can't fit into memory.

What's stopping wider move away from SAS I think is due to not being able to read the data in chunks and then process it and then read the next chunk.

[kylebarron]

Fyi, pandas includes support for streaming iterator reads for Stata, though it doesn't have such SAS support:
https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_stata.html

[davidanthoff]

Somewhere on my todo list is a plan to add streaming support here and integrate it properly with IterableTables.jl, at which point this should be feasible.