Greenplum’s SG Streaming™ technology ensures parallelism by “scattering” data from all source systems across hundreds or thousands of parallel streams that simultaneously flow to all Greenplum Database nodes (Figure 11). Performance scales with the number of Greenplum Database nodes, and the technology supports both large batch and continuous near-real-time loading patterns with negligible impact on concurrent database operations. Data can be transformed and processed in-flight, utilizing all nodes of the database in parallel, for extremely high-performance ELT (extract-load- transform) and ETLT (extract-transform-load-transform) loading pipelines. Final “gathering” and storage of data to disk takes place on all nodes simultaneously, with data.
