Skip to Main Content
Spotfire Ideas Portal
Status Future Consideration
Categories data sources
Created by Guest
Created on Apr 23, 2018

File-Delimited Data Sources - pushdown filtering criteria

For File-Delimited data sources (e.g, CSVs), allow for pushdown of filtering criteria from the CIS level, to the data source.

 

e.g., if given a SELECT query to do, i.e, SELECT * FROM {CSV_DATASOURCE} WHERE column LIKE '%value%' - this should be translated to something suitable to pass onto the data source adapter, as, e.g, a grep statement, so that only lines of the file containing the correct matched value will be returned.

 

With large File-Delimited data sources, pulling in the entire file on every query is extremely inefficient, and bottlenecks reporting processes. Using RDMS caching as an ETL-style workaround is an additional complexity that should not be required. 

  • Attach files