Skip to Main Content
Spotfire Ideas Portal
Status Future Consideration
Product Spotfire
Categories Data Management
Created by Guest
Created on Oct 27, 2016

Data On Demand SQL Optimization - Limit by Filtered Rows

We use data on demand with "Limit by Filtered Rows" from a lookup tabl quite frequently to allow users to define the data set that is loaded and mange data size. Typically there are multiple filters  with a large number of unique values. When the user selects all values [(All) x Values]  in one or more of the filters the SQL where clause  that is generated can become extremely large. This is because each unique combination of  value the filters  becomes one condition in the where clause.  (Observed on Teradata).  This greatly effects the performance of the SQL and in some cases creates SQL that exceeds the maximum length of the database.  If I were manually writing the SQL I would just exclude any filter set to "all values" from the where clause. I understand why this is not done by default by the SQL generator, but I would think we could add an option that could be enabled when appropriate.

 

We are starting to revert to using Python to generate custom SQL rather than using the data on demand because of performance and lack of flexibility.   (I will submit a related idea for allowing to dynamically select which columns in the information link are returned.)

  • Attach files