Skip to Main Content
Spotfire Ideas Portal
Status Future Consideration
Product Spotfire
Categories Data Access
Created by Guest
Created on Apr 7, 2023

Wildcard or partial string matches for Data on Demand

I'm trying to work with Data on Demand with "string" type fields. We have a large database of error logs, and are trying to search through the log text to find mentions of a specific substring that can appear in slightly different places within the string. Unfortunately, it appears that Data on Demand only matches complete strings. For example, we might want to look for any messages mentioning a sensor with ID "11242", and the error logs might be:

"Error reading Sensor ID 11241 - no connection or signal timeout"
"Error reading Sensor ID A3344 - no connection or signal timeout"
"Value mismatch in Sensors: 11241, A3344"
"Unsolicited response: "Sensor with ID 11241 disconnected""

We would want to be able to load on demand items 1, 3, and 4 by searching for "11241" in the related field. I know we can do partial string matching in filters using asterisks (*11241*), but is there any way to support partial string matching for Data on Demand? Our data logs are so massive that loading all logs into memory and then filtering afterward is infeasible. We already make use of Data on Demand to trim our query down on other fields as much as possible, but this still leaves millions of entries in this particular case.
  • Attach files