Skip to Main Content
Spotfire Ideas Portal
Status Already exists
Categories data sources
Created by Guest
Created on Jan 30, 2023

Infer data types by introspecting csv files

The TDV standard File-Delimited adapter introspects all columns in the CSV file as VARCHAR type. It would save time if the data types were inferred by looking at the data in the CSV file.

Implemented in 8.5
  • ADMIN RESPONSE
    Jan 31, 2023

    This is already possible using the local file adapter.

  • Attach files
  • Admin
    Tobias Lehtipalo
    Feb 9, 2023

    Thank you for pointing this out. Most customers use TDV on Linux so we are unlikely to implement support for schema inferencing on Windows. However we are tracking this request in TDV-I-362.

  • Guest
    Feb 9, 2023

    Local File Storage adapter cannot infer the DATE type.


    In addition, the Local File Storage adapter is inferior to File-Delimited in the following respects

    ・Fewer character code types are supported (e.g., SJIS, which is used in Japan, is not supported). (such as SJIS used in Japan)

    ・Slower speed during introspection than File-Delimited.


    This makes the Local File Storage adapter incomplete as an adapter that can infer data types.

    From a performance standpoint, please extend the functionality of the File-Delimited adapter so that it can infer data types.

  • Guest
    Feb 7, 2023

    The local file storage adapter doesn't work on Windows environment due to HDFS issue.