Administration functions in REST API

Background:

As soon as we use Configuration Management tools to manage our multiple Spotfire environments constellation, we need to be able to perform whole Spotfire configuration via automation scripting.

Issue:

During our development we spend a lot of time trying to find out workarounds to automate different actions in Spotfire, that should be simple, but are actually not.. Each time we find that some function is located only in analyst client, or only web console, or only config tool, or not available at all.

Here are use cases that should have more straight way to be automated:

• Setting Spotfire Statistics URL - from time to time we refresh a lot of our environments (DEV/TEST/EXPERIMENTAL) from our PROD database, and need to fix Statistics URL according to target environment URLs. By design, it is possible only using manual clicking in Analyst. We had to implement direct SQL requests surgery on the database in our refresh database pipeline to do such a simple task as changing Stats server URL.

• Verify data source. - Each time when we install hotfixes or add new environment, or there are some firewall/network changes - we need to check if all data sources (literally hundreds of them) are not broken. Only way to check them is a verify button in Analyst client, but it is impossible to test hundreds of data sources that way. Had to use undocumented internal API at '/spotfire/ws/ElementManagerService' to imitate clicking on that button and parse SOAP responses. This is not a public API and we cannot fully rely on it, as it can change with the next Spotfire upgrade

• Sometimes, we need to ensure that service instances are started, and not stuck in failed states due to password rotation. However, there is no way to start service instances, other than manual clicking in web interface. Had to use undocumented internal API used by web interface at spotfire/nodemanager/start/service/ to get service instances launch automated.

• We need to monitor if Scheduled updates are healthy, gather it's state to monitoring system and generate timely alerts. This also by design is only available in web interface, and we had to use internal API to hack into it via spotfire/rest/routingRules/count to get required metrics. This API is not public and is not guaranteed to stay functional in the same way after Spotfire upgrades.

• Same applies for Nodes and service instances health - while we can use http healthchecks for Spotfire servers, - other services like Webplayers, Automation services, Python nodes - do not have any health checks, as they are hidden behind the Spotfire server. There is no way to retrieve their state and generate alerts other then internal API or actual users complaining on the service being unavailable.

• There is also a stack of library manipulation actions that we have to perform by a direct SQL database surgery like:

  • Creating/Renaming/Deleting folders

  • Setting/Retrieving permissions and other library items metadata

  • Moving/Copying library content

For example, such a simple task like creating a new user folder for an each new registered user - cannot be automated without writing Spotfire Library database directly

Proposed solution:

Things mentioned above should be implemented as a public and documented REST api available to engineers who support Spotfire, otherwise we have to build our automation scripting using unreliable methods, with a risk of internal APIs being changed, broken or decommissioned without any notice, or have to make risky direct changes into the database As far as we know, there is already a public REST API implemented in Spotfire (with almost no interesting functions yet, however), and it would be quite logical to provide such functions in the separate space inside this REST API.

P.S. REST API should have additional auth mechanisms. Using kerberos authentication from Python client to access api requires a lot of system dependencies, including gcc and building. It should support basic auth additionally instead.