Spotfire Ideas Portal
This idea is needed for v14+, with the new R service, which is missing a key capability, allowing us to pass in NFS Share volume info at launch time, which prevents us from containerizing it, and further homogenizing our deployment.
For many years, we have been utilizing NFS Shares with our Statistics Services, and our user base is heavily dependent upon this capability. Being able to access data and code from data functions that exists outside of the Spotfire environment is critical to our enterprise. These shares have been mounted on the server directly.
Containerization is a key goal for our operations, for all the known / published reasons. We are actively trying to containerize the entire Spotfire environment.
But, we can't.
For the new R Node and service, we would need to pass in the NFS Share endpoints at launch, since the docker container runs inside an isolated environment without access to the host system. This should be possible, as shown in the docker volume create page, but there is no such capability to pass these as parameters in to the R Node at launch time. We cannot add volumes to a container. Crazy.
This is a fundamental bedrock kind of thing that we need to do. What typical organizations need to do. We need this capability to be added to the R Node, please. Having a mixed environment will add more complexity and we would need to keep two sets of deployments and provisioning scripts to support both EC2 and Container-based deployments.
Please help us get to a more functional. stable state by allowing us to add volumes to our R Node.
unprivating comment:
Let me rephrase the story.
We have tons of R code written by our users for their own needs, stored on the NFS share, and when they work in Spotfire, they reuse part of that code by including sources files they need into the data functions, instead of copy-pasting and duplicating. That's how it worked in Statistics Services for years, and we have a lot of analyses already built that way and depending on that possibility.
Now we are trying to switch to Spotfire 14 which comes with R services.
1) We run Nodemanager on a Linux AWS EC2 virtual machine instance
2) Nodemanager spawns docker container workers for R Services to do the job.
(In other words, we do not use Cloud Development Kit, due to reasons that are out of scope for this feature request)
To maintain the same functionality as we had with Stastistics Services, we need to mount NFS shares inside these containers.
However, by design, within docker container you cannot do mounts unless you have "--cap-add SYS_ADMIN or --privileged=true" enabled, even if you are root.
There are three methods of mounting I am aware of :
1)enable SYS_ADMIN/priviledges in launch parameters, (enabling that not recommended in most cases, but whatever)
2)create docker nfs volume and specify its mounting in launch parameters
3)mount nfs share on host, and make a bind mount ( map host folder into the container), again by specifying it in launch parameters.
Unfortunately, as you can see all these methods require the same - to launch the container with custom parameters.
Meanwhile R services launches docker containers on it's own, without any possibility to customize launch parameters, making it impossible to use any way of mounting.
Therefore, I believe that implementing this parametrization is vital.
Hi JP,
My assumption is that you want to use the R service together with the Spotfire Server, and not with Statistics Services, right?
Are you running the R service with the Spotfire Cloud Deployment Kit, using the recipes for the Spotfire container images and Helm charts? In that case, our charts support the creation of volumes for multiple purposes: https://github.com/spotfiresoftware/spotfire-cloud-deployment-kit/blob/main/helm/charts/spotfire-server/README.md#volumes.
If not, can you describe in which OS and how are your using the R service (node manager installed in a VM and containerized R service? or container that includes both the node manager and the R service (as in the CDK)?
Thanks