You can set "label rendering" on the category axis as text or as an image from URL. Seems that svg file extension should be supported the same way as gif, png, jpg.
It is also not clear on the documentation what type of file extensions are supported:
Allows you to select how the data in the selected column should be interpreted.
For example, use Text for plain text labels. Use Image if there are binary images in your data that you want to use as labels. Use Image from URL if the image is picked from a web resource location. You may need to change the Image from URL Renderer Settings by clicking on the Settings... button and modifying the link syntax so that it fits your current data.
Use Geometry if your data is a shapefile or GeoJSON file and you want to show the geometrical shapes in the labels.
If other types of columns, for example, structure columns, are used to label by, then you would select a renderer for displaying structure images here.