NVIDIA
NVIDIA-SMI: Great starting point for monitoring GPU...
With NVIDIA drivers installs there is the nvidia-smi tool. See the following link for info: https://developer.nvidia.com/nvidia-system-management-interface See this link for the man page and various switches/tools to use: http://developer.download.nvidia.com/compute/cuda/5_5/rel/nvml/nvidia-smi.5.319.43.pdf For GRID specifically, keep in mind the location of the driver. As an example, if using vSGA (shared) in VMware then the driver is the VIB loaded on the ESXi hypervisor, so the nvidia-smi executable is located there, on the hypervisor. If on that same deployment you chose to use on GPU as pass-through then that GPU is looking to the NVIDIA driver on the guest and so nvidia-smi is located on the guest.
With NVIDIA drivers installs there is the nvidia-smi tool. See the following link for info:

https://developer.nvidia.com/nvidia-system-management-interface


See this link for the man page and various switches/tools to use:

http://developer.download.nvidia.com/compute/cuda/5_5/rel/nvml/nvidia-smi.5.319.43.pdf


For GRID specifically, keep in mind the location of the driver. As an example, if using vSGA (shared) in VMware then the driver is the VIB loaded on the ESXi hypervisor, so the nvidia-smi executable is located there, on the hypervisor. If on that same deployment you chose to use on GPU as pass-through then that GPU is looking to the NVIDIA driver on the guest and so nvidia-smi is located on the guest.

Regards,

Luke Wignall
Performance Engineering Manager
NVIDIA | Worldwide Sales ­ GRID Computing
http://www.linkedin.com/in/lukewignall/
https://twitter.com/lwignall

#1
Posted 03/28/2014 05:03 PM   
For vGPU do make sure you run "nvidia-smi -l" without the loop flag you can see stalls as it reenters the API - I added some other tools here: http://www.xenserver.org/partners/developing-products-for-xenserver/20-dev-hints/133-xs-dev-tools.html and GPU specific ones: http://www.xenserver.org/partners/developing-products-for-xenserver/18-sdk-development/136-xs-dev-gpu-tools.html For pass-through you will need to use something in guest e.g. Fraps
For vGPU do make sure you run "nvidia-smi -l" without the loop flag you can see stalls as it reenters the API - I added some other tools here: http://www.xenserver.org/partners/developing-products-for-xenserver/20-dev-hints/133-xs-dev-tools.html


and GPU specific ones: http://www.xenserver.org/partners/developing-products-for-xenserver/18-sdk-development/136-xs-dev-gpu-tools.html


For pass-through you will need to use something in guest e.g. Fraps

#2
Posted 04/03/2014 09:41 AM   
There was a bug in the tech preview / beta release of vGPU where running nvidia-smi would cause a visible glitch or stall in VMs running vGPU - but this was fixed in the vGPU 1.0 production release. You shouldn't see any stall when running nvidia-smi now.
There was a bug in the tech preview / beta release of vGPU where running nvidia-smi would cause a visible glitch or stall in VMs running vGPU - but this was fixed in the vGPU 1.0 production release. You shouldn't see any stall when running nvidia-smi now.

#3
Posted 04/04/2014 05:28 PM   
Scroll To Top

Add Reply