2023-09-12 00:50:00
NVIDIA is the strongest company in the artificial intelligence sector thanks to everything invested in its accelerators, but a good part of its success is due to the level of software available. Now the company has announced a new version of TensorRT-LLMits large language model used for inferences and which is therefore now also aimed at generative artificial intelligences like ChatGPT.
It is open source so this system can be used on any graphics card as long as it has tensor cores. According to the company’s data, in its accelerators such as the H100, inference performance—for a neural network to discover, for example, what objects are in a photo—is doubled. Among the techniques used for optimization is the on-the-fly batch planningwhich ensures that GPU resources are always being used to their maximum.
Fuente:
NVIDIA. Via:
Tom’s Hardware.
#NVIDIA #doubles #performance #H100 #GH200 #accelerators #inference