Foundation models are AI neural networks trained on vast amounts of raw data, typically through unsupervised machine learning. They are a type of AI model trained to understand and generate human-like language. Now, we can provide libraries that allow computers to read and learn from vast amounts of books, enabling them to understand the context and meaning of words and sentences like humans.

NVIDIA, a leader in AI computing technology, announced on the 11th that Foundation models can be run locally on PCs and workstations equipped with NVIDIA GeForce and NVIDIA RTX GPUs.

The Foundation model has a deep knowledge base and natural language communication capabilities. It can be useful for a wide range of applications, including text generation and summarization, copilot production and computer code analysis, image and video production, audio conversion and speech synthesis.

■ AI model local execution and free access to the API catalog
One of the most notable generative AI applications is ChatGPT, a chatbot built on OpenAI's GPT Foundation model. Currently in its fourth version, GPT-4, it is a large-scale multimodal model capable of ingesting text or images and generating text or images.

Online apps built on the Foundation model typically access that model from a data center. However, many of these models and the applications they power can now run locally on PCs and workstations equipped with NVIDIA GeForce and NVIDIA RTX GPUs, he explained.

The Foundation Model can perform various functions such as △language processing - text understanding and generation △code generation - computer code analysis and debugging in various programming languages △visual processing - image analysis and generation △speech - text-to-speech generation and speech-to-text conversion.

These features can be used as-is or refined for further refinement. Training a completely new AI model for each generative AI is expensive and time-consuming. Therefore, users typically fine-tune the foundation model to suit their specific use cases.

Pre-trained foundation models deliver impressive performance thanks to prompts and data retrieval techniques like retrieval-augmented generation (RAG). Furthermore, foundation models excel at transfer learning, allowing them to be trained to perform tasks other than their original purpose.

For example, general-purpose large language models (LLMs) designed to converse with people can be further trained to act as customer service chatbots that can answer questions using a company's knowledge base, and foundation models are being fine-tuned across a variety of industries.

There are currently over 100 foundation models in use, and the number continues to grow. LLM and image generators are the two most popular types of foundation models.

Many of these are available for free use in the NVIDIA API Catalog, highlighting the availability of various foundation models for free use on all hardware.

■ Foundation Model Type

Mistral AI's Mistral LLM can follow commands, complete requests, and generate creative text.

Meta's Llama 2 is a cutting-edge LLM that generates text and code in response to prompts. Mistral and Llama 2 are featured in the NVIDIA ChatRTX technology demo, running on RTX PCs and workstations. ChatRTX allows users to personalize these foundational models by connecting them to personal content, such as documents, doctor's notes, and other data, through RAG.

This is accelerated by TensorRT-LLM, enabling rapid, context-sensitive answers. Furthermore, because it runs locally, results are obtained quickly and safely.

Stability AI's image generators, such as Stable Diffusion XL and SDXL Turbo, enable you to create stunning, realistic visuals. Stability AI's video generator, Stable Video Diffusion, uses a generative diffusion model to synthesize video footage using a single image as a conditioning frame.

Multimodal foundation models can process two or more data types, such as text and images, simultaneously to produce more sophisticated results.

Multimodal models support both text and images, allowing users to upload images and ask questions about them. This type of model is quickly being adopted in real-world applications such as customer service, replacing existing manuals with faster, more user-friendly versions.

Kosmos 2 is a groundbreaking multimodal model from Microsoft designed to understand and reason about the visual elements of images.

Meanwhile, GeForce RTX and NVIDIA RTX GPUs support running Foundation models locally. Results can be obtained quickly and securely, and users can leverage apps like ChatRTX to process sensitive data locally without having to share data with third parties or connect to the internet, instead of relying on cloud-based services.

You can choose from a rapidly growing catalog of open foundation models to download and run on your own hardware. NVIDIA expects this to reduce costs and eliminate latency and network connectivity issues compared to using cloud-based apps and APIs.