Foundation models

Large, pre-trained language models that serve as a basis or foundation for a wide range of natural language processing (NLP), computer vision tasks, etc., are termed ‘Foundation models’. Serving as a 'foundational' building block for various generative AI applications, they generate output from one or more inputs (prompts) in the form of human language instructions. These models are trained on a vast data set and can apply the information learnt about one situation to another using transfer learning. For example, in text generation, a model can predict the next word or sentence based on the words and sentences provided in the input. Foundation models can be used for different tasks, with minimal fine-tuning and be adapted to a wide range of downstream tasks.  

Foundation models serve as the groundwork for developing more sophisticated and customized models, incorporating features tailored to specific domains or use cases. Foundational models employ self-supervised learning techniques, creating labels from input data as part of their training process.