Some Notes on Generative AI
It's been some time since I have made notes on my "Gen AI" adventure. Here is an update in no particular order: THEORY The current wave of AI is based on "Large Language Models" (LLMs). Roughly speaking, these are models of the human language and how words (tokens) relate to each other. The LLM reflects the vocabulary of the particular AI application. Many databases of pictures are "tagged" or described in words the AI "understands" because the words are in the LLM. To this, add the magic of graphics cards, and you will have the ability of AI to find pictures that "look like" any given picture. Given these pictures, you have the associated words or tokens, which can then be matched to the "prompt." Ultimately, you have a two-way conversation about pictures mediated by the AI's vocabulary (the LLM). The key idea is that the words AI uses to describe a picture are automatically part of its LLM, so they can be used to cr...