|
Large Language Model (LLM) Concepts
|
The fundamental functionality of LLM starts with predicting
the next word for an input prompt or a sentence. Next feature
of LLM is, generating the best answer to a user prompt.
As shown earlier,
-
A user inputs a question (prompt)
to
ChatGPT.
-
The LLM framework parses the input string
and retrieves or generates
the best answer to the
question.
The GPT in ChatGPT refers to
Generative Pre-trained Transformer.
To generate very accurate answer to a user prompt,
the LLM system processes the question in several
stages. The LLM is based on
Transformer architecture,
explained later.
|
| |
|
LLM High Level Model
|
|
| |
|
User Prompt Parsing
|
|
User prompt is processed against a large volume of
textual data. Typically unsupervised learning
approach is used.
|
| |
|
Tokenizing of words
|
|
The text is broken down into smaller units
(individual words) and tokens are generated
for them.
|
| |
|
Word Vector Computation
|
|
Further the word vectors are computed for
each word. Using the vector values relationships
are derived.
|
| |
|
Use of Retrieval Augmented Generation (RAG) Framework
|
|
Using RAG framework, vectorized input is computed against
large volume of data stored in a vector database and
accurate documents are generated. The transformer will
compute the best possible answer. Use of RAG framework
minimizes hallucination
and also improves the
I/O performance
of the LLM system.
|
| |
|
|
Transformer
|
|
A transformer is a neural network. Transformer is
based on the
self-attention
mechanism, where the
machine learning model computes the attention
weight of each position (word) of an input
sequence. The self-attention mechanism also
adjusts the significance or influence of the
words in an input string as a whole than
processing individual words. This assists in
efficient prediction of the output, when
input is translated from one language to
another. The position of equivalent words will
be different in two languages for a given
input/prompt. The LLM using the transformer,
outputs the translated string with the same
context/meaning for a given prompt.
|
| |
|
Using the weights computed, the LLM
can predict and decipher the context of a
given input or prompt in non-translational
language processing.
|
| |
|
Transformer Architecture
[Source: [20] Attention Is All You Need]
|
|
| |
| |
|
The two key features of a transformer are
encoder and decoder.
|
| |
|
Encoder
|
|
An encoder parses the input string, in machine
learning terms, creates predictive modeling
such as classification. The
architecture of encoder is based on the
self-attention.
An encoder has 6 identical layers (N = 6) and each
layer has 2 sub-layers. The first sub-layer is the
multi-head self attention mechanism and the
second sub-layer is a fully connected
FFN.
The other feature is, the use of residual connection around
each of the two sub-layers along with layer normalization.
|
| |
|
Output of Sub-layer = LayerNorm(x + Sublayer(x))
|
| |
|
The Sublayer(x) is a function, implemented by the
sub-layer itself.
|
| |
|
The residual connections are implemented by the output
of all sub-layers and embedding layers. The output is
of the dimension dmodel = 512.
|
| |
|
Decoder
|
| Similar to encoder, the decoder has 6-layers (N = 6) with
two sub-layers in each layer. The decoder has a third sub-layer
that performs
Multi-Head Attention (as shown in the
Transformer Architecture
diagram above) over the output of the encoder stack. The
decoder also employs residual connections around each of
the sub-layers, along with layer normalization.
The decoder modifies self-attention sub-layer to
prevent positions from attending to subsequent
positions by masking. Further
the output embeddings are offset by one position.
This results in predictions for position i to
depend only on the known outputs at positions
less than i.
|
| |
|
Attention
|
|
It is know that in language translation, literal
translation word-by-word, is not the correct method.
To solve this, attention mechanism is used in
transformers, which avoids RNN.
To understand self attention mechanism, we need to
compute how relationship (significance or influence)
exists between words in a
sentense.
|
| |
|
Mathematically, attention can be thought of as mapping
of a query and a set of key-value pairs to an output.
The query (Q), key (K), and value (V) are all vectors.
The output is the weighted sum of values. In simple
terms,
|
| |
|
Query (Q)
can be thought of as a search string input to a search engine
|
|
Keys (K)
as a set of values such as - title, author, description
etc. relavent to the search
|
|
Values (V)
represent the possible details, websites, articles, videos etc.
|
| |
|
Using the input matrix X, and learnt matrix W (learnt
from back propogation on the transformer's loss
function), using matrix multiplication we compute the
Query, Key, and Value.
|
| |
|
Q = XWq
|
|
K = XWk
|
|
V = XWv
|
| |
|
The comparison between Query and Key is performed
by using a compatibility matrix.
The compatibility matrix derives the similarities
between the two input vectors using a
Compatibility Function. The
dot product is the compatibility
function.
|
| |
|
Compatibility(Q,K) = Q · K = QKT
|
| |
|
|
[Source: [20] Attention Is All You Need]
|
| |
| Assignment of values to QUERY, KEY and VALUE
|
| |
|
| Softmax Function
|
|
Softmax function is used to compute the probability
of input vector, as shown in the equation below. The
name is derived as a combination of soft and max
probability values. Soft represents the fact that
it assigns some probability or soft
value to low
input vector. It assigns probability values to other
input vectors and max value to the
highest input vector. The sum of all probability
values of softmax function compute to 1.0.
|
| |
|
| |
|
The table below shows the output values of Sigmoid and
Softmax function. For the sample range shown, the output
of sigmoid represents an S-curve output. The Softmax
function shows low and high probability values for
respective low and high input vectors. The the sum of
output of the Softmax function is 1.0.
|
| |
|
0.0002 + 0.0006 + 0.0016+ .... + 0.0856 + 0.2326 + 0.6322 = 1.0
|
| |
|
| |
| |
|
Positional Encoding
|
| As the transformer model does not use recurrence
(RNN) or convolution, the relative or absolute
position of tokens have to be preserved as a vector.
This is done by positional encoding (PE). The
positional
encoding algorithm encodes the input string based on
its position, preserving the sequence. The LLM, can
comprehend the contextual relationships between
words in the input. This enables syntax to be
accurate when input has to be converted from one
language to another. To differenciate between
positions in the input, sine and cosine functions
of different frequencies are used in the computation
of positional encoding.
|
| |
|
PE(pos,2i) = sin(pos/100002i/dm)
[PE1]
|
| |
|
PE(pos,2i+1) = cos(pos/100002i/dm)
[PE2]
|
| |
|
pos is the position in the input sequence
|
|
i is the index value or dimension
of the positional encoding vector where
0 ≤ 2i < dm.
Also this can be expressed as
2i/dm ≤ 1.
|
| |
|
Positional encoding enables each position in the
sequence to allocate a unique positional encoding
vector.
|
|
The formula for wavelength is
λ = 2π/f.
|
|
Positional Encoding (PE) is computed for the wavelengths
in the gemoetric sequence of 2π and 10000 . 2π
using the equation
PE1 and
PE2.
|
|
Positional Encoding Matrix computation as shown by
Alejandro Ito Aramendia
.
|
|
The positional vectors have the respective dimensions dm,
that is fed back to the encoder-decoder layer of the
LLM transformer.
|
| |
|
Encoder Decoder Animation - Jay Alammar
|
| |
|
Large Language Model
|
| |
|
Table of Content
|