VI. GPT Architecture
The conections between lins and token dots are calculated based on probability values of generated tokens.
Solid line: Logical
Dash line: Sentimental
i. Architecture in unfinetuned & pretrained GPT
Temperature = 0.1
Unfinetuned GPT
More Logical, Less Creative
Unfinetuned GPT
More Logical, Less Creative
The connection speed is low and concentrated, which indicates logical LM responds based more on trainng datasets than its own thoughts.
Temperature = 3.0
Unfinetuned GPT
Unfinetuned GPT
More Creative, Less Logical
The connection speed is active and spread-out, which indicates imaginative LM responds based more on its own thoughts than
trainng datasets.
ChatGPT
General
Tokens barely have coonections and hard to concentrate, which indicates that ChatGPT are trained on massive Internet data and can’t respond with persoal or emotional thoughts.
