更新 Readme.md

This commit is contained in:
colin 2023-12-31 15:26:29 +08:00
parent d9b64e4025
commit 1579e0b8f2
1 changed files with 2 additions and 2 deletions

View File

@ -144,7 +144,7 @@ return Linear(context_layer) -> [6, 1, 4096]
```
## GLMBlock
```
input
| \
| RMSNorm
@ -158,5 +158,5 @@ return Linear(context_layer) -> [6, 1, 4096]
| dropout
| /
Add
```
所有的输出shape都是[6, 1, 4096], 6:sequence_length 1:batch_num 4096:hidden_size