colin
/
Witllm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

更新 Readme.md

This commit is contained in:
colin 2023-12-31 15:26:29 +08:00
parent d9b64e4025
commit 1579e0b8f2
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
Readme.md
View File

@ -144,7 +144,7 @@ return Linear(context_layer) -> [6, 1, 4096]
``` ```
## GLMBlock ## GLMBlock
```
input input
| \ | \
| RMSNorm | RMSNorm
@ -158,5 +158,5 @@ return Linear(context_layer) -> [6, 1, 4096]
| dropout | dropout
| / | /
Add Add
```
所有的输出shape都是[6, 1, 4096], 6:sequence_length 1:batch_num 4096:hidden_size 所有的输出shape都是[6, 1, 4096], 6:sequence_length 1:batch_num 4096:hidden_size