更新 Readme.md

2023-12-31 17:42:21 +08:00 · 2023-12-31 17:42:21 +08:00 · bde8f71a7f
parent 1579e0b8f2
commit bde8f71a7f
1 changed files with 3 additions and 2 deletions
--- a/Readme.md
+++ b/Readme.md
@ -107,8 +107,9 @@ Linear(intermediate_parallel)  no bias  ->  [6, 1, 4096]
                  |   expand  expand   -> [6, 1, 32, 128]
                   \   /       |
         ┏----      dot        |
-         ┃        softmax     /
-attention┃             \     /
+         ┃  += attention_mask  /
+attention┃        softmax     /
+         ┃             \     /
         ┗----           dot           ->  [1, 32, 6, 128]  ->  [6, 1, 4096]
                        Linear         ->  [6, 1, 4096]