Colin
|
24957aa2ae
|
Refine train.
|
2024-04-19 19:12:38 +08:00 |
|
Colin
|
2d415d9e44
|
Reduce memory cost when build dataset.
|
2024-04-19 16:57:59 +08:00 |
|
Colin
|
43883be692
|
Speedup meaning dataset build speed.
|
2024-04-19 15:08:40 +08:00 |
|
Colin
|
b062cc9c94
|
Save memory cost when meaning dataset build by np array.
|
2024-04-19 00:51:06 +08:00 |
|
Colin
|
a524d01ac3
|
Add multithread dataloder support.
|
2024-04-17 23:59:09 +08:00 |
|
Colin
|
17de117bda
|
Refine train,
|
2024-04-17 20:22:49 +08:00 |
|
Colin
|
ef08359a94
|
Refine meaning dataset memory cost when building.
|
2024-04-14 23:35:55 +08:00 |
|
Colin
|
c907210fc1
|
Update document.
|
2024-04-14 17:41:30 +08:00 |
|
Colin
|
9926b893a4
|
Fix dataset.
|
2024-04-14 01:27:58 +08:00 |
|
Colin
|
6791264987
|
Update meaning dataset function.
|
2024-04-12 20:04:04 +08:00 |
|
Colin
|
43e486aa1c
|
Add mask when validation.
|
2024-04-10 14:43:16 +08:00 |
|
Colin
|
7434427ec9
|
Update meaning dataset.
|
2024-04-10 00:34:47 +08:00 |
|
Colin
|
7560166b76
|
Dump dataset to single file.
|
2024-04-07 17:32:21 +08:00 |
|
Colin
|
9d3b9a210a
|
Speedup dataset generate.
|
2024-04-07 17:03:35 +08:00 |
|
Colin
|
33d1e22655
|
Refine meaning dataset.
|
2024-04-07 00:25:21 +08:00 |
|
Colin
|
2bc9e3b57e
|
Refine train dataset.
|
2024-04-03 17:09:30 +08:00 |
|
Colin
|
3c774983d4
|
Refine mapping print.
|
2024-04-03 13:03:59 +08:00 |
|
Colin
|
1642a91d80
|
Add meaning map print.
|
2024-04-03 11:24:00 +08:00 |
|
Colin
|
a15e55bead
|
Add mapping output.
|
2024-04-02 19:59:05 +08:00 |
|
Colin
|
7a8815cceb
|
Refine the base code.
|
2024-03-29 22:10:25 +08:00 |
|
Colin
|
618d57f23c
|
Update define.
|
2024-03-26 18:15:55 +08:00 |
|
Colin
|
33b351ff8a
|
Refine train.py.
|
2024-03-26 15:01:19 +08:00 |
|
Colin
|
b0ca4dc35d
|
Update meaning dataset define.
|
2024-03-26 11:32:27 +08:00 |
|
Colin
|
d10e7a8396
|
Refine train.py for train.
|
2024-03-25 19:53:11 +08:00 |
|
Colin
|
4c7fdbe817
|
Add GPU stress test.
|
2024-03-25 17:30:41 +08:00 |
|
Colin
|
c7391b090e
|
Delete unused files.
|
2024-03-20 23:05:05 +08:00 |
|
Colin
|
c4f7ef2813
|
Update special dateset.
|
2024-03-20 23:04:29 +08:00 |
|
Colin
|
01e5f86e94
|
Add inference.
|
2024-03-20 22:27:28 +08:00 |
|
Colin
|
b248d1d890
|
Fix model bug.
|
2024-03-20 22:23:52 +08:00 |
|
Colin
|
72718e6b72
|
Add Batch dataloader support.
|
2024-03-18 11:43:41 +08:00 |
|
Colin
|
9feaafcb7a
|
Apply meaning data train.
|
2024-03-15 11:16:42 +08:00 |
|
Colin
|
0ae63298b2
|
use custom vocab_size.
|
2024-03-14 13:28:40 +08:00 |
|
Colin
|
05f17b1221
|
Refine model config and init.
|
2024-03-14 11:40:26 +08:00 |
|
Colin
|
8330cbb036
|
Add meaning dataset.
|
2024-03-13 19:41:02 +08:00 |
|
Colin
|
c094afb0f9
|
Add tensorboard event out.
|
2024-03-09 16:55:03 +08:00 |
|
Colin
|
f1394d5974
|
Refine code.
|
2024-03-08 20:46:42 +08:00 |
|
Colin
|
601c7f6510
|
Retest wit.
|
2024-03-07 16:30:37 +08:00 |
|
Colin
|
a70d12d04d
|
Rename train file.
|
2024-03-05 22:09:58 +08:00 |
|
Colin
|
9ef3e92b23
|
Try model train.
|
2024-03-05 22:09:28 +08:00 |
|
Colin
|
11fc8f1d39
|
Refine label used.
|
2024-03-05 22:08:37 +08:00 |
|
Colin
|
cf726a5b9f
|
Add loss and logger code.
|
2024-03-05 15:54:03 +08:00 |
|
Colin
|
9e8e92ae25
|
Update trainer to custom data.
|
2024-03-04 21:41:46 +08:00 |
|
Colin
|
1622bf3054
|
add mnbvc dataset .
|
2024-03-03 23:35:40 +08:00 |
|
Colin
|
8120be66a6
|
sperate train and val dataset.
|
2024-02-26 23:59:00 +08:00 |
|
Colin
|
d1906629ab
|
Enable wit train on cutome dataset and loss down.
|
2024-02-26 22:44:26 +08:00 |
|
Colin
|
1ef3e419cb
|
Add custom dataset support.
|
2024-02-26 22:44:26 +08:00 |
|
Colin
|
e5f97af291
|
Add wit train support.
|
2024-02-26 22:44:26 +08:00 |
Colin
|
fc071dce70
|
Remove no use tiktoken.
|
2024-02-21 21:11:15 +08:00 |
|
Colin
|
fe13f12327
|
Add wit.
|
2024-02-06 14:08:45 +08:00 |
|
Colin
|
9d5d590b09
|
Add dataset and wit.
|
2024-02-04 23:48:24 +08:00 |