Colin b062cc9c94 Save memory cost when meaning dataset build by np array.		2024-04-19 00:51:06 +08:00
..
doc	Update document.	2024-04-14 17:41:30 +08:00
MNBVC.py	add mnbvc dataset .	2024-03-03 23:35:40 +08:00
__init__.py	Add dataset and wit.	2024-02-04 23:48:24 +08:00
configuration.py	Refine model config and init.	2024-03-14 11:40:26 +08:00
demo.py	Refine model config and init.	2024-03-14 11:40:26 +08:00
inference.py	Add inference.	2024-03-20 22:27:28 +08:00
lit_module.py	Update meaning dataset function.	2024-04-12 20:04:04 +08:00
logger.py	Add loss and logger code.	2024-03-05 15:54:03 +08:00
meaning_dataset.py	Save memory cost when meaning dataset build by np array.	2024-04-19 00:51:06 +08:00
model.safetensors.index.json	Add dataset and wit.	2024-02-04 23:48:24 +08:00
modeling_wit.py	Fix model bug.	2024-03-20 22:23:52 +08:00
qwen_generation_utils.py	Add dataset and wit.	2024-02-04 23:48:24 +08:00
special_dataset.py	Update special dateset.	2024-03-20 23:04:29 +08:00
stress.py	Add GPU stress test.	2024-03-25 17:30:41 +08:00
tokenization_qwen.py	Add wit train support.	2024-02-26 22:44:26 +08:00
tokenizer_config.json	Add dataset and wit.	2024-02-04 23:48:24 +08:00
train.py	Refine train,	2024-04-17 20:22:49 +08:00
train_special.py	Add Batch dataloader support.	2024-03-18 11:43:41 +08:00
wit_b64.tiktoken	Add wit.	2024-02-06 14:08:45 +08:00
wit_char.tiktoken	Add wit.	2024-02-06 14:08:45 +08:00