Commit Graph

16 Commits

Author SHA1 Message Date
Colin e048d0c531 Refine query. 2025-08-26 13:31:03 +08:00
Colin bd5379f24e Fix meaning dataset get NodeTree with stride. 2025-08-21 22:23:28 +08:00
Colin e18ee0c781 Update meaning dataset stride token. 2025-08-21 18:11:58 +08:00
Colin d48a5563aa Fix get_nodetree. 2025-08-18 13:03:32 +08:00
Colin 085bd92fb9 Refine get_nodetree to support stride and tree tokens. 2025-08-18 11:17:35 +08:00
Colin ee30eb4aab Remove weight and height in the meaning dataset. 2025-08-17 14:43:13 +08:00
Colin b2fe00c157 Add reserve_vocaba support, delete InitValDataset. 2025-08-14 14:51:43 +08:00
Colin a983a2b6e6 Fix rank index and all calculater when tree mode enable. 2025-08-14 13:25:47 +08:00
Colin 59e079f5e7 Update stride test. 2025-08-13 16:37:24 +08:00
Colin ce81f26845 Fix dataset cache check with vocab size. 2025-08-12 11:08:16 +08:00
Colin 2a09b9d9b1 Update meaning of stride mask label. 2025-08-11 16:07:32 +08:00
Colin 3f0eedfef8 Refine meaning dataset cache files. 2025-08-10 22:16:10 +08:00
Colin 133187d7bd Refine vocab_size of meaning dataset. 2025-08-10 21:01:26 +08:00
Colin b56bbb2952 Refine import code. 2025-08-10 15:10:20 +08:00
Colin b62444a9dc Refine meaning dataset import. 2025-08-10 14:30:51 +08:00
Colin 71ab0bb57d Add config of stride with_tree end, Rename dataset to meaning. 2025-08-10 13:17:04 +08:00