File tree Expand file tree Collapse file tree 1 file changed +6
-1
lines changed Expand file tree Collapse file tree 1 file changed +6
-1
lines changed Original file line number Diff line number Diff line change @@ -59,7 +59,7 @@ https://github.com/keras-team/keras/blob/master/requirements-torch-cuda.txt
5959- <strong >GAU-ALPHA</strong >: https://github.com/ZhuiyiTechnology/GAU-alpha
6060
6161
62- ### bert4keras3的新增加的模型权重,不再使用ckpt存储。通过model.load_weights方法读取权重,建议使用keras3加载 :
62+ ### bert4keras3的新增加的模型权重,不再使用ckpt存储。通过build_transformer_model( keras_weights_path='xx.weights.h5')方法读取权重,只能使用keras3加载 :
6363- <strong >自行转换的ChatYuan Keras3版本</strong >:链接:https://pan.baidu.com/s/1huwRA2sOIYL8e7OFg0QTcg?pwd=6u59 提取码:6u59
6464
6565<strong >注意事项</strong >
@@ -75,3 +75,8 @@ https://github.com/keras-team/keras/blob/master/requirements-torch-cuda.txt
7575[ roformer/roformer-v2] ( https://github.com/pass-lin/bert4keras3/blob/main/examples/test_simroformer.py ) 的cache支持,用法和t5一样,example里只是测试一下与greedy是否一致
7676
7777
78+ ## 3.17号更新,发布1.2版本
79+ 增加了对weights.h5的读取支持
80+ 增加了lora支持,可以通过设置os.environ[ "ENABLE_LORA"] ='1' 启动lora训练,注意的是除了lora之外的参数全部会被冻结
81+ 增加了flash-attention支持,可以通过设置os.environ[ "FLASH_ATTN"] ='1'使用flash-attention
82+ 但是需要注意的是,tensorflow不支持。而jax在https://github.com/nshepperd/flash_attn_jax/releases下载,torch则是https://github.com/Dao-AILab/flash-attention
You can’t perform that action at this time.
0 commit comments