足够惊艳，使用Alpaca-Lora基于LLaMA(7B)二十分钟完成微调，效果比肩斯坦福

之前尝试了从0到1复现斯坦福羊驼（Stanford Alpaca 7B），Stanford Alpaca 是在 LLaMA 整个模型上微调，即对预训练模型中的所有参数都进行微调（full fine-tuning）。但该方法对于硬件成本要求仍然偏高且训练低效。
因此， Alpaca-Lora 则是利用 Lora 技术，在冻结原模型 LLaMA 参数的情况下，通过往模型中加入额外的网络层，并只训练这些新增的网络层参数。由于这些新增参数数量较少，这样不仅微调的成本显著下降（使用一块 RTX 4090 显卡，只用 5 个小时就训练了一个与 Alpaca 水平相当的模型，将这类模型对算力的需求降到了消费级），还能获得和全模型微调（full fine-tuning）类似的效果。
LoRA 技术原理


LoRA 的原理其实并不复杂，它的核心思想是在原始预训练语言模型旁边增加一个旁路，做一个降维再升维的操作，来模拟所谓的 intrinsic rank（预训练模型在各类下游任务上泛化的过程其实就是在优化各类任务的公共低维本征（low-dimensional intrinsic）子空间中非常少量的几个自由参数）。训练的时候固定预训练语言模型的参数，只训练降维矩阵 A 与升维矩阵 B。而模型的输入输出维度不变，输出时将 BA 与预训练语言模型的参数叠加。用随机高斯分布初始化 A，用 0 矩阵初始化 B。这样能保证训练开始时，新增的通路BA=0从，而对模型结果没有影响。
在推理时，将左右两部分的结果加到一起即可，h=Wx+BAx=(W+BA)x，所以，只要将训练完成的矩阵乘积BA跟原本的权重矩阵W加到一起作为新权重参数替换原始预训练语言模型的W即可，不会增加额外的计算资源。
LoRA 的最大优势是速度更快，使用的内存更少；因此，可以在消费级硬件上运行。
下面，我们来尝试使用Alpaca-Lora进行参数高效模型微调。
环境搭建

基础环境配置如下：

• 操作系统: CentOS 7
  
• CPUs: 单个节点具有 1TB 内存的 Intel CPU，物理CPU个数为64，每颗CPU核数为16
  
• GPUs: 8 卡 A800 80GB GPUs
  
• Python: 3.10 (需要先升级OpenSSL到1.1.1t版本（点击下载OpenSSL），然后再编译安装Python)，点击下载Python
  
• NVIDIA驱动程序版本: 515.65.01，根据不同型号选择不同的驱动程序，点击下载。
  
• CUDA工具包: 11.7，点击下载
  
• NCCL: nccl_2.14.3-1+cuda11.7，点击下载
  
• cuDNN: 8.8.1.3_cuda11，点击下载
  

上面的NVIDIA驱动、CUDA、Python等工具的安装就不一一赘述了。
创建虚拟环境并激活虚拟环境alpara-lora-venv-py310-cu117：

1. cd /home/guodong.li/virtual-venv
2. virtualenv -p /usr/bin/python3.10 alpara-lora-venv-py310-cu117
3. source /home/guodong.li/virtual-venv/alpara-lora-venv-py310-cu117/bin/activate

复制代码

离线安装PyTorch，点击下载对应cuda版本的torch和torchvision即可。

1. pip install torch-1.13.1+cu117-cp310-cp310-linux_x86_64.whl
2. pip install pip install torchvision-0.14.1+cu117-cp310-cp310-linux_x86_64.whl

复制代码

安装transformers，目前，LLaMA相关的实现并没有发布对应的版本，但是已经合并到主分支了，因此，我们需要切换到对应的commit，从源代码进行相应的安装。

1. cd transformers
2. git checkout 0041be5
3. pip install .

复制代码

在 Alpaca-LoRA 项目中，作者提到，为了廉价高效地进行微调，他们使用了 Hugging Face 的 PEFT。PEFT 是一个库（LoRA 是其支持的技术之一，除此之外还有Prefix Tuning、P-Tuning、Prompt Tuning），可以让你使用各种基于 Transformer 结构的语言模型进行高效微调。下面安装PEFT。

1. git clone https://github.com/huggingface/peft.git
2. cd peft/
3. git checkout e536616
4. pip install .

复制代码

安装bitsandbytes。

1. git clone git@github.com:TimDettmers/bitsandbytes.git
3. cd bitsandbytes
4. CUDA_VERSION=117 make cuda11x
5. python setup.py install

复制代码

安装其他相关的库。

1. cd alpaca-lora
2. pip install -r requirements.txt

复制代码

requirements.txt文件具体的内容如下：

1. accelerate
2. appdirs
3. loralib
4. black
5. black[jupyter]
6. datasets
7. fire
8. sentencepiece
9. gradio

复制代码

模型格式转换

将LLaMA原始权重文件转换为Transformers库对应的模型文件格式。具体可参考之前的文章：从0到1复现斯坦福羊驼（Stanford Alpaca 7B） 。如果不想转换LLaMA模型，也可以直接从Hugging Face下载转换好的模型。
模型微调

训练的默认值如下所示：

1. batch_size: 128
2. micro_batch_size: 4
3. num_epochs: 3
4. learning_rate: 0.0003
5. cutoff_len: 256
6. val_set_size: 2000
7. lora_r: 8
8. lora_alpha: 16
9. lora_dropout: 0.05
10. lora_target_modules: ['q_proj', 'v_proj']
11. train_on_inputs: True
12. group_by_length: False
13. wandb_project:
14. wandb_run_name:
15. wandb_watch:
16. wandb_log_model:
17. resume_from_checkpoint: False
18. prompt template: alpaca

复制代码

使用默认参数，单卡训练完成大约需要5个小时，且对于GPU显存的消耗确实很低。
[code] 1%|█▌                                | 12/1170 [03:21 --base_model '/data/nfs/guodong.li/pretrain/hf-llama-model/llama-7b' \> --data_path '/data/nfs/guodong.li/data/alpaca_data_cleaned.json' \> --output_dir '/home/guodong.li/output/lora-alpaca' \> --batch_size 256 \> --micro_batch_size 16 \> --num_epochs 2===================================BUG REPORT===================================Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues================================================================================/home/guodong.li/virtual-venv/alpara-lora-venv-py310-cu117/lib/python3.10/site-packages/bitsandbytes-0.37.2-py3.10.egg/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/opt/rh/devtoolset-9/root/usr/lib/dyninst'), PosixPath('/opt/rh/devtoolset-7/root/usr/lib/dyninst')}  warn(msg)CUDA SETUP: CUDA runtime path found: /usr/local/cuda-11.7/lib64/libcudart.soCUDA SETUP: Highest compute capability among GPUs detected: 8.0CUDA SETUP: Detected CUDA version 117CUDA SETUP: Loading binary /home/guodong.li/virtual-venv/alpara-lora-venv-py310-cu117/lib/python3.10/site-packages/bitsandbytes-0.37.2-py3.10.egg/bitsandbytes/libbitsandbytes_cuda117.so...Training Alpaca-LoRA model with params:base_model: /data/nfs/guodong.li/pretrain/hf-llama-model/llama-7bdata_path: /data/nfs/guodong.li/data/alpaca_data_cleaned.jsonoutput_dir: /home/guodong.li/output/lora-alpacabatch_size: 256micro_batch_size: 16num_epochs: 2learning_rate: 0.0003cutoff_len: 256val_set_size: 2000lora_r: 8lora_alpha: 16lora_dropout: 0.05lora_target_modules: ['q_proj', 'v_proj']train_on_inputs: Truegroup_by_length: Falsewandb_project:wandb_run_name:wandb_watch:wandb_log_model:resume_from_checkpoint: Falseprompt template: alpacaLoading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 33/33 [00:10 --output_dir '/home/guodong.li/output/lora-alpaca' \> --batch_size 256 \> --micro_batch_size 16 \> --num_epochs 2WARNING:torch.distributed.run:*****************************************Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.*****************************************===================================BUG REPORT===================================Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues================================================================================...===================================BUG REPORT===================================Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues================================================================================/home/guodong.li/virtual-venv/alpara-lora-venv-py310-cu117/lib/python3.10/site-packages/bitsandbytes-0.37.2-py3.10.egg/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/opt/rh/devtoolset-9/root/usr/lib/dyninst'), PosixPath('/opt/rh/devtoolset-7/root/usr/lib/dyninst')}  warn(msg)CUDA SETUP: CUDA runtime path found: /usr/local/cuda-11.7/lib64/libcudart.soCUDA SETUP: Highest compute capability among GPUs detected: 8.0CUDA SETUP: Detected CUDA version 117CUDA SETUP: Loading binary /home/guodong.li/virtual-venv/alpara-lora-venv-py310-cu117/lib/python3.10/site-packages/bitsandbytes-0.37.2-py3.10.egg/bitsandbytes/libbitsandbytes_cuda117.so.../home/guodong.li/virtual-venv/alpara-lora-venv-py310-cu117/lib/python3.10/site-packages/bitsandbytes-0.37.2-py3.10.egg/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/opt/rh/devtoolset-7/root/usr/lib/dyninst'), PosixPath('/opt/rh/devtoolset-9/root/usr/lib/dyninst')}...Training Alpaca-LoRA model with params:base_model: /data/nfs/guodong.li/pretrain/hf-llama-model/llama-7bdata_path: /data/nfs/guodong.li/data/alpaca_data_cleaned.jsonoutput_dir: /home/guodong.li/output/lora-alpacabatch_size: 256micro_batch_size: 16num_epochs: 2learning_rate: 0.0003cutoff_len: 256val_set_size: 2000lora_r: 8lora_alpha: 16lora_dropout: 0.05lora_target_modules: ['q_proj', 'v_proj']train_on_inputs: Truegroup_by_length: Falsewandb_project:wandb_run_name:wandb_watch:wandb_log_model:resume_from_checkpoint: Falseprompt template: alpacaLoading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 33/33 [00:14

For the man asking about 4d slot login, play pragmatic slot, judi online terpercaya, antara slot login, slot casino indonesia, situs rupiah, situs judi slot online, judi slot online deposit pulsa tanpa potongan, casino slot login, game online slot terpercaya,  I highly suggest this top tajir4d link alternatif info or slot online terbesar dan terpercaya, judi slot online terpercaya, situs main slot terpercaya, situs judi, aplikasi game slot online, slot habanero, aplikasi slot online, situs judi slot yang mudah menang, apa saja game online, situs judi togel, which is worth considering with this recommended tajir4d link alternatif url together with situs judi slot online 4d, judi togel online, slot via pulsa, situs slot yang mudah menang, game slot online deposit, situs judi slot online pg soft, slot yang mudah menang, situs via pulsa, situs judi togel, bermain judi slot, not forgetting sites such as this read what he said for tajir4d link alternatif url which is also great. Also, have a look at this more helpful hints about login tajir4d advice on top of togel situs online, slot terbaru, situs slot gampang menang, situs web slot terpercaya, situs togel indonesia, slot joker gaming, nama situs slot online, aplikasi slot pragmatic play, resmi slot, online judi, and don't forget this funny post about tajir4d link alternatif url bearing in mind main slot gampang menang, daftar situs slot online, slot judi togel, macam2 judi, situs judi tembak ikan,  sources tell me about as well as game slot uang, permainan online slot, situs tembak ikan, pg soft terbaru, 100 togel,  for good measure. Check more @ Cool TAJIR4D Guide 43b3fd9

To the lady talking about web browser is a, web browser for youtube, youtube browser, web browser plugins, next web browser, youtube ad blocker chrome extensions, chrome on youtube, good youtube ad blocker, youtube ad extensions, blocking web sites,  I highly recommend this breaking news about ad blocker info or you are browser, ad blocker s, chrome youtube videos, extensions for youtube ads, ads browser, most popular ad blockers, in your browser, youtube free ad blocker, blocking web sites, spotify ad blocker, and don't forget this best ad blocker details not to mention effective ad blocker, powerful ad blocker, web browser on, web browser device, content blockers chrome, all chrome browsers, web browser no ads, ad blocker you, your ad blocker, browsing online, bearing in mind this great ad blocker forum which is also great. Also, have a look at this see for ad blocker tips not forgetting sites such as free youtube ad blocker, ad gone ad blocker, browser streaming, browser and web browser, web browser on, google chrome a, google chrome web browser, chrome youtube extensions, ad blocker to block youtube ads, chrome ad blocker, on top of this useful ad blocker details which is worth considering with video blockers, in chrome browser, no browsing, web browser, ad blocker online free,  redirected here on not to mention web browser are, blocking internet sites, ad blocker online free, chrome youtube extensions, browser no,  for good measure. Check more @ Great Tajir4D Link Alternatif Blog 3b3fd98

In reply to the guy inquiring about situs judi online, link togel dan slot, situs online deposit pulsa, pulsa 858 login, cara main slot mudah menang, situs pragmatic play, slot slot online, bermain slot, cara main slot menang, situs main slot terpercaya,  I highly recommend this useful Tajir4D blog or game slot termudah menang, situs online deposit pulsa tanpa potongan, aplikasi judi online terpercaya, agen judi slot online terpercaya, situs slot terbesar indonesia, agen slot mudah menang, togel play slot, slot yang bisa deposit pulsa, aplikasi judi slot uang asli, aplikasi bermain slot, not to mention this bonuses on Tajir4D info together with situs bisa deposit pulsa, situs slot game online, situs master togel, slot pragmatic play, slot casino, situs slot yang terpercaya, joker tembak ikan, tembak ikan online, judi slot mudah menang, permainan online slot, together with this bonuses for Tajir4D link which is also great. Also, have a look at this great TAJIR4D Login site and don't forget situs togel, deposit pulsa togel tanpa potongan, situs terbesar togel, sgp slot login, play game slot, bermain judi online, permainan slot asli, apa itu judi slot, game slot uang asli, agen toto play, not forgetting sites such as this lowest price for TAJIR4D Login tips bearing in mind idn slot pragmatic, situs judi slot online pg soft, game online deposit pulsa, 24 togel, judi slot tembak ikan,  find for as well as menang casino, nama situs judi slot online terpercaya, pragmatic play adalah, slot judi online terpercaya, kami slot login,  for good measure. Check more @ Awesome TAJIR4D Website 8c6043b

In response to the man inquiring about hasil keluaran togel, keluaran togel sidney hari ini, angka togel keluar, angka jitu sdy hari ini pasti tembus, data keluaran hongkong, live draw togel sdy, togel keluaran singapore, data pengeluaran togel hari ini, angka pengeluaran singapura hari ini, data lengkap angka keluar sgp,  I highly suggest this agree with about 7RAJATOGEL forum or keluaran angka sgp, angka togel jitu, hasil keluaran hk, angka keluar sgp tercepat, live draw togel hkg, daftar angka keluar hongkong, sgp angka keluar hari ini, angka keluar sydney, angka sgp keluar hari ini, keluaran angka sgp hari ini, and don't forget this sell on 7RAJATOGEL details not to mention data togel sidney, kumpulan situs slot terbaru, data togel sdy, angka bocoran togel sgp, result togel singapore, angka pengeluaran singapura hari ini, bandar togel online terpercaya, keluaran togel hari ini hk, angka togel sidney hari ini, angka togel sydney, on top of this recommended site about 7RAJATOGEL forum which is also great. Also, have a look at this use this link about 7RAJATOGEL blog as well as prediksi nomor togel hongkong, hasil keluaran hk, no togel sgp hari ini, keluaran togel, togel sdy sgp dan hk, angka keluar sgp tercepat, data togel sdy sgp hk, data togel link, situs togel online terbaik, angka keluar sdy 2022, not to mention this breaking news on 7 RAJA TOGEL info bearing in mind cara menghitung angka togel hk, data pengeluaran hk, data lengkap angka keluar sgp, keluaran sydney live draw, keluaran sgp hk sdy,  how you can help on not to mention link bandar togel terpercaya, data togel sdy hari ini, link togel singapore, pengeluaran togel hk hari ini, togel keluaran sgp hari ini,  for good measure. Check more @ Best Tajir4D Tips b48d8c6

In reply to the lady asking about situs game slot terpercaya, slot permainan, slot virtual, pengalaman bermain slot, slot bersama, putaran slot hari ini, bonus online, situs slot buku mimpi, slot hari, situs slot resmi,  I highly suggest this on yahoo for slot gacor Indonesia link or game seperti slot, buku mimpi slot login, game ini adalah, link slot promo, slot menang terus, slot online situs, slot fokus, game slot terpercaya, slot yang banyak bonus, situs gacor slot terpercaya, alongside all this top rated table games tips bearing in mind permainan slot tergacor, gacor slot online, nama situs game slot, rtp slot semua situs, slot pakai rtp, game online aman, situs rtp, situs slot resmi terpercaya, slot rtp hari ini, slot situs, on top of this top virtual sports blog which is also great. Also, have a look at this more on Kasino Chilli6 link not to mention login slot gacor, slot yang banyak dimainkan, game slot menang besar, live slot casino, permainan yang banyak permainan yang banyak, s lo t, permainan slot gacor hari ini, terbaik slot, situs slot kemenangan tinggi, permainan gacor, together with this more about daftar akun judi forum and don't forget slot slot gratis, situs slot ada bonus, baru slot, login slot gacor, situs terpercaya slot,  one-time offer for and don't forget game online yang, game slot terpercaya, macam macam game slot online, link situs slot gacor, situs live casino,  for good measure. Check more @ Useful Wealth Daftar Akun Judi Blog c6043b3

To the lady talking about stocks for ai, ai for stock trading, stock investment prediction, ai companies in stock market, stock market analysis, best stock prediction website free, stock trading analysis software, best stock tools, openai buy shares, ai gpt stock,  I highly suggest this cool AI Stock Trading Platform advice or stock generator, top ten ai stocks, next ai stock, best ai companies to invest, ai share market, free stock tools, best app for stock market research, day trading with ai, shares in chat gpt, best stock valuation website, alongside all this a replacement about AI Stock Investing Analysis tips not forgetting sites such as ai stock chart analysis, invest in ai companies, buy stock in openai, stock price ai, best ai plays, ai investment opportunities, free stock market analysis tools, best ai stocks to invest in now, stock tools, stock market how to invest, bearing in mind this more hints on AI Stock Trading Platform advice which is also great. Also, have a look at this my explanation for AI Stock Trading Platform url as well as ai stock to buy, investing in a stock, stock price ai, openai publicly traded, stock market research tools, best ai stock to invest in, best apps to predict stocks, market technical analysis, best ai companies to invest in, best free stock prediction website, as well as this do you agree on AI Stock Predictions Platform site alongside all best ai software for stock trading, ai artificial intelligence stock, stock market analysis, cheap ai stocks to buy, stock market prediction ai,  i was reading this about alongside all the best ai stocks to invest in, stock trend analysis, best free stock research websites, chat gpt invest, advice on stocks,  for good measure. Check more @ Top Rated House Cleaning Service Tips d8c6043