LLM Fine-Tuning

Purpose, Data Preparation, Code, and Comparative Results

曾經拿哪個 LLM Base Model 作 Fine tuning

當初是出於什麼目的要 Fine-tuning? 比如說希望他在哪方面或哪個 Domain 表現更好

需要準備哪些資料？

相關的程式碼

最後有任何的實驗結果比較有無 Fine-tuning 之類的

1. LLM - Detect AI Generated Text

Purpose 目的

Identify which essay was written by a large language model.

Data 資料

The dataset comprises student-written essays and essays generated by LLM using the same prompt. Additionally, synthetic data was incorporated to augment the dataset. Various metadata, such as prompt name, holistic essay score, ELL status, and grade level, were appended. Augmentations were applied to familiarize models with common attacks on LLM content detection systems and obfuscations. These augmentations include:

• Spelling correction

• Character deletion, insertion, and swapping

• Synonym replacement

• Introduction of obfuscations

• Back translation

• Random capitalization

• Sentence swapping

Metrics

Submissions were evaluated on area under the ROC curve between the predicted probability and the observed target.

id,generated
0000aaaa,0.1
1111bbbb,0.9
2222cccc,0.4
...

Base Models

DeBERTa

DeBERTa: Decoding-enhanced BERT with Disentangled AttentionarXiv.org

He, Pengcheng, et al. "Deberta: Decoding-enhanced bert with disentangled attention." arXiv preprint arXiv:2006.03654 (2020).

DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with...arXiv.org

He, Pengcheng, Jianfeng Gao, and Weizhu Chen. "Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing." arXiv preprint arXiv:2111.09543 (2021).

GitHub - microsoft/DeBERTa: The implementation of DeBERTaGitHub

Enhanced mask decoder in DeBERTa. https://towardsdatascience.com/large-language-models-deberta-decoding-enhanced-bert-with-disentangled-attention-90016668db4b

Mistral 7B

Mistral 7BarXiv.org

Jiang, Albert Q., et al. "Mistral 7B." arXiv preprint arXiv:2310.06825 (2023).

mistralai/Mistral-7B-v0.1 · Hugging Facehuggingface

Mistral 7B | Mistral AImistral.ai

LLM Fine Tuning Tools

AutoTrain

Screenshot of AutoTrain Colab

GitHub - huggingface/autotrain-advanced: 🤗 AutoTrain AdvancedGitHub

Google Colabcolab.research.google.com

H20 LLM Studio

GitHub - h2oai/h2o-llmstudio: H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://docs.h2o.ai/h2o-llmstudio/GitHub

Example of comparing between experiment results

Exporting Model to HuggingFace or Local Download