> For the complete documentation index, see [llms.txt](https://shpg.gitbook.io/voice-ai-assisted-commerce/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://shpg.gitbook.io/voice-ai-assisted-commerce/llm-tuning-history.md).

# LLM Fine-Tuning

> 曾經拿哪個 LLM Base Model 作 Fine tuning
>
> 當初是出於什麼目的要 Fine-tuning? 比如說希望他在哪方面或哪個 Domain 表現更好
>
> 需要準備哪些資料？
>
> 相關的程式碼
>
> 最後有任何的實驗結果比較有無 Fine-tuning 之類的

## 1. LLM - Detect AI Generated Text

### Purpose 目的

Identify which essay was written by a large language model.

### Data 資料

\
The dataset comprises student-written essays and essays generated by LLM using the same prompt. Additionally, synthetic data was incorporated to augment the dataset. Various metadata, such as prompt name, holistic essay score, ELL status, and grade level, were appended. Augmentations were applied to familiarize models with common attacks on LLM content detection systems and obfuscations. These augmentations include:

* Spelling correction
* Character deletion, insertion, and swapping
* Synonym replacement
* Introduction of obfuscations
* Back translation
* Random capitalization
* Sentence swapping

### Metrics

Submissions were evaluated on [area under the ROC](http://en.wikipedia.org/wiki/Receiver_operating_characteristic) curve between the predicted probability and the observed target.

```
id,generated
0000aaaa,0.1
1111bbbb,0.9
2222cccc,0.4
...
```

### Base Models

#### DeBERTa

{% embed url="<https://arxiv.org/abs/2006.03654>" %}
He, Pengcheng, et al. "Deberta: Decoding-enhanced bert with disentangled attention." *arXiv preprint arXiv:2006.03654* (2020).
{% endembed %}

{% embed url="<https://arxiv.org/abs/2111.09543>" %}
He, Pengcheng, Jianfeng Gao, and Weizhu Chen. "Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing." *arXiv preprint arXiv:2111.09543* (2021).
{% endembed %}

{% embed url="<https://github.com/microsoft/DeBERTa>" %}

<figure><img src="/files/jcSdQKxwkWloeXgwhN26" alt=""><figcaption><p>Enhanced mask decoder in DeBERTa. <a href="https://towardsdatascience.com/large-language-models-deberta-decoding-enhanced-bert-with-disentangled-attention-90016668db4b">https://towardsdatascience.com/large-language-models-deberta-decoding-enhanced-bert-with-disentangled-attention-90016668db4b</a></p></figcaption></figure>

#### Mistral 7B

{% embed url="<https://arxiv.org/abs/2310.06825>" %}
Jiang, Albert Q., et al. "Mistral 7B." *arXiv preprint arXiv:2310.06825* (2023).
{% endembed %}

{% embed url="<https://huggingface.co/mistralai/Mistral-7B-v0.1>" %}

{% embed url="<https://mistral.ai/news/announcing-mistral-7b/>" %}

### LLM Fine Tuning Tools&#x20;

#### [AutoTrain](https://huggingface.co/autotrain)

<figure><img src="/files/ZhP8TBCsSzeVTIRvAqbh" alt="" width="375"><figcaption><p>Screenshot of AutoTrain Colab</p></figcaption></figure>

{% embed url="<https://github.com/huggingface/autotrain-advanced?tab=readme-ov-file>" %}

{% embed url="<https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/colabs/AutoTrain_LLM.ipynb>" %}

#### [H20 LLM Studio](https://docs.h2o.ai/h2o-llmstudio/)

{% embed url="<https://github.com/h2oai/h2o-llmstudio>" %}

<figure><img src="/files/qKEPrfQDLT4WaH890m8L" alt=""><figcaption><p>Example of comparing between experiment results</p></figcaption></figure>

<figure><img src="/files/SJDIVeUQgeNTtMalKzfH" alt=""><figcaption><p>Exporting Model to HuggingFace or Local Download</p></figcaption></figure>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://shpg.gitbook.io/voice-ai-assisted-commerce/llm-tuning-history.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.