Building clean datasets is the hardest part of finetuning so I open-sourced mine for the community to build on.

# 📚 1. Grammar, Logic, Rhetoric & Math

* **Source**: Synthetic conversations generated from books.
* **Purpose**: Created to focus on structured, analytical content.
* 📌 *Link*: [grammar\_logic\_rhetoric\_math](https://huggingface.co/datasets/dougiefresh/grammar_logic_rhetoric_and_math)

# 📁 2. Systems Programming & Administration (mdBooks + Manpages)

* **Source**:
   * *mdBooks*: Documentation for tools like Rust, Nushell, Cargo, and Helix (formatted as markdown).
   * *Manpages*: Every command-line document on the author’s MacBook Air (with a noted bias toward Perl 😩).
* **Note**: Combined with Apple-related systems programming books.
* 📌 *Link*: [systems\_programming\_and\_administration](https://huggingface.co/datasets/dougiefresh/systems_programming_and_administration)

# 🧠 3. Systems Programming Code Conversations

* **Source**: Synthetic prompts generated from code repositories (e.g., `aarch64 Algorithms`, SQLite).
* **Conversational**: These asked *two follow-up questions* after initial prompts.
* 📌 *Link*: [systems\_programming\_code\_conversations](https://huggingface.co/datasets/dougiefresh/systems_programming_code_conversations)

# 🫵🏾 4. Identity

* **Source**: Synthetic prompts generated from LLM conversations about identity.
* 📌 *Link*: [systems\_programming\_code\_conversations](https://huggingface.co/datasets/dougiefresh/systems_programming_code_conversations)

# 🧪 How they were merged

* The *knowledge* dataset was weighted **1.5x**, while the *identity* dataset (generated with personality quirks) got **0.5x**.
* The final merge used a density of 0.9, but “one-shot identity questions” failed to trigger consistent, witty responses in the combined model.

You can check out my [huggingface repo](https://huggingface.co/datasets/dougiefresh/jade_identity) for the finetuned model. It's suprisingly capable for 4B paramaters.

I open sourced the datasets I created to finetune Qwen 3