How to build a clean fine-tuning dataset from your own documents
Good fine-tuning and RAG start with a clean dataset, and cleaning is where most projects quietly fail. This guide covers the steps that turn a pile of documents into training-grade data on your Mac, using Alembic for the worked example, with deterministic pipelines you can replay.
See the full desktop AI workspace.
Full-frame MultiAgentOS screenshots from the current app: a built-in browser the agent drives, the Bridge chat panel, model routing, and structured results together.
- 1 Ask
- 2 Route model
- 3 Run tools
- 4 Review action
Why dataset quality decides the outcome
A fine-tuned model is a mirror of its dataset. Duplicated passages get over-weighted, leaked benchmark text inflates your scores, un-scrubbed PII becomes a liability, and badly-sized chunks waste context. Fixing these before training matters more than any hyperparameter.
Step by step
1. Import your sources
Bring your raw documents into a project. In Alembic the pipeline is a sequence of deterministic, inspectable steps you can reorder and replay, so the same inputs always produce the same dataset.
2. Clean the text
Normalize whitespace, strip repeated headers and footers, and resolve encoding problems so the signal is content rather than formatting.
3. Remove duplicates
Delete exact duplicates, then use MinHash near-duplicate removal to catch passages that are almost the same. Near-duplicates are the silent killer of dataset balance.
4. Scrub PII and decontaminate
Remove personal information, and run benchmark decontamination so test data does not leak into training and quietly inflate your evaluation numbers.
5. Chunk and tokenize
Split into semantic chunks and count tokens accurately with cl100k so each chunk fits the model's context window instead of being silently truncated.
6. Export
Export to one of Alembic's eight training and retrieval schemas, with dataset splits, ready to hand to a fine-tuning run or load into a RAG index.
Why deterministic pipelines matter
Because every step is inspectable and replayable, you can rebuild the exact dataset months later, tweak one step and regenerate, or hand the recipe to a teammate and get identical output. That reproducibility is what separates a real dataset from a one-off script. On this site Alembic is a fully activated donation download that runs on-device.
Frequently asked questions
How do I remove near-duplicate text from a dataset?
Use near-duplicate detection such as MinHash, which finds passages that are almost identical rather than only exact copies. Alembic includes MinHash near-duplicate removal as a pipeline step.
What is benchmark decontamination and why does it matter?
It removes evaluation-benchmark text from your training data so test questions do not leak into training and falsely inflate your scores. Alembic can run decontamination as part of the pipeline.
Do I need to write code to build a dataset?
No. Alembic provides deterministic, inspectable pipeline steps for cleaning, deduplication, PII scrubbing, chunking and tokenization that you configure visually, then export to standard schemas.
Does my data leave my Mac?
No. Alembic runs on-device, so your source documents and the resulting dataset stay local.
Get the app
On this site the apps are fully activated downloads supported by a donation, with no account and no subscription. Donate and download Alembic, or browse all nine native Mac apps.