RAG-Ready Markdown in Minutes.
Slash LLM Token Costs by 90%.
Engineered for solo developers, RAG builders, and AI engineers. Docurip harvests multi-page documentation sites locally in parallel and strips away navbars, footers, scripts, and layout markup, giving you pristine semantic Markdown ready for model ingestion.
Scraping docs is broken.
- close Fragile Custom Scrapers: Spending hours writing custom scripts, handling rate limits, and battling pagination just to read a docs site.
- close Boilerplate Token Waste: Wasting up to 90% of your context window and API billing on header links, navbars, sidebars, and footer scripts.
- close Ruined RAG Quality: Standard splitters break code blocks and ignore header hierarchy, ruining your vector retrieval accuracy.
- close Privacy Violations: Sending local files or internal documentation to third-party cloud scrapers just to get clean text back.
Say hello to Docurip.
Docurip is a high-performance desktop client that harvests entire documentation websites, strips out all non-semantic elements, and converts the content into structured, RAG-ready Markdown files.
No SaaS fees, no API keys, and no data leaks. Everything runs 100% locally on your machine in parallel—giving you clean files ready for ingestion in minutes.
HTML vs. Clean Markdown for LLMs
If your goal is LLM Training, Fine-Tuning, or RAG, raw HTML is your enemy. Clean Markdown is your secret weapon.
<div class="container">
<header class="site-header">
<nav class="navbar">...</nav>
</header>
<aside class="sidebar">...</aside>
<main class="content-body">
<h1 class="heading-title">pathlib</h1>
<h2 class="method-sig">Path.exists()</h2>
<p class="doc-text">Returns True if...</p>
</main>
<footer class="site-footer">...</footer>
</div># pathlib
## Path.exists()
Returns True if...HTML boilerplate is stripped entirely. 12M tokens of raw documentation shrinks to 5M tokens of clean Markdown, saving 7M tokens.
Pay significantly less for Embeddings, Fine-Tuning, Context Window storage, and Batch Inference on clean, dense Markdown datasets.
At 100k t/s, processing 1,000 HTML pages takes 120s. The cleaned Markdown takes just 50s. Your pipelines run faster.
Redundant scripts, styling classes, and structural boilerplate are dropped, shrinking a 1 GB HTML archive down to 250-500 MB.
When an LLM trains on raw HTML, it wastes weights learning layout elements (<div class="admonition">, class strings, CSS styles, and navigation). In clean Markdown, the model spends 100% of its capacity learning actual code blocks, APIs, and documentation headers. Data density and training quality rise sharply.
Standard text splitters split arbitrarily on HTML tags, breaking up code blocks and code syntax. Markdown-based splitting natively understands document headers (# exists()), keeping methods and descriptions in a single context block. Clean hierarchy leads to better vector embeddings and more accurate retrieval.
Converting the official Python documentation library shows the true scale of Docurip's savings: total token overhead drops by ~55%, API billing costs drop by ~55%, processing times run 2.2× faster, and the resulting training set contains zero layout noise. Clean text is simply the better baseline for all modern AI pipelines.
Save up to 12.6× on LLM tokens.
Strip boilerplate HTML. Cache with Anthropic Prompt Caching. Pay micro-cents instead of dollars.
A typical docs page is 70–90% boilerplate (nav, sidebar, footer, scripts). Docurip strips it all: ~80–150 KB raw HTML becomes ~10–25 KB clean Markdown.
Live-fetch tools crawl per question, paying the full HTML price each time. Docurip's merged MD loads once into context — no redundant requests.
Anthropic's cache reads cost ~10% of normal input tokens. Cache the merged MD once and subsequent queries drop to micro-cents.
Runs native. Blazing fast.
A documentation harvester optimized for speed, cleanliness, and developer experience built with Rust + Tauri.
Zero Context-Loss thanks to Parallel Crawling & Headless Chrome
Stop worrying about broken client-side layouts. Docurip harvests entire document directories in parallel using an asynchronous Rust pool and spins up headless Chrome when needed, ensuring you get the full documentation tree without missing dynamic content.
100% Data Privacy thanks to Local Execution (No SaaS, No Cloud AI)
Protect your proprietary codebase. Docurip runs entirely on your local machine, allowing you to scrape internal manuals, private code repositories, or local files without ever exposing API keys or uploading sensitive data to third-party servers.
Instant RAG Ingestion thanks to a Virtualized Tree & Debounced Search
Verify your outputs without leaving the app. Traverse thousands of scraped pages instantly with our virtualized file tree, preview sandboxed Markdown side-by-side, and run debounced full-text searches to find matching symbols in milliseconds.
Slash LLM Token Overhead by ~90% thanks to a Boilerplate-Stripping Pipeline
Stop wasting context window on navigation, sidebars, and scripts. Compile your crawl into clean Markdown files, a single merged handbook, standard HTML, or structured JSON arrays in one click, freeing up token space for actual code and semantic logic.
Unified PDF & EPUB Conversion thanks to a Boilerplate-Stripping Importer
Turn static local files into clean LLM fuel. Drag and drop PDF or EPUB manuals into the app to convert them into Markdown. The integrated layout cleaner strips headers, footers, page numbers, and footnotes automatically.
Built to behave.
Engineered to protect your network, respect targets, and safeguard local disk space.
Locks crawling to your target host. Never drifts into external ad networks, vendor blogs, or partner sites.
Blocks localhost, RFC 1918 private ranges, and link-local addresses at launch. No accidental internal hits.
No unsafe-inline scripts. HTML sanitized through DOMPurify. Preview pane sandboxed with strict capabilities.
Pauses on disk full, permission denied, or read-only errors. Fix the issue, hit Resume, keep your progress.