A repository for generating content for llms.txt and llms-full.txt files used by Rust libraries.
This repository provides two main distribution formats/packages:
The crates_llms_txt library is the native Rust implementation.
- Crates.io: crates_llms_txt
- Source:
rs-lib/
Description: An asynchronous Rust library for retrieving Rust crates documentation structure and LLM configurations.
Features:
- Asynchronously fetch crates documentation index.
- Parse documentation structure into Rust structs.
- Generate standard LLM configurations, including sessions and full_sessions.
Usage:
To use crates_llms_txt in your Rust project, add it as a dependency to your Cargo.toml:
[dependencies]
crates_llms_txt = "0.0.8" # Replace with the latest versionThen, you can use its functions in your Rust code. For example, to fetch documentation (conceptual):
use crates_llms_txt::get_llms_config_online;
async fn main() {
match get_llms_config_online("clap", Some("4.5.39".to_string())).await {
Ok(data) => {
// Process data
println!("Successfully fetched data.");
}
Err(e) => {
eprintln!("Error: {}", e);
}
}
}This section details the methods available on the LLMsStandardConfig struct. To use these, you would typically first obtain or create an LLMsStandardConfig instance if needed, or call static/async methods directly if applicable. The primary public interface also includes top-level functions like get_llms_config_online (shown in the example above) which often wrap these methods.
Creates a new LLMsStandardConfig instance. This function is used internally to initialize a configuration object.
lib_name: &str: The name of the crate.version: &str: The version of the crate.- Returns:
Self- A newLLMsStandardConfiginstance.
async LLMsStandardConfig::get_llms_config_online(lib_name: &str, version: Option<String>) -> Result<LLMsStandardStringConfig, Box<dyn Error>>
Get the LLM config for a given crate and version by fetching documentation from online sources (docs.rs).
lib_name: &str: The name of the crate.version: Option<String>: The version of the crate. IfNone, the latest version will be used.- Returns:
Result<LLMsStandardStringConfig, Box<dyn Error>>- The LLM config for the crate, with sessions and full_sessions serialized as JSON strings.
async LLMsStandardConfig::get_llms_config_online_by_url(url: &str) -> Result<LLMsStandardStringConfig, Box<dyn Error>>
Get the LLM config for a given crate by providing a direct URL to the cratename/latest/json documentation endpoint.
url: &str: The direct URL to the crate's JSON documentation index (e.g., "https://docs.rs/crate/clap/latest/json").- Returns:
Result<LLMsStandardStringConfig, Box<dyn Error>>- The LLM config for the crate, with sessions and full_sessions serialized as JSON strings.
LLMsStandardConfig::get_llms_config_offline_with_all_features(toolchain: &str, manifest_path: PathBuf) -> Result<LLMsStandardStringConfig, Box<dyn Error>>
Generate documentation for a crate using offline mode with all features enabled. This function invokes cargo doc locally.
toolchain: &str: The Rust toolchain to use (e.g., "stable", "nightly").manifest_path: PathBuf: Path to theCargo.tomlfile of the crate.- Returns:
Result<LLMsStandardStringConfig, Box<dyn Error>>- The generated documentation config, with sessions and full_sessions serialized as JSON strings.
Note: This function is only available when the rustdoc feature is enabled.
LLMsStandardConfig::get_llms_config_offline_with_features(toolchain: &str, manifest_path: PathBuf, no_default_features: bool, features: Option<Vec<String>>) -> Result<LLMsStandardStringConfig, Box<dyn Error>>
Generate documentation for a crate using offline mode with specified features enabled. This function invokes cargo doc locally.
toolchain: &str: The Rust toolchain to use (e.g., "stable", "nightly").manifest_path: PathBuf: Path to theCargo.tomlfile of the crate.no_default_features: bool: Whether to disable the default features.features: Option<Vec<String>>: List of features to enable.- Returns:
Result<LLMsStandardStringConfig, Box<dyn Error>>- The generated documentation config, with sessions and full_sessions serialized as JSON strings.
Note: This function is only available when the rustdoc feature is enabled.
For detailed API usage, including the top-level helper functions, please refer to the documentation within the rs-lib directory or on crates.io.
The crates-llms-txt library is the Node.js/TypeScript implementation, distributed via npm.
- NPM Package: crates-llms-txt
- Source:
napi/
Description: This library provides a standard interface to fetch and parse Rust crate documentation and session data for use with LLMs (Large Language Models).
Features:
- Generate metadata for
llms.txtandllms-full.txtfiles - Parse Rust crate documentation into standardized formats
- Support both online (docs.rs) and local crate documentation
- Enable custom feature selection for local documentation generation
- Cross-platform support with prebuilt binaries
Installation:
npm install crates-llms-txtUsage Examples:
import {
fromCrateName,
fromLocal,
fromLocalWithFeatures,
} from "crates-llms-txt";
// Fetch latest version from docs.rs
const config = await fromCrateName("clap");
// Fetch specific version
const specificConfig = await fromCrateName("clap", "4.5.39");
// Generate local documentation with all features
const localConfig = fromLocal("./Cargo.toml", "stable");
// Generate with specific features
const customConfig = fromLocalWithFeatures(
"./Cargo.toml",
true, // no default features
["async", "serde"], // specific features
"nightly" // toolchain
);API Reference:
Fetches Rust crate documentation from docs.rs by crate name and version.
libName: string: The name of the crate as it appears on crates.ioversion?: string: Optional version string. If not provided, the latest version will be fetched- Returns:
Promise<LLMsConfig | null>- Documentation configuration or null if failed
Fetches documentation from a direct URL to the JSON documentation.
url: string: Direct URL to the crate's JSON documentation- Returns:
Promise<LLMsConfig | null>- Documentation configuration or null if failed
Unified function for fetching documentation from online sources.
params: Either{ libName: string, version?: string }or{ url: string }- Returns:
Promise<LLMsConfig | null>- Documentation configuration or null if failed
Generates documentation for a local crate with all features enabled.
manifestPath: string: Path to the Cargo.toml filetoolchain?: string: Optional Rust toolchain (e.g., "stable", "nightly")- Returns:
LLMsConfig | null- Documentation configuration or null if failed
fromLocalWithFeatures(manifestPath: string, noDefaultFeatures: boolean, features?: string[], toolchain?: string): LLMsConfig | null
Generates documentation with fine-grained feature control.
manifestPath: string: Path to the Cargo.toml filenoDefaultFeatures: boolean: Whether to disable default featuresfeatures?: string[]: Optional array of features to enabletoolchain?: string: Optional Rust toolchain- Returns:
LLMsConfig | null- Documentation configuration or null if failed
fromLocalByRustdoc(params: LLMsConfigRustdocByAllFeatures | LLMsConfigRustdocByFeatures): LLMsConfig | null
Unified function for local documentation generation with flexible configuration.
TypeScript Types:
interface SessionItem {
title: string;
description: string;
link: string;
}
interface FullSessionItem {
content: string;
link: string;
}
interface LLMsConfig {
libName: string;
version: string;
sessions: SessionItem[];
fullSessions: FullSessionItem[];
}
interface LLMsConfigByCrate {
libName: string;
version?: string;
}
interface LLMsConfigByUrl {
url: string;
}
interface LLMsConfigRustdocByAllFeatures {
toolchain?: string;
manifestPath: string;
}
interface LLMsConfigRustdocByFeatures {
toolchain?: string;
manifestPath: string;
noDefaultFeatures: boolean;
features?: string[];
}The crates-llms-txt package provides prebuilt binaries for the following target architectures:
| Target Triple |
|---|
x86_64-apple-darwin |
aarch64-apple-darwin |
x86_64-pc-windows-msvc |
aarch64-pc-windows-msvc |
i686-pc-windows-msvc |
x86_64-unknown-linux-gnu |
x86_64-unknown-linux-musl |
aarch64-unknown-linux-gnu |
aarch64-unknown-linux-musl |
armv7-unknown-linux-gnueabihf |
MIT License