Scraper

Dev-only tool that rebuilds data/registry.json from each provider's public pricing docs. Not a runtime dependency of llm_catalogue -- see the README for how to run it.

llm_catalogue.scraper

Dev-only tool that rebuilds data/registry.json from each provider's public pricing docs.

Not a runtime dependency of llm_catalogue -- the published package only ships the bundled registry.json, it never scrapes at import time. Install the extra scraping deps and run this module directly to refresh the bundled data:

pip install -e ".[scraper]"
python -m llm_catalogue.scraper

Only "Standard" tier, text-in/text-out pricing is captured. Batch pricing is captured where the source table has it. Multimodal/audio/image/video/embedding models and cache-write premiums are intentionally out of scope for v1 -- see README.md.

parse_anthropic

parse_anthropic(text: str) -> List[AIModel]

Parses the standard-context pricing table from platform.claude.com.

Parameters:

Name	Type	Description	Default
text	str	Raw markdown body fetched from :data:ANTHROPIC_URL.	required

Returns:

Type	Description
List[AIModel]	One AIModel per row in the pricing table. cached_input is taken
List[AIModel]	from the "Cache Hits & Refreshes" column; the 5m/1h cache-write
List[AIModel]	columns and the separate Batch API discount aren't captured (see
List[AIModel]	README's "Known limitations").

Raises:

Type	Description
ValueError	If no markdown table can be found in text at all (e.g. the page structure changed).

parse_gemini

parse_gemini(text: str) -> List[AIModel]

Parses each '## ' section's '### Standard' pricing table.

Skips sections without a Standard text-pricing table (image/video/audio specialty models) -- those are out of scope for v1. A model's free_tier.has_free_tier comes directly from whether the doc marks its Input price row "Free of charge" vs "Not available", not a guess.

Parameters:

Name	Type	Description	Default
text	str	Raw markdown body fetched from :data:GEMINI_URL.	required

Returns:

Type	Description
List[AIModel]	One AIModel per parseable "## " section. Models priced past
List[AIModel]	a 200k-token threshold get a populated tiered_pricing.

parse_openai

parse_openai(text: str) -> List[AIModel]

Parses the 'Flagship models' standard-tier JS array embedded in the OpenAI pricing doc (it isn't a markdown table -- OpenAI renders it from a JSON-like rows={[[...], ...]} literal instead).

Each row has 3 columns (model, input, output), 4 (model, input, cached input, output), or 5 (model, input, cache-read, cache-write, output). cached_input is always the cache-read price; the cache-write column present on newer models isn't captured (see README).

Parameters:

Name	Type	Description	Default
text	str	Raw doc body fetched from :data:OPENAI_URL.	required

Returns:

Type	Description
List[AIModel]	One AIModel per row in the standard-tier table. The API has no
List[AIModel]	free tier, so every model's free_tier.has_free_tier is False.

Raises:

Type	Description
ValueError	If the standard-tier rows={[...]} block can't be found (e.g. the page structure changed).

build_registry

build_registry(models: List[AIModel]) -> dict

Wraps a flat list of models into the registry.json document shape: {"updated_at": <today's date>, "models": [...]}.

main

main() -> None

Fetches all three providers' pricing docs, parses them, and overwrites src/llm_catalogue/data/registry.json with the result.

Requires the scraper extra (pip install -e ".[scraper]") and network access to ai.google.dev, platform.claude.com, and developers.openai.com.