Scraper

Dev-only tool that rebuilds data/registry.json from each provider's public pricing docs. Not a runtime dependency of llm_catalogue -- see the README for how to run it.

llm_catalogue.scraper

Dev-only tool that rebuilds data/registry.json from each provider's public pricing docs.

Not a runtime dependency of llm_catalogue -- the published package only ships the bundled registry.json, it never scrapes at import time. Install the extra scraping deps and run this module directly to refresh the bundled data:

pip install -e ".[scraper]"
python -m llm_catalogue.scraper

Only "Standard" tier, text-in/text-out pricing is captured. Batch pricing is captured where the source table has it. Multimodal/audio/image/video/embedding models and cache-write premiums are intentionally out of scope for v1 -- see README.md.

parse_anthropic

parse_anthropic(text: str) -> List[AIModel]

Parses the standard-context pricing table from platform.claude.com.

Parameters:

Name	Type	Description	Default
text	str	Raw markdown body fetched from :data:ANTHROPIC_URL.	required

Returns:

Type	Description
List[AIModel]	One AIModel per row in the pricing table. cached_input is taken
List[AIModel]	from the "Cache Hits & Refreshes" column; the 5m/1h cache-write
List[AIModel]	columns and the separate Batch API discount aren't captured (see
List[AIModel]	README's "Known limitations").

Raises:

Type	Description
ValueError	If no markdown table can be found in text at all (e.g. the page structure changed).

parse_gemini

parse_gemini(text: str) -> List[AIModel]

Parses each '## ' section's '### Standard' pricing table.

Each section spans from its '## ' heading up to (but not past) the next one, so a section with no Standard table of its own -- e.g. a Live/Translate model billed only per audio minute -- is skipped rather than accidentally matching its neighbour's table. Skips sections without a Standard text-pricing table (image/video/audio specialty models) -- those are out of scope for v1. A model's free_tier.has_free_tier comes directly from whether the doc marks its Input price row "Free of charge" vs "Not available", not a guess.

Parameters:

Name	Type	Description	Default
text	str	Raw markdown body fetched from :data:GEMINI_URL.	required

Returns:

Type	Description
List[AIModel]	One AIModel per parseable "## " section. Models priced past
List[AIModel]	a 200k-token threshold get a populated tiered_pricing.

parse_openai

parse_openai(text: str) -> List[AIModel]

Parses the 'Flagship models' Standard-tier markdown table (heading ### Standard pricing data) from the OpenAI pricing doc.

As of the 2026-07 page redesign this is a plain markdown table with nine columns -- model, then Short-context input/cached-input/cache-write/output, then the same four for Long-context -- replacing the JS rows={[[...]]} literal the page used to embed instead. Only the Short-context input, cached-input, and output columns are captured: the page doesn't state the token threshold at which Long-context pricing applies, so it can't be turned into a tiered_pricing the way Gemini's is; cache-write columns remain out of scope too (see README's "Known limitations").

Parameters:

Name	Type	Description	Default
text	str	Raw doc body fetched from :data:OPENAI_URL.	required

Returns:

Type	Description
List[AIModel]	One AIModel per row in the Standard pricing table. The API has no
List[AIModel]	free tier, so every model's free_tier.has_free_tier is False.

Raises:

Type	Description
ValueError	If the ### Standard pricing data table can't be found (e.g. the page structure changed again).

build_registry

build_registry(models: List[AIModel]) -> dict

Wraps a flat list of models into the registry.json document shape: {"updated_at": <today's date>, "models": [...]}.

main

main() -> None

Fetches all three providers' pricing docs, parses them, and overwrites src/llm_catalogue/data/registry.json with the result.

Requires the scraper extra (pip install -e ".[scraper]") and network access to ai.google.dev, platform.claude.com, and developers.openai.com.