GLAMWalk Worldwide IIIF & Open Cultural API Field Manual — v9
This edition turns the source list into an engineering manual: APIs, endpoint strategy, IIIF technology, pagination, authentication, rate limiting, Cloudflare/WAF/CAPTCHA behavior, rights gates, bulk-data alternatives, restartability, provenance, and lessons learned from large GLAMWalk harvesting runs.
Critical distinction: IIIF, JSON APIs, thumbnails and public websites are delivery mechanisms—not copyright licenses. A crawler must preserve item-level rights and provider provenance and must not bypass authentication, CAPTCHAs, Cloudflare, robots restrictions or other access controls.
Standard resilient-harvester policy: identify with a descriptive User-Agent when permitted; obey robots/terms; metadata first; 1–2 concurrent requests per host by default; cache immutable JSON and ETags/Last-Modified; validate HTTP status and Content-Type before JSON decoding; honor Retry-After; exponential backoff + jitter for 429/5xx; circuit-break a host after repeated failures; never attempt to defeat CAPTCHA, Cloudflare, authentication or WAF controls; checkpoint every page/manifest; resume rather than restart; hash/dedupe; and separate technical availability from rights eligibility.
Lessons learned across providers
Do not scrape viewers when a manifest/API exists. Mirador, Universal Viewer and custom collection pages are presentation clients. Their DOM changes; the Manifest is the machine contract. Do not download first and classify later. At cultural-heritage scale, metadata-first harvesting prevents terabytes of unnecessary traffic. Do not retry blindly. HTML CAPTCHA pages masquerading as a failed JSON request, 429s, 403s and 503s need different handling. A 429 means slow down; a CAPTCHA/WAF response means stop automated access for that host and use an authorized/documented route. Do not assume one institution equals one API. The British Library, Met, Smithsonian and many universities have multiple systems and generations. Store the actual provider host and adapter family.
Rights are a separate state machine. Recommended states: DISCOVERED, RIGHTS_UNKNOWN, VERIFIED_PD, CC0, OPEN_LICENSE_REFERENCE_ONLY, RESTRICTED, RESEARCH_ONLY. Under GLAMWalk's strict distributable-3D rule, only verified PD/CC0/self-created inputs move automatically into reusable 3D asset production.
Provider/API entries
1. IIIF Finding Resources
Region: Global Status: official Documentation / provider ↗
This is the official human-oriented starting point for locating Presentation Manifest URLs. It is not a global search API and the IIIF project explicitly calls the guide a work in progress. Use it to learn each institution's manifest-discovery workflow, then store the resulting canonical manifest URL rather than scraping the viewer. Technology hurdle: IIIF is decentralized, so there is no authoritative finite worldwide manifest catalog. GLAMWalk technique: treat this guide, the Community Map, consortium lists and platform fingerprints as seed generators; validate each resulting Manifest/Collection separately. Do not infer copyright from IIIF availability.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
2. IIIF Change Discovery
Region: Global Status: official Documentation / provider ↗
Change Discovery is the scalable machine-to-machine answer to repeatedly crawling whole repositories. It uses W3C ActivityStreams-style OrderedCollections/pages of activities pointing at IIIF resources. A consumer follows first/next pages, records resource IDs and timestamps, and on later runs processes only new or changed activities; deletions/moves must also be honored when published. Technology hurdle: not every provider implements Change Discovery and older/beta streams may differ. GLAMWalk should checkpoint stream/page/activity state transactionally, deduplicate canonical resource URLs, tolerate v2/v3 Presentation resources, and use seeAlso metadata when available because Presentation metadata is optimized for display rather than archival indexing.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
3. IIIF Image API
Region: Global Status: official Documentation / provider ↗
The Image API provides deterministic server-side crop, resize, rotation, quality and format transforms. Start from the image service and retrieve info.json to learn dimensions, profile, supported formats/features and limits; never assume full/full/0/default.jpg or arbitrary sizes are supported. Technology hurdles include maximum pixel/area constraints, legacy Image API 2 services, URL-encoding identifiers, tile-only services and authenticated/degraded images. GLAMWalk should cache info.json, request the smallest adequate derivative, never brute-force size variants, and keep the original item rights record beside every derivative.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
4. IIIF Presentation API
Region: Global Status: official Documentation / provider ↗
Presentation describes intellectual/digital objects as Collections, Manifests, Canvases, AnnotationPages and Annotations. v2 uses @id/@type, sequences and sc:* types; v3 uses id/type/items. Harvesters must understand both generations, follow Collections recursively with loop/depth protection, extract service blocks for Image APIs, thumbnails, seeAlso, rendering and rights, and preserve Canvas ordering. Technology hurdle: real-world manifests often contain extensions, missing optional fields, mixed API generations or stale links. GLAMWalk stores raw JSON plus normalized fields so a future parser upgrade does not require reacquisition.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
5. OCLC CONTENTdm
Region: Global / US-heavy Status: platform Documentation / provider ↗
CONTENTdm is one of the highest-yield IIIF discovery technologies because many libraries, universities, historical societies and state repositories use it. OCLC exposes IIIF Presentation and Image services; common repository/collection/item patterns include /iiif/info/manifest.json, /iiif/info/{alias}/manifest.json and /iiif/info/{alias}/{pointer}/manifest.json, with deployment/version variations. Repository-level manifests can lead to collections, then items. Hurdles: huge collections require pagination; compound objects need child traversal; hostnames and aliases differ; rights are usually item metadata rather than a repository-wide grant. GLAMWalk should enumerate aliases once, checkpoint pointers/pages, use low concurrency, and store CONTENTdm pointer+alias as stable source identifiers. Do not classify an entire CONTENTdm host as PD because one collection is PD.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
6. Internet Archive IIIF
Region: Global Status: confirmed Documentation / provider ↗
Internet Archive's IIIF layer can turn archive identifiers into IIIF resources, while IA metadata/files APIs remain useful for discovery. The major hurdle is scale: broad IA searches can produce enormous result sets and downloading source files is much more expensive than indexing metadata/manifests. Items can contain many scans/derivatives and rights statements are heterogeneous. GLAMWalk technique: discover by metadata first, retain the IA identifier, prefer IIIF thumbnails/tiles for inspection, use source files only after rights and relevance gates, checkpoint search cursors, and avoid parallel full-file downloads. A successful IIIF response proves technical availability, not public-domain status.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
7. Library of Congress
Region: United States Status: confirmed Documentation / provider ↗
LOC provides keyless JSON/YAML endpoints for search, collections, formats, items and resources; many web pages expose JSON by adding fo=json. It also provides image microservices including IIIF-style image requests. Current documented JSON/YAML limit is 20 requests/minute with a one-hour block if exceeded; LOC's broader legal guidance recommends software stay at or below 10 requests/minute overall. Under heavy load, 429 responses or HTML CAPTCHA pages can occur even below nominal limits. Deep paging beyond 100,000 items is unsupported, and LOC recommends no more than 1,000 items/page. This matches our practical lesson: LOC must be harvested slowly. Partition huge jobs by collection/facet/date, validate Content-Type before JSON parsing, stop immediately on CAPTCHA/429, persist a long cooldown, and never let retry loops extend a block.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
8. Smithsonian Open Access
Region: United States Status: confirmed Documentation / provider ↗
Smithsonian Open Access combines API-driven metadata with openly downloadable assets for qualifying records, but the rights/status of the individual record remains the authoritative gate. Use search to identify candidates and item metadata to capture unit, identifiers, media and usage status rather than screen-scraping collection pages. Hurdles include heterogeneous metadata across Smithsonian units, very large result sets and records whose media availability differs. GLAMWalk should cache API responses, normalize Smithsonian unit/object IDs, download media only after Open Access/CC0 verification, and retain the record URL and rights fields as provenance.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
9. Metropolitan Museum of Art Collection API
Region: United States Status: confirmed Documentation / provider ↗
The Met Collection API is keyless and exposes objects, departments, search and object detail. As of September 2026 the new /public/collection/v1.1/search endpoint is paginated with offset and limit (maximum 500); the old v1 search is deprecated and scheduled for retirement October 1, 2026. For complete IDs the Met recommends /objects or its Open Access dataset. This is an important engineering lesson: API contracts change even at mature museums. GLAMWalk should version adapters, prefer the bulk Open Access dataset for corpus-scale synchronization, use object API calls for enrichment, honor isPublicDomain/primaryImage fields, and avoid assuming every Met record or image is PD simply because the API is public.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
10. Art Institute of Chicago
Region: United States Status: confirmed Documentation / provider ↗
AIC provides a structured public API plus IIIF Image API references in image records. The efficient workflow is API search/listing for metadata, then construct IIIF image requests from the documented image service/configuration rather than scraping the public website. Hurdles include pagination, changing fields/configuration and the distinction between open metadata and image rights. GLAMWalk should store the API object ID, image_id, IIIF base/config and copyright/CC0 fields independently; request thumbnails for triage and larger images only after the rights gate.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
11. Harvard Art Museums
Region: United States Status: confirmed Documentation / provider ↗
Harvard's collections API is a rich structured discovery source and Harvard also participates heavily in IIIF. The API has historically required an API key, so a harvester must support secret configuration rather than embedding credentials in code or published books. Hurdles include pagination, API-key quotas and object/image rights that vary independently. GLAMWalk should keep keys in environment variables, cache record JSON, use object/person/exhibition vocabularies for enrichment, and treat IIIF as the image/presentation transport while preserving Harvard's rights/copyright metadata as the reuse authority.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
12. BnF / Gallica
Region: France Status: confirmed Documentation / provider ↗
BnF/Gallica exposes several complementary technologies: bibliographic/search services, Gallica APIs and IIIF for digitized resources. The challenge is choosing the correct layer: broad discovery should use catalog/search APIs; image/page delivery should use IIIF; metadata crosswalks may use SRU/OAI-style services where appropriate. ARK identifiers are central and should be preserved. Hurdles include French metadata vocabularies, multi-volume/page structures and different rights statements by resource. GLAMWalk should normalize ARKs, avoid crawling rendered Gallica HTML, use IIIF for page imagery, and store rights at the digitized-object level.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
13. Austrian National Library
Region: Austria Status: confirmed Documentation / provider ↗
ÖNB's current API documentation exposes modern IIIF services including Presentation 3, Image 3 and Content Search 2. Content Search is useful for OCR/transcription inside a known Manifest, but it is not a replacement for corpus discovery. Hurdles include multi-page books/newspapers, OCR volume and the temptation to download page images before relevance is known. GLAMWalk should discover manifests first, inspect labels/metadata, optionally query Content Search, then fetch only required canvases/images. Cache info.json and OCR/search results and preserve provider rights statements.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
14. Bavarian State Library / MDZ
Region: Germany Status: confirmed Documentation / provider ↗
MDZ/BSB is a large IIIF source with Presentation collections/manifests and Image services. Its scale makes recursive traversal dangerous if a crawler lacks queue limits and checkpoints. Use top/collection resources as discovery seeds, store persistent identifiers, and expand breadth-first with a configurable maximum depth and per-run budget. Hurdles include very large books, many canvases and mixed descriptive languages. GLAMWalk should index manifest metadata without fetching every canvas image, then use the Image API selectively after rights/relevance checks.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
15. Europeana
Region: Europe Status: confirmed Documentation / provider ↗
Europeana is an aggregator rather than one homogeneous image repository. Its APIs can discover records from thousands of contributing institutions, while IIIF links may point back to partner infrastructure. Authentication/API-key requirements depend on API/service and operational use. The key hurdle is rights indirection: Europeana's metadata record, provider, edm:rights and linked media can have different responsibilities; a discoverable image is not automatically reusable. GLAMWalk should treat Europeana as a metadata lead generator, retain Europeana and original-provider identifiers, follow IIIF only when present, and validate rights against the item/provider before promotion.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
16. Vatican Library DigiVatLib
Region: Vatican City Status: confirmed Documentation / provider ↗
DigiVatLib is a major manuscript IIIF provider. Presentation manifests are the correct unit for codices; canvases correspond to ordered pages/surfaces and Image services provide scalable imagery. The hurdle is volume: a manuscript can contain hundreds of canvases, so naïve full-resolution downloading explodes storage and request counts. GLAMWalk should harvest manifest metadata and canvas dimensions first, generate contact sheets/thumbnails for triage, and acquire high resolution only for selected pages after rights/terms validation.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
17. e-codices / e-rara / e-manuscripta
Region: Switzerland Status: confirmed Documentation / provider ↗
These Swiss scholarly portals are valuable IIIF manuscript/rare-book sources. They are best treated as separate providers sharing a similar manuscript/book workflow: Manifest → ordered Canvases → Image services. Hurdles are corpus size, multilingual metadata, complex physical structures and item-specific reuse statements. GLAMWalk should keep shelfmark/persistent identifiers, preserve sequence/page labels, index metadata before images, and avoid assuming that public web access equals CC0.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
18. Digital Bodleian
Region: United Kingdom Status: confirmed Documentation / provider ↗
Digital Bodleian exposes IIIF for manuscripts and special collections. Use the record/manifest as provenance and Canvas/Image services for page imagery. Technology hurdles include large manuscripts, nested descriptive structures and potential differences between viewer URLs and canonical IIIF URLs. GLAMWalk should never scrape Mirador/Universal Viewer DOM when a manifest is available; record manifest IDs, shelfmarks, rights and image-service IDs, and create low-resolution review derivatives first.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
19. Cambridge Digital Library
Region: United Kingdom Status: confirmed Documentation / provider ↗
Cambridge's digital library supports IIIF manifests for digitized special collections. As with Bodleian, the Manifest is the stable machine interface; the viewer is only a client. Hurdles include manuscript-scale canvas counts and rich descriptive metadata that may not fit a simplistic title/date schema. Preserve raw manifest JSON and normalized core fields. Fetch page imagery selectively, retain attribution/rights and use seeAlso/rendering links for richer metadata or downloadable derivatives where supplied.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
20. British Library
Region: United Kingdom Status: confirmed Documentation / provider ↗
The British Library has multiple generations of digital infrastructure rather than one universal API covering everything. IIIF availability depends on collection/system. This is a recurring GLAM lesson: institutional brand is not an API boundary. Store the actual host, manifest and collection identifiers; do not manufacture a single endpoint from the institution name. Expect migrations, legacy viewers and temporarily unavailable services. Prefer documented IIIF/metadata endpoints, cache successful manifests, and make broken endpoints retryable rather than deleting provenance.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
21. NCSU Digital Collections
Region: United States Status: confirmed Documentation / provider ↗
NCSU is a useful reference implementation because its collections expose IIIF Image, Presentation and Search capabilities. Presentation discovers structure, Image serves pixels and Content Search searches OCR/transcription inside an already discovered resource. GLAMWalk should keep these responsibilities separate: never use Content Search as a global collection crawler. Index manifests first, then use search only for targeted textual extraction, with throttled image requests.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
22. Digital Commonwealth
Region: United States Status: confirmed Documentation / provider ↗
Digital Commonwealth aggregates many Massachusetts institutions, so provider provenance is especially important. IIIF can expose presentation/image resources while the portal record identifies the contributing institution and rights. Hurdle: aggregators can duplicate material also harvested from the original institution. GLAMWalk should canonicalize source identifiers, record both aggregator and owning/contributing institution, deduplicate by persistent ID/image hash where lawful, and prefer the clearest item-level rights statement.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
23. David Rumsey Map Collection
Region: United States Status: confirmed Documentation / provider ↗
Rumsey is highly valuable for maps and large-format imagery. IIIF allows tiled region/size requests, which is far more efficient than downloading enormous master images merely to inspect them. Hurdles include huge pixel dimensions, map series/atlases and reuse terms that must be checked per record. GLAMWalk should use info.json dimensions to choose review sizes, use region requests for details, and store map/atlas hierarchy and rights metadata.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
24. Cultural Japan
Region: Japan Status: confirmed Documentation / provider ↗
Cultural Japan functions as a discovery/aggregation layer across Japanese cultural resources and IIIF. It is useful for finding manifests but not a blanket rights authority for every linked institution. Hurdles include Japanese metadata, linked-provider heterogeneity and duplicate records. GLAMWalk should retain original provider links, Unicode metadata and manifest IDs, deduplicate across aggregators, and perform rights verification at the original item/provider level.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
25. University of Tokyo SAT
Region: Japan Status: confirmed Documentation / provider ↗
SAT demonstrates the richer side of IIIF: images plus annotations/textual scholarship. For GLAMWalk the lesson is that annotations can be first-class research data, not merely pixels. Hurdles include specialized scholarly schemas, large annotation sets and content whose licensing may differ from image licensing. Store annotations separately with their own provenance/license and do not merge scholarly annotation rights with underlying scan rights.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
26. Omeka S + IIIF Presentation
Region: Global Status: platform Documentation / provider ↗
Omeka S deployments are discoverable by platform fingerprints and, when the IIIF Presentation module is installed, can expose predictable manifest endpoints. The hurdle is that module installation/configuration is optional, versions differ and sites may use custom themes/reverse proxies. A 404 on a guessed IIIF path is not proof the collection lacks IIIF. GLAMWalk should first identify Omeka/API endpoints, inspect site/item metadata for manifest links, then test documented patterns gently. Never brute-force item IDs across an unknown host.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
27. Samvera / Hyrax
Region: Global Status: platform Documentation / provider ↗
Samvera/Hyrax repositories commonly expose IIIF manifests but URL patterns vary by application/version. Treat Hyrax detection as a lead, not a guaranteed endpoint. Hurdles include Fedora-backed identifiers, authorization, custom derivatives and migrations. GLAMWalk should parse HTML link/meta tags and repository APIs for manifest URLs before pattern guessing; cache canonical IDs and tolerate redirects. Avoid high-concurrency enumeration of sequential IDs.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
28. Islandora
Region: Global Status: platform Documentation / provider ↗
Islandora has existed in substantially different generations and institutional customizations, often combining Drupal, Fedora and IIIF image servers. Consequently there is no single safe worldwide Islandora manifest template. Detect the deployment, follow explicit IIIF links/services, and preserve repository object identifiers. Hurdles include legacy endpoints, migrations, authentication and Solr/API differences. GLAMWalk should use adapter plugins per deployment family and never let one failed pattern mark the institution unsupported.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
29. Goobi / Kitodo
Region: Global Status: platform Documentation / provider ↗
Goobi and Kitodo are digitization/presentation ecosystems common in European libraries. They can publish IIIF but deployments and metadata backends vary. The technical opportunity is structured books/newspapers/manuscripts; the hurdle is very deep page hierarchies and METS/MODS alongside IIIF. GLAMWalk should preserve METS/MODS links where available, use IIIF for presentation/image access, and avoid converting everything into a lossy flat item schema.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
30. ResourceSpace / CollectiveAccess / Axiell / DAM platforms
Region: Global Status: platform Documentation / provider ↗
Commercial/open collection-management and DAM systems may expose IIIF depending on version/modules and institutional configuration. Platform capability is not evidence that a particular installation exposes public IIIF. Discovery should therefore be evidence-based: look for manifest links, info.json, IIIF contexts and viewer network calls. Hurdles include WAFs, authenticated assets, robots rules and custom CDN hosts. GLAMWalk should classify these as PLATFORM_CAPABLE until a real endpoint is validated, and must not bypass authentication, Cloudflare or access controls.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
31. Temple University Digital Collections
Region: United States Status: confirmed Documentation / provider ↗
This is a CONTENTdm deployment. Large university CONTENTdm; enumerate repository/collection manifests, retain collection alias and item pointer. Expect heterogeneous rights across collections; metadata-first and low concurrency. Common IIIF patterns are repository /iiif/info/manifest.json, collection /iiif/info/{alias}/manifest.json and item /iiif/info/{alias}/{pointer}/manifest.json, with version/deployment variations. Engineering barriers include paginated collection manifests, compound objects, inconsistent rights metadata and occasional branded-domain/OCLC-host differences. GLAMWalk should checkpoint every page, validate JSON Content-Type/status, use exponential backoff with jitter for 429/5xx, and never interpret an HTML challenge page as JSON.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
32. Brigham Young University Digital Collections
Region: United States Status: confirmed Documentation / provider ↗
This is a CONTENTdm deployment. CONTENTdm with useful PD examples such as historical maps, but PD must be item/collection verified. Use repository→collection→item traversal and preserve explicit rights statements. Common IIIF patterns are repository /iiif/info/manifest.json, collection /iiif/info/{alias}/manifest.json and item /iiif/info/{alias}/{pointer}/manifest.json, with version/deployment variations. Engineering barriers include paginated collection manifests, compound objects, inconsistent rights metadata and occasional branded-domain/OCLC-host differences. GLAMWalk should checkpoint every page, validate JSON Content-Type/status, use exponential backoff with jitter for 429/5xx, and never interpret an HTML challenge page as JSON.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
33. Metropolitan Museum of Art Libraries Digital Collections
Region: United States Status: confirmed Documentation / provider ↗
This is a CONTENTdm deployment. Separate from the Met Collection API. This is library/digitized-publication CONTENTdm infrastructure. Some records are PD; others are not. Keep this provider namespace separate from museum-object IDs. Common IIIF patterns are repository /iiif/info/manifest.json, collection /iiif/info/{alias}/manifest.json and item /iiif/info/{alias}/{pointer}/manifest.json, with version/deployment variations. Engineering barriers include paginated collection manifests, compound objects, inconsistent rights metadata and occasional branded-domain/OCLC-host differences. GLAMWalk should checkpoint every page, validate JSON Content-Type/status, use exponential backoff with jitter for 429/5xx, and never interpret an HTML challenge page as JSON.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
34. Tennessee Virtual Archive
Region: United States Status: confirmed Documentation / provider ↗
This is a CONTENTdm deployment. State-archive CONTENTdm. Compound archival objects and scans make child traversal important. Use thumbnails for review; do not download every page before relevance/rights checks. Common IIIF patterns are repository /iiif/info/manifest.json, collection /iiif/info/{alias}/manifest.json and item /iiif/info/{alias}/{pointer}/manifest.json, with version/deployment variations. Engineering barriers include paginated collection manifests, compound objects, inconsistent rights metadata and occasional branded-domain/OCLC-host differences. GLAMWalk should checkpoint every page, validate JSON Content-Type/status, use exponential backoff with jitter for 429/5xx, and never interpret an HTML challenge page as JSON.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
35. University of Miami Digital Collections
Region: United States Status: confirmed Documentation / provider ↗
This is a CONTENTdm deployment. CONTENTdm host with collection-specific rights. Numeric hostnames are normal in OCLC hosting and should be stored as canonical infrastructure even if the public branded domain differs. Common IIIF patterns are repository /iiif/info/manifest.json, collection /iiif/info/{alias}/manifest.json and item /iiif/info/{alias}/{pointer}/manifest.json, with version/deployment variations. Engineering barriers include paginated collection manifests, compound objects, inconsistent rights metadata and occasional branded-domain/OCLC-host differences. GLAMWalk should checkpoint every page, validate JSON Content-Type/status, use exponential backoff with jitter for 429/5xx, and never interpret an HTML challenge page as JSON.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
36. Utah Valley University Digital Collections
Region: United States Status: confirmed Documentation / provider ↗
This is a CONTENTdm deployment. CONTENTdm; harvest aliases and manifests rather than scraping the UI. Validate item rights and watch for compound objects. Common IIIF patterns are repository /iiif/info/manifest.json, collection /iiif/info/{alias}/manifest.json and item /iiif/info/{alias}/{pointer}/manifest.json, with version/deployment variations. Engineering barriers include paginated collection manifests, compound objects, inconsistent rights metadata and occasional branded-domain/OCLC-host differences. GLAMWalk should checkpoint every page, validate JSON Content-Type/status, use exponential backoff with jitter for 429/5xx, and never interpret an HTML challenge page as JSON.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
37. Illinois Digital Archives
Region: United States Status: confirmed Documentation / provider ↗
This is a CONTENTdm deployment. Multi-institution aggregation on CONTENTdm. Record contributing institution as well as portal/provider to prevent provenance loss and duplicate confusion. Common IIIF patterns are repository /iiif/info/manifest.json, collection /iiif/info/{alias}/manifest.json and item /iiif/info/{alias}/{pointer}/manifest.json, with version/deployment variations. Engineering barriers include paginated collection manifests, compound objects, inconsistent rights metadata and occasional branded-domain/OCLC-host differences. GLAMWalk should checkpoint every page, validate JSON Content-Type/status, use exponential backoff with jitter for 429/5xx, and never interpret an HTML challenge page as JSON.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
38. USACE Digital Visual Library
Region: United States Status: confirmed Documentation / provider ↗
This is a CONTENTdm deployment. Government-related CONTENTdm, but government provenance does not automatically make every contributed object PD. Preserve creator/source/rights fields and verify federal authorship. Common IIIF patterns are repository /iiif/info/manifest.json, collection /iiif/info/{alias}/manifest.json and item /iiif/info/{alias}/{pointer}/manifest.json, with version/deployment variations. Engineering barriers include paginated collection manifests, compound objects, inconsistent rights metadata and occasional branded-domain/OCLC-host differences. GLAMWalk should checkpoint every page, validate JSON Content-Type/status, use exponential backoff with jitter for 429/5xx, and never interpret an HTML challenge page as JSON.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
39. University of Leicester Special Collections
Region: United Kingdom Status: confirmed Documentation / provider ↗
This is a CONTENTdm deployment. CONTENTdm outside the US; demonstrates why platform discovery must be worldwide. Rights wording may be UK-specific and should not be reduced to a US-PD heuristic. Common IIIF patterns are repository /iiif/info/manifest.json, collection /iiif/info/{alias}/manifest.json and item /iiif/info/{alias}/{pointer}/manifest.json, with version/deployment variations. Engineering barriers include paginated collection manifests, compound objects, inconsistent rights metadata and occasional branded-domain/OCLC-host differences. GLAMWalk should checkpoint every page, validate JSON Content-Type/status, use exponential backoff with jitter for 429/5xx, and never interpret an HTML challenge page as JSON.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
40. CONTENTdm Demo Repository
Region: Global Status: confirmed Documentation / provider ↗
This is a CONTENTdm deployment. Use as a protocol test fixture for repository/collection/item manifest parsing. It is excellent for regression tests but should not be treated as a cultural-content source. Common IIIF patterns are repository /iiif/info/manifest.json, collection /iiif/info/{alias}/manifest.json and item /iiif/info/{alias}/{pointer}/manifest.json, with version/deployment variations. Engineering barriers include paginated collection manifests, compound objects, inconsistent rights metadata and occasional branded-domain/OCLC-host differences. GLAMWalk should checkpoint every page, validate JSON Content-Type/status, use exponential backoff with jitter for 429/5xx, and never interpret an HTML challenge page as JSON.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
41. National Gallery of Art Open Data
Region: United States Status: confirmed Documentation / provider ↗
NGA is especially useful because bulk/open-data workflows can be superior to hammering a transactional API. For large synchronization, prefer published datasets/dumps when available, then use web/IIIF services only for media enrichment. This reduces throttling, Cloudflare exposure and millions of tiny requests. Keep dataset version/date and object identifiers so incremental refreshes are auditable.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
42. Cleveland Museum of Art Open Access
Region: United States Status: confirmed Documentation / provider ↗
CMA's Open Access API is a structured museum-object source. Query metadata first and use the record's image/rights fields to determine eligible media. Hurdles are pagination, image derivative choices and assuming 'API' means every work is open. GLAMWalk should cache pages, preserve accession/object IDs, and route only explicitly eligible assets into distributable derivatives.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
43. Victoria and Albert Museum API
Region: United Kingdom Status: confirmed Documentation / provider ↗
The V&A API is valuable for object metadata and image discovery. UK museum rights and object copyright can differ from photograph/digital-image permissions, so rights parsing must remain explicit. Use API records instead of scraping collection HTML, keep system numbers/object identifiers, throttle politely and cache immutable records. Treat image delivery endpoints as media transport, not a license grant.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
44. Rijksmuseum
Region: Netherlands Status: research Documentation / provider ↗
Rijksmuseum has historically offered collection API access, but authentication/version/availability should be verified against current documentation before an automated run. This illustrates a core lesson: API lists age quickly. GLAMWalk marks adapters with last_verified and disables stale adapters rather than silently falling back to scraping. Rights/open-data status must be read from current records/policy.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
45. DPLA
Region: United States Status: confirmed Documentation / provider ↗
DPLA is an aggregator and therefore best used for discovery, not as a blanket rights authority. API records point to contributing hubs/institutions and may contain standardized rights statements. Hurdles are duplicates, partner redirects and heterogeneous media. GLAMWalk should preserve sourceResource/provider/isShownAt/rights data, deduplicate against direct-provider harvests and prefer original-provider media/IIIF when available.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.
46. Wellcome Collection
Region: United Kingdom Status: confirmed Documentation / provider ↗
Wellcome exposes structured collection APIs and many digitized assets, but our historical-human research showed why item-level rights matter: some Bertillon-related material is public-domain-marked while other cards/material remain in copyright. Never apply collection-topic rights wholesale. Use API identifiers, locations/digitalLocation metadata and explicit licenses, and segregate RESEARCH_ONLY material from PD/CC0 export candidates.
GLAMWalk operational rule: store raw source metadata, canonical IDs/URLs, retrieval timestamp, adapter version, HTTP status and rights evidence. Prefer incremental discovery/bulk data over exhaustive recrawls. On 403/CAPTCHA/Cloudflare challenges, stop rather than trying to evade the barrier; mark the provider for manual/API-key/bulk-route review.