Using the COPTR data feed

Accessing the COPTR data

Question:

• I'd like to take the data in COPTR and create a funky 3D spinny view that helps people navigate the data and find what they are looking for. Can I do this, and can I host it somewhere else?

Answer:

• Yes, that would be excellent! The COPTR content is made available under a CC-BY-SA license but we ask that if possible you point your users back to this site if they have corrections or new tools to add.

Ways to access the COPTR data

We offer two main channels for accessing the data - the MediaWiki API, and XML dumps of the MediaWiki content.

The COPTR API

As COPTR is based on MediaWiki, you can access the COPTR content via the MediaWiki API. The Official MediaWiki API Documentation is a pretty good starting point, and the endpoint itself is fairly self-documenting. You can also use the API sandbox to experiment with the API. However, to help get you started, here are a couple of examples.

Machine-Readable Recent Changes

If you want access to information on what has changed recently, just like the RecentChanges page, but in a machine-readable form, you can construct a URL like this:

https://coptr.digipres.org/api.php?action=query&list=recentchanges&format=json&rclimit=500&rctype=new&rcprop=title%7Cids%7Csizes%7Cflags%7Cuser%7Ctimestamp

Breaking down those query parameters, we have:

Parameter	Value	Meaning
action	query	Perform a query action, rather than e.g. adding or editing a page.
list	recentchanges	Perform a query on the 'recent changes' list (one of many lists available).
format	json	Return the results in JSON format.
rclimit	500	Return up to 500 results at once (500 is the maximum per request, but there are other parameters for paging).
rctype	new	Return only 'new page' events.
rcprop	title|ids|sizes|flags|user|timestamp	For each event, return these fields (see the API docs for more detail).

So, for example, if you would rather receive XML, you can just modify the format parameter.

Category Members

There is also a Category Members list you can query. For example:

• This query lists all the subcategories belonging to the Function category.
• This query lists the Page IDs and Title of all the pages belonging to the Fixity category.

Semantic API modules

COPTR runs the Semantic MediaWiki extension, which provides semantic API modules that can be used to query the COPTR data. For instance, the following URL uses the ask module to query for all tools that have function Rendering and content type Container, and retrieves the tools' purpose:

https://coptr.digipres.org/api.php?action=ask&query=[[Category:Tools]][[Function::Rendering]][[Content::Container]]|%3FPurpose

The following properties (some of which may be empty for some tools) can be queried

Property	Meaning
Purpose	A brief description of the tool's purpose
License	Information about the tool's license
Cost	Information about the cost of the tool
Platforms	A description of the platform(s) that the tool runs on
Input format	The input format(s) that the tool supports
Output format	The output format(s) that the tool supports
Function	The function(s) of the tool
Content	The tool's content type(s)

The Input and Output formats also have queryable properties

Property	Meaning
Wikidata ID	The identifier of the file format in Wikidata
File formats wiki ID	The identifier of the file format in the File formats wiki

You can use the Semantic search interface to construct and test a query that can subsequently be used with the ask API module. For more information about the query syntax and more advanced query constructs - such as property chaining and inverse queries - please refer to the Semantic MediaWiki documentation on selecting pages and displaying information.

The COPTR Database Backup Dumps

We also make daily XML dumps available, for backup and re-use purposes. You can find them in the backup-dumps folder,