Web Archive Discovery - Revision history

Prwheatley at 14:34, 21 April 2021

2021-04-21T14:34:05Z

Rcdeboer: Web Archive redirects to Web

2021-04-20T10:34:05Z

Web Archive redirects to Web

MediaWiki default: /* Description */

2018-09-24T15:33:13Z

Description

MediaWiki default at 15:32, 24 September 2018

2018-09-24T15:32:28Z

MediaWiki default: /* Description */

2018-09-24T15:27:32Z

Description

MediaWiki default: Added activity feeds.

2018-09-24T14:16:07Z

Added activity feeds.

MediaWiki default at 15:29, 28 October 2014

2014-10-28T15:29:24Z

MediaWiki default: Added initial webarchive-discovery outline

2014-02-14T10:22:07Z

Added initial webarchive-discovery outline

New page

{{Infobox_tool
|purpose=Indexing and discovery tools for web archives.
|homepage=https://github.com/ukwa/webarchive-discovery
|license=Mixed
|platforms=Java
}}


[[Category:Metadata Extraction]]
[[Category:File Format Identification]]
[[Category:Content Profiling]]
[[Category:Discovery]]


[[Category:Web]]
[[Category:Web Archive]]

== Description ==

Full-text indexing system, using Apache Solr as the search back-end. Supports command-line and large-scale map-reduce (Hadoop) processing of ARC and WARC files. Also integrates file format analysis and scans for some known preservation risks.

== User Experiences ==

* Used by the [http://www.webarchive.org.uk/ UK Web Archive] to provide access to their collections. More details TBA.

== Development Activity ==



{{Infobox_tool_details
|ohloh_id=webarchive-discovery
}}

@@ Line 25: / Line 25: @@
 It runs format identification using both [[Apache Tika]] and [[DROID]], and also extracts filename extensions and the first few bytes of each resource, to enable analysis of formats unknown to those tools.
-If can also be configured to use [[Apache PDFBox Preflight]] to scan PDF's for PDF/A violations, and record them in the index for analysis.
+If can also be configured to use [[Apache PDFBox]]'s Preflight tool to scan PDF's for PDF/A violations, and record them in the index for analysis.
 It also records any parsing errors or other access problems it discovers, which can help find problematic resources.

@@ Line 1: / Line 1: @@
-<!-- Use the structure provided in this template, do not change it! -->
+{{Infobox tool
 |purpose=Indexing and discovery tools for web archives.
 |homepage=https://github.com/ukwa/webarchive-discovery
 |license=Mixed
 |platforms=Java
 }}
 == Description ==
 <!-- Describe the what the tool does, focusing on it's digital preservation value. Keep it factual. -->
@@ Line 49: / Line 41: @@
 Below the last 5 commits:
 <rss max=5>https://github.com/ukwa/webarchive-discovery/commits/master.atom</rss>

@@ Line 17: / Line 17: @@
 <!-- Add relevant categories to describe the content type that the tool addresses. Choose carefully, and view the list of existing categories first (see the Navigation sidebar on the left). If the tool works on any content type, do not add a category. The following are common category examples, remove those that don't apply -->
 [[Category:Web]]
 == Description ==

@@ Line 33: / Line 33: @@
 == User Experiences ==
 <!-- Add hotlinks to user experiences with the tool (eg. blog posts). These should illustrate the effectiveness (or otherwise) of the tool. -->
-* Used by the [http://www.webarchive.org.uk/ UK Web Archive] to provide access to their collections. More details TBA.
+* Used by the [http://www.webarchive.org.uk/ UK Web Archive] to provide access to their collections. Some details [http://blogs.bl.uk/webarchive/2017/06/revitalising-the-uk-web-archive.html in this blog post]. Feel free to contact [https://twitter.com/UKWebArchive @UKWebArchive] for more information.
 = Development Activity =