Warctools - Revision history

Rcdeboer at 21:33, 25 May 2021

2021-05-25T21:33:55Z

Prwheatley at 14:19, 21 April 2021

2021-04-21T14:19:15Z

MediaWiki default at 15:30, 28 October 2014

2014-10-28T15:30:51Z

MediaWiki default: Added ARC to WARC note.

2014-07-02T10:47:54Z

Added ARC to WARC note.

MediaWiki default: Hooked to formats wiki.

2014-07-01T12:09:10Z

Hooked to formats wiki.

MediaWiki default: Fixed formats linkage.

2014-07-01T11:56:09Z

Fixed formats linkage.

MediaWiki default: Includes summary of other Python codebases.

2014-07-01T11:54:57Z

Includes summary of other Python codebases.

New page

{{Infobox_tool
|purpose=Command line tools and libraries for handling and manipulating WARC files (and HTTP contents)
|homepage=https://pypi.python.org/pypi/warctools/
|sourcecode=https://github.com/internetarchive/warctools/
|license=MIT License
|platforms=Cross-platform
|language=Python
|formats_in=[[formats:WARC|WARC]], [[formats:ARC|ARC]]
|formats_out=[[formats:WARC|WARC]]
}}


[[Category:Metadata Extraction]]
[[Category:Validation]]


[[Category:Web Archive]]

== Description ==


This is the most current and well-maintained Python codebase for working with WARC files. It provides a number of command-line tools for common WARC/ARC operations, and can also act as a library to create or work with WARC files directly from Python.

Pull requests and releases are currently managed by [https://twitter.com/tef Thomas Figg], who can be contacted via Twitter.

=== Older Python WARC Implementations ===

This codebase was initially funded by IIPC and developed by Hanzo Archives. This lead to the [https://pypi.python.org/pypi/hanzo-warc-tools hanzo-warc-tools package] and [https://bitbucket.org/hanzo/warc-tools source code].

There is also a separate [https://pypi.python.org/pypi/warc/ warc package] that was created by the Internet Archive (see
[https://github.com/internetarchive/warc source code]), but is no longer in use.

Both of these projects are defunct and are now superseded by the internetarchive/warctools project.

== User Experiences ==


== Development Activity ==


=== Releases ===
<rss max=5>https://github.com/internetarchive/warctools/releases.atom</rss>

=== Development ===

{{Infobox_tool_details
|ohloh_id=warctools
}}

<rss max=5>https://github.com/internetarchive/warctools/commits/master.atom</rss>

@@ Line 6: / Line 6: @@
 |platforms=Cross-platform
 |language=Python
-|formats_in={{Format|WARC}}, {{Format|ARC (Internet Archive)}}
+|formats_in=WARC, ARC (Internet Archive)
-|formats_out={{Format|WARC}}
+|formats_out=WARC
-|function=Metadata Extraction, Validation, File Format Migration
+|function=File Format Migration, Metadata Extraction, Validation
 |content=Web
 }}

@@ Line 8: / Line 8: @@
 |platforms=Cross-platform
 |language=Python
-|formats_in=[[formats:WARC|WARC]], [[formats:ARC (Internet Archive)|ARC]]
+|formats_in={{Format|WARC}}, {{Format|ARC (Internet Archive)}}
-|formats_out=[[formats:WARC|WARC]]
+|formats_out={{Format|WARC}}
 }}

@@ Line 1: / Line 1: @@
-<!-- Use the structure provided in this template, do not change it! -->
+{{Infobox tool
 |purpose=Command line tools and libraries for handling and manipulating WARC files (and HTTP contents)
 |homepage=https://pypi.python.org/pypi/warctools/
 |sourcecode=https://github.com/internetarchive/warctools/
 |license=MIT License
 |platforms=Cross-platform
@@ Line 10: / Line 8: @@
 |formats_in={{Format|WARC}}, {{Format|ARC (Internet Archive)}}
 |formats_out={{Format|WARC}}
 }}
 == Description ==
 <!-- Describe the what the tool does, focusing on it's digital preservation value. Keep it factual. -->
@@ Line 47: / Line 41: @@
 === Development ===
 <!-- Add the Ohloh.com ID for the tool, if known. -->
-{{Infobox_tool_details
 <rss max=5>https://github.com/internetarchive/warctools/commits/master.atom</rss>

@@ Line 18: / Line 18: @@
 <!-- Add relevant categories to describe the content type that the tool addresses. Choose carefully, and view the list of existing categories first (see the Navigation sidebar on the left). If the tool works on any content type, do not add a category. The following are common category examples, remove those that don't apply -->
-[[Category:Web Archive]]
+[[Category:Web]]
 == Description ==

@@ Line 15: / Line 15: @@
 [[Category:Metadata Extraction]]
 [[Category:Validation]]
 <!-- Add relevant categories to describe the content type that the tool addresses. Choose carefully, and view the list of existing categories first (see the Navigation sidebar on the left). If the tool works on any content type, do not add a category. The following are common category examples, remove those that don't apply -->