NetarchiveSuite

From COPTR
Jump to: navigation, search


NetarchiveSuite
NetarchiveSuite is a web archiving software package designed to plan, schedule and run web harvests of parts of the Internet.
Homepage:http://netarchive.dk/suite/Welcome
License:GNU Lesser General Public License
Platforms:Linux


Contents

[edit] Description

The NetarchiveSuite is a web archiving software package designed to plan, schedule and run web harvests of parts of the Internet. It is also serves as an archiving platform for the collected materials, storing and performing consistency checks of the data.

[edit] Provider

The Royal Library of Denmark and the State and University Library of Denmark

[edit] Platform and interoperability

Netarchive requires a computer running a Linux operating system with Sun Java 1.6, as well as Java Messaging Service. The software uses Heritrix as its web crawler.

[edit] Functional notes

The NetarchiveSuite is split into four main modules: three modules corresponding to processes of harvesting, archiving and accessing materials, and one module to coordinate functions. The Harvester module can organise both snapshot and recurring harvests; it supports packaging metadata about the harvest together with the harvested data. The Archive module advertises bit consistency checks and the ability to support distributed batch jobs; it also supports storage. The Access Module uses a proxy solution to give access to the material. As the software uses Heritrix for its crawls, the materials collected take the form of arc files (not to be confused with ARC files).

[edit] Documentation and user support

Netarchive’s website includes extensive documentation, including an Overview and Quick Start Manual. Detailed guidance is found in the Configuration, Installation, and User Manuals. Developer guidance is found in the System Design and Additional Tools Manual. The website also points to a new Wiki that as of writing is in some places incomplete, and in general rather difficult to navigate. The project supports four mailing lists, all of which are currently active: -announce; -curator; -devel; and -users. The site also includes a contact page with email addresses for individuals at KB, SB, BnF, and ONB.

[edit] Usability

The Suite advertises its ability to be used by librarians rather than systems administrators; its Quickstart installation option is designed to take an hour, and the GUI for ongoing use is extremely simple.

[edit] Expertise required

With any web archiving project, deep understanding of the project’s scope and collections policy is essential in order to set up appropriate targets.

[edit] Standards compliance

No standards compliance is obviously advertised.

[edit] Influence and take-up

The Royal Library of Denmark and the State and University Library have used NetarchiveSuite to harvest the Danish world wide web since 2005. The software is also used by the Bibliothèque nationale de France, and Österreichische Nationalbibliothek.


[edit] User Experiences

[edit] Development Activity

Version 4.1 was released 27th May 2013. Version 4.2 should be released at the end of June 2013. The software appears to be an integral part of the Libraries’ ongoing web archiving effort, indicating continuing support. The website includes a roadmap for the software.

All development activity is visible on GitHub: http://github.com/netarchivesuite/netarchivesuite/commits


[edit] Release Feed

Below the last 3 release feeds:

2017-06-08 07:46:35
[tag:github.com,2008:Repository/19818273/netarchivesuite-5.3.1 netarchivesuite-5.3.1]
by csrster
2017-03-10 14:32:48
[tag:github.com,2008:Repository/19818273/netarchivesuite-5.3 netarchivesuite-5.3]
by csrster
2017-03-01 13:18:34
[tag:github.com,2008:Repository/19818273/5.3-RC1 5.3-RC1]
by csrster


[edit] Activity Feed

Below the last 5 commits:

2017-11-08 20:13:06
[tag:github.com,2008:Grit::Commit/334efe5e5af796980ab14428389a3fcc09137eb2 Modified test to assume domains file is already present locally.]
by csrster https://github.com/csrster
2017-11-08 16:47:12
[tag:github.com,2008:Grit::Commit/3a4b8afbdbc5284d9402299072e7c82be2374a40 Modified test to assume domains file is already present locally.]
by csrster https://github.com/csrster
2017-11-08 16:21:35
[tag:github.com,2008:Grit::Commit/33102f42c137be622e2fee4fb7f760257cc901f2 Merge remote-tracking branch 'origin/master']
by csrster https://github.com/csrster
2017-11-08 16:21:23
[tag:github.com,2008:Grit::Commit/703e2a26aa895d021254c8764c665e8a8e5364c6 Modified test to assume domains file is already present locally.]
by csrster https://github.com/csrster
2017-10-17 13:31:17
[tag:github.com,2008:Grit::Commit/698cfa3775f0356c2f19e713884b18b9ae1e2b8d Removed duplicate dependency 'javax.inject' in pom.xml]
by svcarlsen https://github.com/svcarlsen



Contributors

Chlara (25.0%), Nullhandle (10.4%), COPTR Bot (64.6%)