Difference between revisions of "NetarchiveSuite"
(Trial import from script.) |
Nullhandle (talk | contribs) |
||
| Line 1: | Line 1: | ||
{{Infobox_tool | {{Infobox_tool | ||
| − | |purpose= | + | |purpose=NetarchiveSuite is a web archiving software package designed to plan, schedule and run web harvests of parts of the Internet. |
| − | |image= | + | |image=NAS.gif |
|homepage=http://netarchive.dk/suite/Welcome | |homepage=http://netarchive.dk/suite/Welcome | ||
| − | |license= | + | |license=[http://www.gnu.org/licenses/lgpl.html#translations GNU Lesser General Public License] |
|platforms= | |platforms= | ||
}} | }} | ||
| Line 13: | Line 13: | ||
= Description = | = Description = | ||
| − | The [https://sbforge.org/display/NAS/NetarchiveSuite | + | The [https://sbforge.org/display/NAS/NetarchiveSuite NetarchiveSuite] is a web archiving software package designed to plan, schedule and run web harvests of parts of the Internet. It is also serves as an archiving platform for the collected materials, storing and performing consistency checks of the data. |
====Provider==== | ====Provider==== | ||
The Royal Library of Denmark and the State and University Library of Denmark | The Royal Library of Denmark and the State and University Library of Denmark | ||
====Licensing and cost==== | ====Licensing and cost==== | ||
| − | [http://www.gnu.org/licenses/lgpl.html#translations GNU Lesser General Public License ]– free. | + | [http://www.gnu.org/licenses/lgpl.html#translations GNU Lesser General Public License] – free. |
====Development activity==== | ====Development activity==== | ||
Version 4.1 was released 27th May 2013. Version 4.2 should be released at the end of June 2013. | Version 4.1 was released 27th May 2013. Version 4.2 should be released at the end of June 2013. | ||
| − | The software appears to be an integral part of the Libraries’ ongoing web archiving effort, indicating continuing support. | + | The software appears to be an integral part of the Libraries’ ongoing web archiving effort, indicating continuing support. The website includes a roadmap for the software. |
====Platform and interoperability==== | ====Platform and interoperability==== | ||
| − | Netarchive requires a computer running a Linux operating system with Sun Java 1.6, as well as Java Messaging Service. | + | Netarchive requires a computer running a Linux operating system with Sun Java 1.6, as well as Java Messaging Service. The software uses Heritrix as its web crawler. |
====Functional notes==== | ====Functional notes==== | ||
| − | The | + | The NetarchiveSuite is split into four main modules: three modules corresponding to processes of harvesting, archiving and accessing materials, and one module to coordinate functions. The Harvester module can organise both snapshot and recurring harvests; it supports packaging metadata about the harvest together with the harvested data. The Archive module advertises bit consistency checks and the ability to support distributed batch jobs; it also supports storage. The Access Module uses a proxy solution to give access to the material. As the software uses Heritrix for its crawls, the materials collected take the form of arc files (not to be confused with ARC files). |
| − | The Harvester module can organise both snapshot and recurring harvests; it supports packaging metadata about the harvest together with the harvested data. | ||
| − | The Archive module advertises bit consistency checks and the ability to support distributed batch jobs; it also supports storage. | ||
| − | The Access Module uses a proxy solution to give access to the material. | ||
| − | As the software uses | ||
====Documentation and user support==== | ====Documentation and user support==== | ||
| − | Netarchive’s website includes extensive [https://sbforge.org/display/NAS/Documentation documentation], including an Overview and Quick Start Manual. | + | Netarchive’s website includes extensive [https://sbforge.org/display/NAS/Documentation documentation], including an Overview and Quick Start Manual. Detailed guidance is found in the Configuration, Installation, and User Manuals. Developer guidance is found in the System Design and Additional Tools Manual. The website also points to a new Wiki that as of writing is in some places incomplete, and in general rather difficult to navigate. The project supports four mailing lists, all of which are currently active: -announce; -curator; -devel; and -users. The site also includes a contact page with email addresses for individuals at KB, SB, BnF, and ONB. |
| − | The project supports four mailing lists, all of which are currently active: -announce; -curator; -devel; and -users. | ||
====Usability==== | ====Usability==== | ||
The Suite advertises its ability to be used by librarians rather than systems administrators; its Quickstart installation option is designed to take an hour, and the GUI for ongoing use is extremely simple. | The Suite advertises its ability to be used by librarians rather than systems administrators; its Quickstart installation option is designed to take an hour, and the GUI for ongoing use is extremely simple. | ||
| Line 40: | Line 35: | ||
No standards compliance is obviously advertised. | No standards compliance is obviously advertised. | ||
====Influence and take-up==== | ====Influence and take-up==== | ||
| − | The Royal Library of Denmark and the State and University Library have used | + | The Royal Library of Denmark and the State and University Library have used NetarchiveSuite to harvest the Danish world wide web since 2005. The software is also used by the Bibliothèque nationale de France, and Österreichische Nationalbibliothek. |
| Line 49: | Line 44: | ||
{{Infobox_tool_details | {{Infobox_tool_details | ||
| + | |releases_rss=https://github.com/netarchivesuite/netarchivesuite/commits/master.atom | ||
|ohloh_id=NetarchiveSuite | |ohloh_id=NetarchiveSuite | ||
}} | }} | ||
Revision as of 17:06, 2 March 2016
Description
The NetarchiveSuite is a web archiving software package designed to plan, schedule and run web harvests of parts of the Internet. It is also serves as an archiving platform for the collected materials, storing and performing consistency checks of the data.
Provider
The Royal Library of Denmark and the State and University Library of Denmark
Licensing and cost
GNU Lesser General Public License – free.
Development activity
Version 4.1 was released 27th May 2013. Version 4.2 should be released at the end of June 2013. The software appears to be an integral part of the Libraries’ ongoing web archiving effort, indicating continuing support. The website includes a roadmap for the software.
Platform and interoperability
Netarchive requires a computer running a Linux operating system with Sun Java 1.6, as well as Java Messaging Service. The software uses Heritrix as its web crawler.
Functional notes
The NetarchiveSuite is split into four main modules: three modules corresponding to processes of harvesting, archiving and accessing materials, and one module to coordinate functions. The Harvester module can organise both snapshot and recurring harvests; it supports packaging metadata about the harvest together with the harvested data. The Archive module advertises bit consistency checks and the ability to support distributed batch jobs; it also supports storage. The Access Module uses a proxy solution to give access to the material. As the software uses Heritrix for its crawls, the materials collected take the form of arc files (not to be confused with ARC files).
Documentation and user support
Netarchive’s website includes extensive documentation, including an Overview and Quick Start Manual. Detailed guidance is found in the Configuration, Installation, and User Manuals. Developer guidance is found in the System Design and Additional Tools Manual. The website also points to a new Wiki that as of writing is in some places incomplete, and in general rather difficult to navigate. The project supports four mailing lists, all of which are currently active: -announce; -curator; -devel; and -users. The site also includes a contact page with email addresses for individuals at KB, SB, BnF, and ONB.
Usability
The Suite advertises its ability to be used by librarians rather than systems administrators; its Quickstart installation option is designed to take an hour, and the GUI for ongoing use is extremely simple.
Expertise required
With any web archiving project, deep understanding of the project’s scope and collections policy is essential in order to set up appropriate targets.
Standards compliance
No standards compliance is obviously advertised.
Influence and take-up
The Royal Library of Denmark and the State and University Library have used NetarchiveSuite to harvest the Danish world wide web since 2005. The software is also used by the Bibliothèque nationale de France, and Österreichische Nationalbibliothek.
User Experiences
Development Activity
Release Feed
- 2023-12-07 12:58:46
- [tag:github.com,2008:Grit::Commit/d4dba40d1f9c6046e921105d9d1f7e9e5dca31e0 Added a flag to allow upload failures without causing a job to be mar…]
- by csrster https://github.com/csrster
- 2023-10-05 09:23:56
- [tag:github.com,2008:Grit::Commit/630eb935e6adbe10b6ad1934dfeaf7f64f3510d0 Added correct behaviour for failed cdx/wayback indexing for hadoop jobs.]
- by csrster https://github.com/csrster
- 2023-09-07 10:23:45
- [tag:github.com,2008:Grit::Commit/1eb19c31aefc3e4b99cb76e4d7429abc70bdd79e [maven-release-plugin] prepare for next development iteration]
- by csrster https://github.com/csrster
- 2023-09-07 10:23:41
- [tag:github.com,2008:Grit::Commit/2b944a5f0e29712688320fa810b169f37dc04551 [maven-release-plugin] prepare release netarchivesuite-7.5]
- by csrster https://github.com/csrster
- 2023-09-07 10:18:23
- [tag:github.com,2008:Grit::Commit/41e47add9592ffd781bd97b46d8688865704b1d4 Update heritrix version in Constants class]
- by csrster https://github.com/csrster