Difference between revisions of "Archivematica"
(→Activity Feed: remove error) |
Ross-spencer (talk | contribs) (Add METS as output to record) |
||
| (25 intermediate revisions by 7 users not shown) | |||
| Line 1: | Line 1: | ||
| − | {{ | + | {{Infobox tool |
|purpose=Archivematica is a digital preservation system that automates the process of preparing digital objects for ingest into a repository and an access system | |purpose=Archivematica is a digital preservation system that automates the process of preparing digital objects for ingest into a repository and an access system | ||
| − | |||
|homepage=https://www.archivematica.org | |homepage=https://www.archivematica.org | ||
|license=[http://www.gnu.org/licenses/agpl.html AGPL version 3] | |license=[http://www.gnu.org/licenses/agpl.html AGPL version 3] | ||
| − | | | + | |formats_in=PREMIS (Preservation Metadata Implementation Strategies) |
| + | |formats_out=METS (Metadata Encoding and Transmission Standard),PREMIS (Preservation Metadata Implementation Strategies) | ||
| + | |function=File Format Migration, Preservation System | ||
}} | }} | ||
| + | {{Infobox tool details}} | ||
| + | = Description = | ||
| + | Archivematica is a digital preservation system that automates the process of preparing digital objects for ingest into a repository and an access system, ingesting them into archival storage and providing access to the archived material as well as uploading access copies to an access system. The process is monitored and controlled through a Web-based dashboard that co-ordinates a suite of micro-services. It relies on normalisation with preservation as the original object and comprehensive PREMIS metadata in METS.xml as its primary preservation technique. | ||
| + | |||
| + | Any content that can be placed in a directory can be ingested, including images, text files, office documents, publications, audiovisual files, databases and research datasets. Archivematica performs digital preservation actions such as identifier assignment, checksum generation, virus scan, format identification and metadata extraction on all ingested objects. For certain types of files, specialized tools (such as ImageMagick for raster images, ffprobe and ffmpeg for AV files, Ghostscipt for PDF files, Inkscape for vector images, and bulk_extractor and sleuthkit for forensic disk images) are used for metadata extraction, format normalization and other preservation actions. New tools, rules and commands for preservation actions can be added to the workflow using the Format Policy Registry (FPR) via the Preservation Planning tab in the web-based dashboard. | ||
| + | |||
| + | Archivematica does not include a web crawler, although it can ingest warc (Web ARChive format) files. Archivematica can also ingest email archives and forensic disk images. | ||
| + | |||
| + | ==Provider== | ||
| + | |||
| + | This project is managed by [https://www.artefactual.com/ Artefactual Systems]. | ||
| − | + | Artefactual Systems was founded in January 2000 as a digital preservation consulting company, and expanded into open-source software development starting in 2006. It began in collaboration with the UNESCO Memory of the World's [http://goo.gl/tlRHbk Subcommittee on Technology] and the [https://vancouver.ca/your-government/city-of-vancouver-archives.aspx City of Vancouver Archives], but continues active development along with its partners at the [http://diginit.library.ubc.ca/ University of British Columbia Library], the [http://rockarch.org/ Rockefeller Archive Center], [http://www.sfu.ca/archives/ Simon Fraser University Archives and Records Management], [http://library.bentley.edu/ Bentley Historical Library] and a number of other collaborators. The company now offers software development, technical support and hosting services, data migrations (from legacy systems into AtoM), training and consulting. The company currently has more than 90 hosting/technical support clients and at any given time has around 12- to 15 software development, data migration and consulting clients. The company has 20+ personnel, a diverse group which includes archivists and librarians, software developers, systems administrators, project managers and technical services managers. | |
| − | [[ | ||
| − | [ | ||
| − | = | + | == Platform and interoperability == |
| − | Archivematica | + | |
| − | === | + | === Installation === |
| − | + | ||
| + | Archivematica may be installed directly on a Linux system. The following operating systems are supported: | ||
| + | |||
| + | * Ubuntu 14.04 64-bit Server Edition | ||
| + | * Ubuntu 16.04 64-bit Server Edition (beta) | ||
| + | * CentOS 7 64-bit | ||
| + | |||
| + | Other Linux distributions should work, but will require customization of these installation instructions. | ||
| + | |||
| + | Archivematica has a long list of software it depends on. They are installed when Archivematica is installed on a system. | ||
| + | |||
| + | Note that it is possible to install some of the components on separate machines in order to improve performance, such as: | ||
| + | |||
| + | * MySQL | ||
| + | * Gearman | ||
| + | * Elasticsearch (optional as of Archivematica 1.7) | ||
| + | |||
| + | === Interoperability === | ||
| + | |||
| + | Archivematica has out-of-the-box integrations with AtoM (AccessToMemory), Dspace, ArchivesSpace and CONTENTdm. Configuration and workflow instructions are available in the online user manual. Archivematica can also be used with Islandora, Dataverse and Samvera, but some customization may be required for setup. | ||
| + | |||
| + | === Storage === | ||
| + | Types of storage systems used with Archivematica include local file-based systems; cloud-based storage such as Amazon S3, Microsoft Azure and OpenStack Swift; and specialized storage tools and services such as LOCKSS, DuraCloud and Arkivum. Artefactual in partnership with DuraSpace offers [https://duraspace.org/archivesdirect/ ArchivesDirect] as an Archivematica and DuraCloud hosting option. | ||
| + | |||
| + | Users can ingest content manually via the transfer tab or can use automation tools to automatically ingest content from designated source locations such as a folder on a network drive or from systems such as DSpace, Islandora and Dataverse. | ||
| + | |||
| + | === Access === | ||
| + | |||
| + | Dissemination Information Packages to access systems such as AtoM or ArchivesSpace, where rich metadata enhancement can be undertaken for discoverability and access purposes. | ||
| + | |||
| + | === Functional notes === | ||
| − | |||
| − | |||
| − | |||
| − | |||
Archivematica uses a micro-services approach, which means it acts as a wrapper for many task-specific applications such as the BagIt library, Clam Anti-Virus, DigiKam, FFmpeg, FITS (File Information Tool Set), ImageMagick, Inkscape, OpenOffice.org, and 7-Zip. | Archivematica uses a micro-services approach, which means it acts as a wrapper for many task-specific applications such as the BagIt library, Clam Anti-Virus, DigiKam, FFmpeg, FITS (File Information Tool Set), ImageMagick, Inkscape, OpenOffice.org, and 7-Zip. | ||
The typical workflow is for the curator to assemble a transfer package in the filesystem: a script is provided for setting up the right folder structure or the structure can be assembled manually for some workflows, then digital objects are added to one folder and contextual information (submission documentation in the form of e.g. transfer forms, donation agreements) to another. The package is moved to an input folder 'watched' by the main Archivematica Web tool. Through the Web interface, the curator can decide to accept or reject the transfer. If the transfer is accepted, the tool performs an initial analysis – calculating checksums, assigning UUIDs, scanning for viruses, identifying formats, extracting metadata – and then offers to create a Submission Information Package (SIP); it is also possible to create one or more SIPs manually. Metadata (simple Dublin Core and PREMIS 2.2 rights/restrictions) can then be added to the SIP before it is ingested. At ingest, the curator can choose various routes such as Preservation (where the digital objects are normalised to archival formats and transformed into an Archival Information Package, or AIP), Access (where the digital objects are normalised to dissemination formats and transformed into a Dissemination Information Package, or DIP), repackaging without normalisation, or many combinations of the aforementioned. Further functions are provided for moving AIPs into archival storage and uploading DIPs to AtoM or another access portal. Workflows and decision points are configurable via preconfiguration settings in the administration tab of the web-based dashboard. | The typical workflow is for the curator to assemble a transfer package in the filesystem: a script is provided for setting up the right folder structure or the structure can be assembled manually for some workflows, then digital objects are added to one folder and contextual information (submission documentation in the form of e.g. transfer forms, donation agreements) to another. The package is moved to an input folder 'watched' by the main Archivematica Web tool. Through the Web interface, the curator can decide to accept or reject the transfer. If the transfer is accepted, the tool performs an initial analysis – calculating checksums, assigning UUIDs, scanning for viruses, identifying formats, extracting metadata – and then offers to create a Submission Information Package (SIP); it is also possible to create one or more SIPs manually. Metadata (simple Dublin Core and PREMIS 2.2 rights/restrictions) can then be added to the SIP before it is ingested. At ingest, the curator can choose various routes such as Preservation (where the digital objects are normalised to archival formats and transformed into an Archival Information Package, or AIP), Access (where the digital objects are normalised to dissemination formats and transformed into a Dissemination Information Package, or DIP), repackaging without normalisation, or many combinations of the aforementioned. Further functions are provided for moving AIPs into archival storage and uploading DIPs to AtoM or another access portal. Workflows and decision points are configurable via preconfiguration settings in the administration tab of the web-based dashboard. | ||
| − | |||
| − | |||
| − | |||
| − | ====Usability== | + | = Documentation, community, and support = |
| − | The majority of operations are accomplished through a | + | The online [https://www.archivematica.org/en/ documentation] for Archivematica includes a User and an Administrative Manual. |
| − | + | ||
| + | The [https://wiki.archivematica.org/Main_Page project wiki] provides, screencasts, requirements specifications (including use cases, activity diagrams, recognised significant properties of various media and media preservation plans) and a description of the technical architecture. | ||
| + | |||
| + | Community support is available through the [http://groups.google.com/group/archivematica Archivematica Discussion Group]. Several [https://wiki.archivematica.org/Community/Regional_User_Groups region-based user groups] exist. | ||
| + | |||
| + | [https://wiki.archivematica.org/Community/Camps Archivematica Camps] are intended to provide a space for anyone interested in or currently using Archivematica, to come together, learn about the platform and share their experiences. | ||
| + | |||
| + | Sample data for testing Archivematica is [https://github.com/artefactual/archivematica-sampledata available online] and when Archivematica is installed. | ||
| + | |||
| + | Artefactual Systems, Inc., the primary developer of Archivematica, also offers [https://www.artefactual.com/services/ support options]. | ||
| + | |||
| + | Digital POWRR offers [http://www.sustainableheritagenetwork.org/system/files/atoms/file/Tutorial_for_Archivematica_POWRR_WarmSprings.pdf a How-to Tech Tutorial] (PDF). | ||
| + | |||
| + | Archivist Ethan Gates offers a presentation entitled [https://ethan-gates.com/files/Archivematica_Workshop_2017.pdf A Place Where You Process: An Introduction to Archivematica Workflows] (slides,PDF). | ||
| + | |||
| + | ==Usability== | ||
| + | |||
| + | The majority of operations are accomplished through a Web-based graphical user interface. | ||
| + | |||
| + | Reports on the ease of installation and the robustness of the system are mixed but improving. Example experiences of installing Archivematica (note that current version is 1.7): | ||
| + | * [http://libraryblogs.is.ed.ac.uk/bitsandpieces/2016/01/13/archivematica-youre-the-one-that-i-want/ Edinburgh University] with version 1.3 | ||
| + | * [http://digital-archiving.blogspot.co.uk/2012/09/installing-archivematica-and-running.html Jenny Mitcham] with version 0.9 | ||
| + | * [http://larchivista.blogspot.co.uk/2011/04/installing-archivematica.html Bonnie Weddle] with version 0.7 | ||
| + | * [http://e-records.chrisprom.com/evaluating-open-source-digital-preservation-systems-a-case-study-2/ Angela Jordan] with version 0.7 | ||
| + | |||
| + | ==Expertise required== | ||
| + | |||
The system is easy to use, though as it draws heavily on the [http://www.dcc.ac.uk/resources/briefing-papers/introduction-curation/using-oais-curation OAIS Reference Model] some familiarity with that model is needed to understand the workflows Archivematica supports. When installing directly on a Linux desktop or server, even if it is deployed in a virtual machine, a little technical expertise is required (e.g. for setting up ports correctly). | The system is easy to use, though as it draws heavily on the [http://www.dcc.ac.uk/resources/briefing-papers/introduction-curation/using-oais-curation OAIS Reference Model] some familiarity with that model is needed to understand the workflows Archivematica supports. When installing directly on a Linux desktop or server, even if it is deployed in a virtual machine, a little technical expertise is required (e.g. for setting up ports correctly). | ||
| − | |||
| − | |||
====Influence and take-up==== | ====Influence and take-up==== | ||
Archivematica is used by at least [https://www.archivematica.org/wiki/Community 30 organisations]. | Archivematica is used by at least [https://www.archivematica.org/wiki/Community 30 organisations]. | ||
| − | = | + | =Standards compliance= |
| + | The functionality of Archivematica is clearly based on that defined by the OAIS Reference Model. The Archival Information Packages generated by the system use the [[BagIt]] packaging format, in conjunction with a METS packaging manifest incorporating PREMIS metadata. | ||
= Development Activity = | = Development Activity = | ||
<!-- Provide *evidence* of development activity of the tool. For example, RSS feeds for code issues or commits. --> | <!-- Provide *evidence* of development activity of the tool. For example, RSS feeds for code issues or commits. --> | ||
All development activity is visible on GitHub: http://github.com/artefactual/archivematica/commits | All development activity is visible on GitHub: http://github.com/artefactual/archivematica/commits | ||
| − | + | ||
| − | + | Documentation and information on the latest Archivematica release is available here: https://www.archivematica.org/en/docs/latest | |
| − | |||
| − | |||
| − | |||
Latest revision as of 15:54, 26 November 2021
Description[edit]
Archivematica is a digital preservation system that automates the process of preparing digital objects for ingest into a repository and an access system, ingesting them into archival storage and providing access to the archived material as well as uploading access copies to an access system. The process is monitored and controlled through a Web-based dashboard that co-ordinates a suite of micro-services. It relies on normalisation with preservation as the original object and comprehensive PREMIS metadata in METS.xml as its primary preservation technique.
Any content that can be placed in a directory can be ingested, including images, text files, office documents, publications, audiovisual files, databases and research datasets. Archivematica performs digital preservation actions such as identifier assignment, checksum generation, virus scan, format identification and metadata extraction on all ingested objects. For certain types of files, specialized tools (such as ImageMagick for raster images, ffprobe and ffmpeg for AV files, Ghostscipt for PDF files, Inkscape for vector images, and bulk_extractor and sleuthkit for forensic disk images) are used for metadata extraction, format normalization and other preservation actions. New tools, rules and commands for preservation actions can be added to the workflow using the Format Policy Registry (FPR) via the Preservation Planning tab in the web-based dashboard.
Archivematica does not include a web crawler, although it can ingest warc (Web ARChive format) files. Archivematica can also ingest email archives and forensic disk images.
Provider[edit]
This project is managed by Artefactual Systems.
Artefactual Systems was founded in January 2000 as a digital preservation consulting company, and expanded into open-source software development starting in 2006. It began in collaboration with the UNESCO Memory of the World's Subcommittee on Technology and the City of Vancouver Archives, but continues active development along with its partners at the University of British Columbia Library, the Rockefeller Archive Center, Simon Fraser University Archives and Records Management, Bentley Historical Library and a number of other collaborators. The company now offers software development, technical support and hosting services, data migrations (from legacy systems into AtoM), training and consulting. The company currently has more than 90 hosting/technical support clients and at any given time has around 12- to 15 software development, data migration and consulting clients. The company has 20+ personnel, a diverse group which includes archivists and librarians, software developers, systems administrators, project managers and technical services managers.
Platform and interoperability[edit]
Installation[edit]
Archivematica may be installed directly on a Linux system. The following operating systems are supported:
- Ubuntu 14.04 64-bit Server Edition
- Ubuntu 16.04 64-bit Server Edition (beta)
- CentOS 7 64-bit
Other Linux distributions should work, but will require customization of these installation instructions.
Archivematica has a long list of software it depends on. They are installed when Archivematica is installed on a system.
Note that it is possible to install some of the components on separate machines in order to improve performance, such as:
- MySQL
- Gearman
- Elasticsearch (optional as of Archivematica 1.7)
Interoperability[edit]
Archivematica has out-of-the-box integrations with AtoM (AccessToMemory), Dspace, ArchivesSpace and CONTENTdm. Configuration and workflow instructions are available in the online user manual. Archivematica can also be used with Islandora, Dataverse and Samvera, but some customization may be required for setup.
Storage[edit]
Types of storage systems used with Archivematica include local file-based systems; cloud-based storage such as Amazon S3, Microsoft Azure and OpenStack Swift; and specialized storage tools and services such as LOCKSS, DuraCloud and Arkivum. Artefactual in partnership with DuraSpace offers ArchivesDirect as an Archivematica and DuraCloud hosting option.
Users can ingest content manually via the transfer tab or can use automation tools to automatically ingest content from designated source locations such as a folder on a network drive or from systems such as DSpace, Islandora and Dataverse.
Access[edit]
Dissemination Information Packages to access systems such as AtoM or ArchivesSpace, where rich metadata enhancement can be undertaken for discoverability and access purposes.
Functional notes[edit]
Archivematica uses a micro-services approach, which means it acts as a wrapper for many task-specific applications such as the BagIt library, Clam Anti-Virus, DigiKam, FFmpeg, FITS (File Information Tool Set), ImageMagick, Inkscape, OpenOffice.org, and 7-Zip. The typical workflow is for the curator to assemble a transfer package in the filesystem: a script is provided for setting up the right folder structure or the structure can be assembled manually for some workflows, then digital objects are added to one folder and contextual information (submission documentation in the form of e.g. transfer forms, donation agreements) to another. The package is moved to an input folder 'watched' by the main Archivematica Web tool. Through the Web interface, the curator can decide to accept or reject the transfer. If the transfer is accepted, the tool performs an initial analysis – calculating checksums, assigning UUIDs, scanning for viruses, identifying formats, extracting metadata – and then offers to create a Submission Information Package (SIP); it is also possible to create one or more SIPs manually. Metadata (simple Dublin Core and PREMIS 2.2 rights/restrictions) can then be added to the SIP before it is ingested. At ingest, the curator can choose various routes such as Preservation (where the digital objects are normalised to archival formats and transformed into an Archival Information Package, or AIP), Access (where the digital objects are normalised to dissemination formats and transformed into a Dissemination Information Package, or DIP), repackaging without normalisation, or many combinations of the aforementioned. Further functions are provided for moving AIPs into archival storage and uploading DIPs to AtoM or another access portal. Workflows and decision points are configurable via preconfiguration settings in the administration tab of the web-based dashboard.
Documentation, community, and support[edit]
The online documentation for Archivematica includes a User and an Administrative Manual.
The project wiki provides, screencasts, requirements specifications (including use cases, activity diagrams, recognised significant properties of various media and media preservation plans) and a description of the technical architecture.
Community support is available through the Archivematica Discussion Group. Several region-based user groups exist.
Archivematica Camps are intended to provide a space for anyone interested in or currently using Archivematica, to come together, learn about the platform and share their experiences.
Sample data for testing Archivematica is available online and when Archivematica is installed.
Artefactual Systems, Inc., the primary developer of Archivematica, also offers support options.
Digital POWRR offers a How-to Tech Tutorial (PDF).
Archivist Ethan Gates offers a presentation entitled A Place Where You Process: An Introduction to Archivematica Workflows (slides,PDF).
Usability[edit]
The majority of operations are accomplished through a Web-based graphical user interface.
Reports on the ease of installation and the robustness of the system are mixed but improving. Example experiences of installing Archivematica (note that current version is 1.7):
- Edinburgh University with version 1.3
- Jenny Mitcham with version 0.9
- Bonnie Weddle with version 0.7
- Angela Jordan with version 0.7
Expertise required[edit]
The system is easy to use, though as it draws heavily on the OAIS Reference Model some familiarity with that model is needed to understand the workflows Archivematica supports. When installing directly on a Linux desktop or server, even if it is deployed in a virtual machine, a little technical expertise is required (e.g. for setting up ports correctly).
Influence and take-up[edit]
Archivematica is used by at least 30 organisations.
Standards compliance[edit]
The functionality of Archivematica is clearly based on that defined by the OAIS Reference Model. The Archival Information Packages generated by the system use the BagIt packaging format, in conjunction with a METS packaging manifest incorporating PREMIS metadata.
Development Activity[edit]
All development activity is visible on GitHub: http://github.com/artefactual/archivematica/commits
Documentation and information on the latest Archivematica release is available here: https://www.archivematica.org/en/docs/latest