Difference between revisions of "Libsafe"

From COPTR
Jump to navigation Jump to search
m
 
(12 intermediate revisions by 5 users not shown)
Line 1: Line 1:
<!-- Use the structure provided in this template, do not change it! -->
+
{{Infobox tool
 
+
|purpose=libsafe allows the organizations to create a full OAIS compliant Archive, including active and passive digital preservation workflows and is particularly suited for master image files of digitizing processes.
{{Infobox_tool
 
|purpose=libsafe allows to the organizations to create a full OAIS compliant Archive, including active and passive digital preservation workflows and is particularly suited for master image files of digitizing processes.
 
|image={{PAGENAMEE}}.png
 
 
|homepage=http://www.preservaciondigital.es
 
|homepage=http://www.preservaciondigital.es
 +
|formats_in=METS (Metadata Encoding and Transmission Standard)
 +
|formats_out=METS (Metadata Encoding and Transmission Standard)
 +
|function=Preservation System
 +
|content=Audio, Document, Image, Video, Web
 
}}
 
}}
<!-- Note that to use the image field, you should leave the value as {{PAGENAMEE}}.png (or similar) and upload a copy of the image. Hot-linking is not supported. If you don't want an image, just remove that line. -->
+
{{Infobox tool details}}
 
+
<!-- Use the structure provided in this template, do not change it! -->
<!-- Add one or more categories to describe the function of the tool, such as: -->
 
[[Category:Preservation System]]
 
[[Category:Audio]]
 
[[Category:Document]]
 
[[Category:Image]]
 
[[Category:Video]]
 
[[Category:Web Archive]]
 
  
  
 +
<!-- Note that to use the image field, you should leave the value as {{PAGENAMEE}}.png (or similar) and upload a copy of the image. Hot-linking is not supported. If you don't want an image, just remove that line. -->
  
 
== Description ==
 
== Description ==
 
<!-- Describe the what the tool does, focusing on it's digital preservation value. Keep it factual. -->
 
<!-- Describe the what the tool does, focusing on it's digital preservation value. Keep it factual. -->
libsafe allows to the organizations to create a full OAIS compliant Archive, including active and passive digital preservation workflows and is particularly suited for master image files of digitizing processes. It's used by large institutions like the National Library of Spain or the Royal Spanish Language Academy but also by smaller organizations like universities or hospitals.
+
LIBSAFE Advanced allows the organizations to create a full OAIS aligned Archive, including active and passive digital preservation workflows and is particularly suited for master image files of digitizing processes. It's used by large institutions like the British Library, the National Library of Spain or the United States Holocaust Memorial Museum but also by smaller organizations like universities or hospitals.
 +
 
 +
It allows to manage high volume data archives with ease, having deployments of 6PB in size.
  
It allows to manage high volume data archives with ease, having deployments of 2Pb of capacity.
+
More information: https://www.libnova.com
  
More information and schenshoots are available on http://www.preservaciondigital.es (in spanish) and http://www.digitalpreservationsoftware.com (in english).
 
  
 
Key features and benefits:
 
Key features and benefits:
 
* Easy to use web interface
 
* Easy to use web interface
 
* Integrated catalog
 
* Integrated catalog
* Passive preservation (content disseminating, periodic storage hash auditing, etc)
+
* Full preservation lifecycle management
* Active preservation that allows evolving file formats but also metadata schemas
 
* Day one full export
 
 
* Object versioning
 
* Object versioning
* Powerfull metadata engine (versioning of the metadata, unique metadata and obligatory metadata) over any schema. Loaded with DC, MAC21, EAD and ISAD.G. Allows creating new schemas and migrating one to another. Allows sync to other tools. libsafe allows any metadata schema to be defined and used.
+
* Powerful metadata engine (versioning of the metadata, unique metadata and mandatory metatadata) over any schema. Full METS and PREMIS aligment. Allows creating new schemas and migrating one to another. Allows sync to other tools. Fully configurable metadata schemas.
 
* REST API for ingestion, catalog search and object retrieve.
 
* REST API for ingestion, catalog search and object retrieve.
* Full WORM, the software never deletes anything. The user can never delete once ingested.
+
* Works with any SMB, NFS or Object based storage.
* Works with any storage mountable with CIFS.
+
* Microservices: It's possible to expand various functions easily with plug-ins.
* plug-in architecture: It's possible to expand various functions easily with plug-ins.
+
* Linear scalability: It is possible to scale the platform to achieve any needed performance.
  
  
Line 48: Line 42:
 
* Exploring: How metadata should be extracted.
 
* Exploring: How metadata should be extracted.
 
* Dissemination: how many copies and in wich "storage pools" should the information be stored. libsafe does not destroy the folder/file structure or filenames.
 
* Dissemination: how many copies and in wich "storage pools" should the information be stored. libsafe does not destroy the folder/file structure or filenames.
* Auditing policy: Between how many days the object should be audited by the system in background.
+
* Auditing policy: How often the object should be audited by the system in background.
  
 
The whole system is "job" based. Every process is a job (ingestion job, auditing job, migration job, etc).
 
The whole system is "job" based. Every process is a job (ingestion job, auditing job, migration job, etc).
Line 97: Line 91:
 
==== Active preservation ====
 
==== Active preservation ====
 
Including options to evolve not only file formats but also metadata schemas.
 
Including options to evolve not only file formats but also metadata schemas.
 
  
 
== User Experiences ==
 
== User Experiences ==
 
<!-- Add hotlinks to user experiences with the tool (eg. blog posts). These should illustrate the effectiveness (or otherwise) of the tool. Use a bullet list. -->
 
<!-- Add hotlinks to user experiences with the tool (eg. blog posts). These should illustrate the effectiveness (or otherwise) of the tool. Use a bullet list. -->
Please go to http://www.preservaciondigital.es (in spanish) and http://www.digitalpreservationsoftware.com (in english) for customer stories.
+
https://www.libnova.com
  
 
== Development Activity ==
 
== Development Activity ==
 
<!-- Provide *evidence* of development activity of the tool. For example, RSS feeds for code issues or commits. -->
 
<!-- Provide *evidence* of development activity of the tool. For example, RSS feeds for code issues or commits. -->
 
<!-- Add the OpenHub.com ID for the tool, if known. -->
 
<!-- Add the OpenHub.com ID for the tool, if known. -->
Please go to http://www.preservaciondigital.es/acerca-de-libnova/blog-de-preservacion-digital/ (spanish) or
+
https://www.libnova.com
http://www.digitalpreservationsoftware.com/about-libnova/digital-preservation-blog/ (english)
 

Latest revision as of 15:35, 8 June 2021




libsafe allows the organizations to create a full OAIS compliant Archive, including active and passive digital preservation workflows and is particularly suited for master image files of digitizing processes.
Homepage:http://www.preservaciondigital.es
Input Formats:METS (Metadata Encoding and Transmission Standard)
Output Formats:METS (Metadata Encoding and Transmission Standard)
Function:Preservation System
Content type:Audio,Document,Image,Video,Web




Description[edit]

LIBSAFE Advanced allows the organizations to create a full OAIS aligned Archive, including active and passive digital preservation workflows and is particularly suited for master image files of digitizing processes. It's used by large institutions like the British Library, the National Library of Spain or the United States Holocaust Memorial Museum but also by smaller organizations like universities or hospitals.

It allows to manage high volume data archives with ease, having deployments of 6PB in size.

More information: https://www.libnova.com


Key features and benefits:

  • Easy to use web interface
  • Integrated catalog
  • Full preservation lifecycle management
  • Object versioning
  • Powerful metadata engine (versioning of the metadata, unique metadata and mandatory metatadata) over any schema. Full METS and PREMIS aligment. Allows creating new schemas and migrating one to another. Allows sync to other tools. Fully configurable metadata schemas.
  • REST API for ingestion, catalog search and object retrieve.
  • Works with any SMB, NFS or Object based storage.
  • Microservices: It's possible to expand various functions easily with plug-ins.
  • Linear scalability: It is possible to scale the platform to achieve any needed performance.


Detailed description[edit]

The OAIS submission agreement concept maps over a "Preservation area" on libsafe, that defines all the preservation policy to apply:

  • Sanitizing policy (deleting temp files, etc)
  • Preprocessing: new formats generation for files (doc > pdf, ARC > WARC, etc) or metadata pre-processing.
  • Checking: fixity checking (antivirus, files and folders names, jhove validation, file size, etc). Each check can be enabled, disabled or let the system choose using a regex expression.
  • Exploring: How metadata should be extracted.
  • Dissemination: how many copies and in wich "storage pools" should the information be stored. libsafe does not destroy the folder/file structure or filenames.
  • Auditing policy: How often the object should be audited by the system in background.

The whole system is "job" based. Every process is a job (ingestion job, auditing job, migration job, etc).

It implements full OAIS framework including:

Ingestion[edit]

  • DirectSIP - Create a SIP with the browser from a set of folders (one folder > one object) or ingest SIP.
  • Preprocessing: launches selected plugings over the content. A preprocessor is a plugin (essentialy a .exe file) that makes any changes on a object. Most common preprocessors can be preloaded souch as ImageMagik, FFMPEG, GhostScript, inkscape and others, but it's easy to extend the functionality.
  • Checking: A check verifies something about the object, it's files or it's metadata. A check is a plugin (essentialy a .exe file) that is launched against any elemment (object, any file, etc) if it matches a regex or extension (for ex: *.TIF) and verifies something about it, returning a "OK" or "ERROR". libsafe has about 40 checks (jhove validation, file size, file numbers, etc) bu adding more is very easy. If a check fails the ingestion job stops and the system returns the control to the user to correct the problems and re-launch the job.
  • Exploring: Extracts (from XML, CSV, text file or quering other system) and generates metadata (using DROID for instance) and loads it to the catalog. libsafe can load METS files, but reading metadata from custom XML file is easy using XSLT file. It's possible to use conversion or generation plug-ins here also (for instance, quering a OAI-PMH server using the folder name of the object).
  • Dissemination: Storage Groups contain Storage Volumnes (mountpoints). Based on the Preservation Area storage policy, libsafe selects the Storage Volumes to store the object. and starts coping them. libsafe stores the path and the hash to each file. In this part of the object, security XML files are also created, containing all the metadata, events, other locations of each file, etc.
  • Auditing: Inmediatly after finishing the object copy, libsafe scans again all the object files hasing them again and comparing the hash between the database stored hash, the ingestion job temp files hash, security XML hash and the real files hash on every copy. If the audit job is finished with a OK result, all objects are considered preserved.

Data Management[edit]

  • Bulk transforms: Allows massive metadata schema changes (DC > Marc21 for instance), but it's possible to evolve to any schema defining a crosswalk.
  • API for metadata versionig: It's possible to edit the objects metadata using an API. Previous versions of the metadata sets are always preserved. It's not really editing, it's versioning.

Access[edit]

  • Catalog: It's possible to search for any metadata of the object, locate it and visualize it's details, including:
    • Object creation details (when, who, etc)
    • Metadata details
    • Object previuos versions
    • Object files and folders hierarchy, being able to download the files or the full object in a AIP.
    • Active preservation
      • Risks
      • File formats used and risks (using PRONOM among others)
      • Metadata formats
    • Passive preservation
    • Auditing results
    • Storage details (where it's this object stored)
    • Object events (creation, access, transformation, etc)

It's also posible to query the system using the API and to retrieve files or complete objects using a RESTful API call.

Administration[edit]

Reports: various reports including collection wide. Users: create, set roles, etc. Settings: Change system settings, submission agreements, etc.


Active preservation[edit]

Including options to evolve not only file formats but also metadata schemas.

User Experiences[edit]

https://www.libnova.com

Development Activity[edit]

https://www.libnova.com