COPTR - User contributions [en-gb]

ArchiFiltre

2024-09-26T21:16:03Z

172.104.134.96: Fixed the license used to Apache 2.0

{{Infobox tool
|purpose=Overview of folder trees with fine diagrams
|homepage=https://archifiltre.fabrique.social.gouv.fr/
|license=Apache 2.0
|platforms=?
|function=File Management, Appraisal
|content=Database, Container, Binary Data, 3D, Audio, Document, Email, Ebook, Geospatial, Image, Research Data, Video, Web, Image, Software
}}
{{Infobox tool details}}
== Description ==
Archifiltre gives you orientation in deeply nested folder trees. It visualises folder contents in diagrams. It helps with cleaning up storage gone out of control. Further development is aimed at supporting archival processes like appraisal and transfer.

== User Experiences ==
* https://github.com/SocialGouv/archifiltre/wiki/Wiki-Archifiltre (French only)

== Development Activity ==

Archifiltre-Mails

2024-07-23T13:18:52Z

172.104.134.96: Created page with '{{Infobox tool |purpose=Archifiltre-Mails connects to email containers and visualizes their content, helping you in exploring and adding metadata. |homepage=https://github.com...'

{{Infobox tool
|purpose=Archifiltre-Mails connects to email containers and visualizes their content, helping you in exploring and adding metadata.
|homepage=https://github.com/SocialGouv/archifiltre-mails/wiki/Wiki-Mails-par-Archifiltre
|sourcecode=https://github.com/SocialGouv/archifiltre-docs/wiki/Wiki-Archifiltre
|license=Version 2.0
|formats_in=PST
|function=Appraisal, Metadata Processing, Annotation, Data capture and Deposit, Transfer
|content=Email
}}
{{Infobox tool details}}
== Description ==
Mails is a product of the Archifiltre solution. Its purpose is to allow any holder of an Outlook messaging to be able to analyze it via the archive format Outlook PST. Archifiltre-Mails was born out of the need for apprehension of a messaging. Currently, there is no tool for understanding a messaging system. Until then, it is impossible to know the oldest e-mail, the most important recipient, the volume of attachments contained in the messaging system, etc.

ArchiFiltre

2024-07-23T12:57:25Z

172.104.134.96: Added support for various content types

{{Infobox tool
|purpose=Overview of folder trees with fine diagrams
|homepage=https://archifiltre.fabrique.social.gouv.fr/
|license=MIT License
|platforms=?
|function=File Management, Appraisal
|content=Database, Container, Binary Data, 3D, Audio, Document, Email, Ebook, Geospatial, Image, Research Data, Video, Web, Image, Software
}}
{{Infobox tool details}}
== Description ==
Archifiltre gives you orientation in deeply nested folder trees. It visualises folder contents in diagrams. It helps with cleaning up storage gone out of control. Further development is aimed at supporting archival processes like appraisal and transfer.

== User Experiences ==
* https://github.com/SocialGouv/archifiltre/wiki/Wiki-Archifiltre (French only)

== Development Activity ==

Puremagic

2024-04-30T10:26:54Z

172.104.134.96: add puremagic

{{Infobox tool
|purpose=Puremagic is a cross-platform pure python module that will identify a file based off it's magic numbers
|sourcecode=https://github.com/cdgriffith/puremagic
|license=MIT
|cost=Free
|function=File Format Identification
}}
{{Infobox tool details}}
== Description ==

It is designed to be minimalistic and inherently cross platform compatible. It is also designed to be a stand in for python-magic, it incorporates the functions from_file(filename[, mime]) and from_string(string[, mime]) however the magic_file() and magic_string() are more powerful and will also display confidence and duplicate matches.

It does NOT try to match files off non-magic string. In other words it will not search for a string within a certain window of bytes like others might.

Advantages over using a wrapper for 'file' or 'libmagic':

* Faster
* Lightweight
* Cross platform compatible
* No dependencies

Disadvantages:

* Does not have [https://github.com/cdgriffith/puremagic/blob/master/puremagic/magic_data.json as many] file types. ("Only" 1600 at the time of posting)
* No multilingual comments
* Duplications due to small or reused magic numbers

(Help fix the first two disadvantages by contributing!)

Library (xklb)

2024-04-27T04:56:08Z

172.104.134.96: remove input formats

{{Infobox tool
|purpose=Media indexing multi-tool
|sourcecode=https://github.com/chapmanjacobd/library/
|license=BSD 3-Clause
|formats_out=DB
|function=File Management, Quality Assurance, Web Capture
}}
{{Infobox tool details}}
== Description ==


Web Capture subcommands:

* web-add: index open web directories using ffprobe and exifTool to fetch additional metadata from remote file headers (without downloading the full file) for later automated selective downloading.
* tube-add: index video site metadata via yt-dlp
* gallery-add: index image gallery site metadata via gallery-dl
* extract-links: extract links from within a webpage, even if the page uses ShadowDOM, postMessage, and nested frames
* links-add: build updatable link-scraping databases for paginated content

Local file management subcommands:

* fs-add: index local files with ffprobe, exifTool, and textract
* cluster-sort: sort lines of text by similarity (a common use for this is to identify similar file paths)
* merge-folders: merge file trees (similar to [https://github.com/chapmanjacobd/journal/blob/main/programming/linux/misconceptions.md#mv-src-vs-mv-src rclone move] but it will print detailed information about overwrites and trumps (future overwrites from multiple source folders) before moving anything)
* relmv: move but preserve parent folder information
* process-image: convert large images as scaled AVIF files as an alternative to file deletion
* process-ffmpeg: convert large video/audio files to AV1/Opus as an alternative to file deletion

Quality Assurance subcommands:

* media-check: check video and audio files for corruption by decoding small sections or the whole file

== User Experiences ==


- [https://old.reddit.com/r/opendirectories/comments/1adbv4b/i_made_a_little_cli_opendirectory_scanner_tool/ Introducing webadd to the /r/opendirectories community]

Library (xklb)

2024-04-27T04:48:22Z

172.104.134.96: /* make sentence simpler */

{{Infobox tool
|purpose=Media indexing multi-tool with more than 70 CLI subcommands
|sourcecode=https://github.com/chapmanjacobd/library/
|license=BSD 3-Clause
|formats_in=DB, Text
|formats_out=CSV (Comma Separated Values), DB, SQL
|function=File Management, Quality Assurance, Web Capture
}}
{{Infobox tool details}}
== Description ==


Web Capture subcommands:

* web-add: index open web directories using ffprobe and exifTool to fetch additional metadata from remote file headers (without downloading the full file) for later automated selective downloading.
* tube-add: index video site metadata via yt-dlp
* gallery-add: index image gallery site metadata via gallery-dl
* extract-links: extract links from within a webpage, even if the page uses ShadowDOM, postMessage, and nested frames
* links-add: build updatable link-scraping databases for paginated content

Local file management subcommands:

* fs-add: index local files with ffprobe, exifTool, and textract
* cluster-sort: sort lines of text by similarity (a common use for this is to identify similar file paths)
* merge-folders: merge file trees (similar to [https://github.com/chapmanjacobd/journal/blob/main/programming/linux/misconceptions.md#mv-src-vs-mv-src rclone move] but it will print detailed information about overwrites and trumps (future overwrites from multiple source folders) before moving anything)
* relmv: move but preserve parent folder information
* process-image: convert large images as scaled AVIF files as an alternative to file deletion
* process-ffmpeg: convert large video/audio files to AV1/Opus as an alternative to file deletion

Quality Assurance subcommands:

* media-check: check video and audio files for corruption by decoding small sections or the whole file

== User Experiences ==


- [https://old.reddit.com/r/opendirectories/comments/1adbv4b/i_made_a_little_cli_opendirectory_scanner_tool/ Introducing webadd to the /r/opendirectories community]

Library (xklb)

2024-04-27T04:36:55Z

172.104.134.96: fix bullet-points

{{Infobox tool
|purpose=Media indexing multi-tool with more than 70 CLI subcommands
|sourcecode=https://github.com/chapmanjacobd/library/
|license=BSD 3-Clause
|formats_in=DB, Text
|formats_out=CSV (Comma Separated Values), DB, SQL
|function=File Management, Quality Assurance, Web Capture
}}
{{Infobox tool details}}
== Description ==


Web Capture subcommands:

* web-add: index open web directories using ffprobe and exifTool to fetch additional metadata from remote file headers without downloading the full file into SQLite for later automated selective downloading.
* tube-add: index video site metadata via yt-dlp
* gallery-add: index image gallery site metadata via gallery-dl
* extract-links: extract links from within a webpage, even if the page uses ShadowDOM, postMessage, and nested frames
* links-add: build updatable link-scraping databases for paginated content

Local file management subcommands:

* fs-add: index local files with ffprobe, exifTool, and textract
* cluster-sort: sort lines of text by similarity (a common use for this is to identify similar file paths)
* merge-folders: merge file trees (similar to [https://github.com/chapmanjacobd/journal/blob/main/programming/linux/misconceptions.md#mv-src-vs-mv-src rclone move] but it will print detailed information about overwrites and trumps (future overwrites from multiple source folders) before moving anything)
* relmv: move but preserve parent folder information
* process-image: convert large images as scaled AVIF files as an alternative to file deletion
* process-ffmpeg: convert large video/audio files to AV1/Opus as an alternative to file deletion

Quality Assurance subcommands:

* media-check: check video and audio files for corruption by decoding small sections or the whole file

== User Experiences ==


- [https://old.reddit.com/r/opendirectories/comments/1adbv4b/i_made_a_little_cli_opendirectory_scanner_tool/ Introducing webadd to the /r/opendirectories community]

Library (xklb)

2024-04-27T04:35:22Z

172.104.134.96: add library (xklb)

{{Infobox tool
|purpose=Media indexing multi-tool with more than 70 CLI subcommands
|sourcecode=https://github.com/chapmanjacobd/library/
|license=BSD 3-Clause
|formats_in=DB, Text
|formats_out=CSV (Comma Separated Values), DB, SQL
|function=File Management, Quality Assurance, Web Capture
}}
{{Infobox tool details}}
== Description ==


Web Capture subcommands:

- web-add: index open web directories using ffprobe and exifTool to fetch additional metadata from remote file headers without downloading the full file into SQLite for later automated selective downloading.
- tube-add: index video site metadata via yt-dlp
- gallery-add: index image gallery site metadata via gallery-dl
- extract-links: extract links from within a webpage, even if the page uses ShadowDOM, postMessage, and nested frames
- links-add: build updatable link-scraping databases for paginated content

Local file management subcommands:

- fs-add: index local files with ffprobe, exifTool, and textract
- cluster-sort: sort lines of text by similarity (a common use for this is to identify similar file paths)
- merge-folders: merge file trees (similar to [https://github.com/chapmanjacobd/journal/blob/main/programming/linux/misconceptions.md#mv-src-vs-mv-src rclone move] but it will print detailed information about overwrites and trumps (future overwrites from multiple source folders) before moving anything)
- relmv: move but preserve parent folder information
- process-image: convert large images as scaled AVIF files as an alternative to file deletion
- process-ffmpeg: convert large video/audio files to AV1/Opus as an alternative to file deletion

Quality Assurance subcommands:

- media-check: check video and audio files for corruption by decoding small sections or the whole file

== User Experiences ==


- [https://old.reddit.com/r/opendirectories/comments/1adbv4b/i_made_a_little_cli_opendirectory_scanner_tool/ Introducing webadd to the /r/opendirectories community]

FileTrove

2024-02-16T14:34:27Z

172.104.134.96: Created page with "{{Infobox tool |purpose=FileTrove indexes files and creates metadata from them. The single binary application walks a directory tree and identifies all regular files by type w..."

{{Infobox tool
|purpose=FileTrove indexes files and creates metadata from them. The single binary application walks a directory tree and identifies all regular files by type with Siegfried.
|homepage=https://github.com/steffenfritz/FileTrove
|function=Metadata Extraction
}}
{{Infobox tool details}}
== Description ==
FileTrove indexes files and creates metadata from them. The single binary application walks a directory tree and identifies all regular files by type with siegfried, giving you the MIME type, PRONOM identifier, Format version, Identification proof and note.

== User Experiences ==

PREMIS Utility

2024-01-29T23:02:33Z

172.104.134.96: Added new PREMIS Utility page to COPTR.

{{Infobox tool
|purpose=The PREMIS Utility is a graphical program used to generate PREMIS metadata records for use in digital preservation systems and digital asset management systems in JSON and XML format, and attempts to cover gaps not programmatically generated by system logs.
|homepage=https://github.com/rochester-rcl/premis-generator
|sourcecode=https://github.com/rochester-rcl/premis-generator/archive/refs/tags/v1.1.0.zip
|license=Apache License, version 2.0
|cost=None
|platforms=Windows
|language=Python
|formats_in=CSV (Comma Separated Values)
|formats_out=XML, PREMIS (Preservation Metadata Implementation Strategies), JSON
|function=Metadata Processing
|content=Metadata
}}
{{Infobox tool details}}
== Description ==
The PREMIS Utility is a graphical utility used to generate PREMIS metadata records for use in digital preservation systems and digital asset management systems. Records can be exported in both XML and JSON formats. This utility is specifically created to address gaps in administrative metadata that might not be automatically created by a software platform, and specifically seeks to address the following:

- Unambiguous assertion of whether the resource is born digital or digitized
- Rights related information about the resource, such as if the intellectual content is in the public domain or protected by copyright
- Information about any digital preservation activities happening outside the software platform, such as manual migrations

The graphical utility was created to make the creation of these records easier and more approachable for librarians, archivists, and other cultural heritage workers that may not be comfortable in the command line or know Python. The overall workflow is that a list of identifiers are provided to the utility, selections are made in the interface and information is provided, and ultimately it spits out XML or JSON files that each have that identifier as the filename. The idea here is that while I don't have any notion what system you might be working within, you should hopefully have a means of using that identifier in the filename to link up or import the metadata into your system.

Currently this utility is only packaged as an .exe for Windows computers. If you are comfortable with some Python you should be able use the raw Python files (after installing the nonstandard Python libraries listed below) to run the graphical utility in a Linux or MacOS environment. If there are Linux/MacOS users out there that would like to help me create packages for those operating systems, I'm super game.

Additionally this utility is designed and meant for a United States locality. A big part of this tool relates to copyright, and while I am not a lawyer and the utility in graphical format or code and the metadata records that it exports absolutely do not constitute legal advice, I do have training in US copyright law and am leveraging that for the tool. I do not however have any non-US copyright training, so this will be of very limited use outside the United States.

== User Experiences ==
Full and current documentation on how the utility functions and what each field and button does can be found at the GitHub repository readme.

Rescarta

2024-01-26T15:38:42Z

172.104.134.96:

{{Infobox tool
|purpose=The ResCarta Tools software empowers users to create non-proprietary digital objects with LOC standard METS, MODS, MIX and AudioMD metadata from existing TIFF, JPEG, PDF and WAV data through user-friendly interfaces.
|homepage=http://www.ResCarta.org
|license=Apache License v2.0
|platforms=Linux, Windows, and OSX operating systems
|formats_out=METS (Metadata Encoding and Transmission Standard)
|function=Access, Personal Archiving, Preservation System
|content=Audio, Document, Research Data, Metadata
}}
{{Infobox tool details
|ohloh_id=rescarta
}}
== Description ==

ResCarta Toolkit allows users to create digital archives from scans, digital photographs or recordings of analog objects. Metadata is added using simple forms and is written into each digital object to Library of Congress Standards of METS, MODS, MIX, AudioMD, and reVTMD. Audio/Video files containing spoken words can be automatically transcribed. Textural content of documents, audio or video can be edited to create highly accurate transcriptions using graphical tools. Standard directory and file naming is produced during use with autogenerated checksums. A complete Lucene tm index of metadata and textural content can be created for use in a fully functional web application for discovery and display of digital objects. A checksum validation tool is included to assure long term stability of the archive.

====Platform====
The ResCarta Toolkit runs on Windows, Mac and Linux operating systems in single user or coordinated multiuser mode.

== User Experiences ==

* https://sourceforge.net/projects/rescarta/reviews?source=navbar

== Development Activity ==

Years of archived releases can be found at...
http://sourceforge.net/projects/rescarta/files/ResCarta%20Tools/

Workflow:Validation Error Analysis and Treatment for PDF-hul 122 Invalid destination - Destination NULL

2024-01-21T21:34:02Z

172.104.134.96:

{{Infobox COW
|status=Production
|tools=Pdfcpu, Qpdf, JHOVE, Adobe Acrobat Pro, PDF Checker
|input=File with JHOVE validation error PDF-HUL-122 “Invalid Destination” / Well-Formed, but not valid.
This specific workflow documents the sample treatment for a file currently found here: https://www.schleswig-holstein.de/DE/landesregierung/ministerien-behoerden/VIII/Service/Broschueren/Broschueren_VIII/Kita/Mein_Kind_kommt_in_die_Kita_Rechte.pdf?__blob=publicationFile&v=2
|output=Fixed file
|organisation=TIB
|organisationurl=https://wiki.tib.eu/confluence/display/lza/Digital+preservation+at+TIB
}}
==Workflow Description==



The workflow describes the analysis and the fix of a specific instance of a PDF-HUL-122 error.
 CAUTION! PDF-HUL-122 errors can be very different in nature and impact. 
The process described here can be used for internal document links that are not working. It is a manual workflow. More information about this specific case has been discussed in a blog (see link under "Further information"). The methodology used here is that introduced in https://hdl.handle.net/2142/121092.

'''Step 1: Validation Error'''

JHOVE v1.28 PDF-hul v1.12.4 PDF-HUL-122 Invalid Destination with offset given. Well-formed, but not valid.

'''Step 2: Cross-Check with other Tools'''

Cross-checked with:
pdfcpu v0.6.0dev relaxed mode - no error
pdfcpu v0.6.0dev strict mode - unrelated error (Font error)
qpdf v9.1.1 - no error
PDF Checker 2.1.0 - no error
'''
Step 3: Matching Results?'''

No. Error not reported by other tools, most likely due to low priority of error (only impacts validity, not well-formedness).
One additional un-related error picked up.

'''Step 4: Choose Error to Treat'''

Original PDF-HUL-122. Ignore Font error.

'''Step 5A: Locate Error in Spec'''

ISO 32000-2:2017 12.3.2.4 Named destinations must contain name as well as target destination. In case of internal destinations this is typically an object reference.

'''Step 5B: Locate Error in File
'''
Offset given by JHOVE only points to place where reference was used in GoTo destination. The reader tries to resolved the named destination used in the GoTo action via the name tree. Here the object is missing and replaced by "Null".
(Rechte_von_Eltern_in_der_Kita_2018_V7_bf.indd:.45593:62)[null/Fit ]

'''Step 6: Match?'''

Yes.

'''Step 7: Fixable?'''

Yes.

'''Step 8: Fix'''

Find location by checking page object the wrong destination is used on. With Adobe Acrobat Pro's "Edit Link" option, the erronous link can be removed and replaced by a correct one, if known.

'''Step 9: Check'''

Re-validated file with JHOVE: now well-formed and valid. Link is now actionable.

'''Step 10: Success?'''

Yes.

[[File:PDF-hul-122 1.jpg|Textual description]] 



==Purpose, Context and Content==

This workflow describes the analysis and treatment of JHOVE PDF-Hul error message PDF-HUL-122 Invalid destination. It describes the process and results of a manual validation error analysis.

==Evaluation/Review==

The workflow is effective for this specific instance of the error. It should be replicable for pdfs with similar invalid destination problems, i.e., internal links have been replaced with NULL. However, the fix needs to be considered carefully as it impacts the internal structure of the PDF file.

==Further Information==

See a further discussion of this in the OPF blog: https://openpreservation.org/blogs/destination-null-one-of-the-many-cases-of-pdf-hul-122/?q=1

Warc Analyzer

2024-01-02T15:32:30Z

172.104.134.96: Created page with "{{Infobox tool |purpose=A proof-of-concept client side webapp for analyzing WARC data using Webrecorder's warcio.js. No WARC data is uploaded anywhere it runs on your machine...."

{{Infobox tool
|purpose=A proof-of-concept client side webapp for analyzing WARC data using Webrecorder's warcio.js. No WARC data is uploaded anywhere it runs on your machine. The idea is that it would be useful for archivists who have been given a pile of WARC data and they would like to quickly know what it contains.
|homepage=https://github.com/edsu/warc-analyzer
|formats_in=WARC
|function=Discovery
|content=Web
}}
{{Infobox tool details}}
== Description ==


== User Experiences ==

Workflow:OVP migration flow

2023-10-16T21:18:29Z

172.104.134.96:

{{Infobox COW
|status=Production
|tools=python, OVP, API
|input=Media assets and descriptive metadata in source OVP.
|output=Media assets and descriptive metadata in destination OVP.
|organisation=La Digitalizadora de la Memoria Colectiva
|organisationurl=https://ladigitalizadora.org/
}}
OVP migration flow - an agnostic blueprint

To initiate the transition from any given Source [https://en.wikipedia.org/wiki/Online_video_platform On-line Video Platform] to a Destination OVP, the first step involves retrieving video information from the OVP Source. You should be able to achieve this by making a GET request to the OVP Source's API authenticating against it, typically using an access token and video ID. This request should typically be directed to a generic API endpoint.

Upon receiving a response, you should have the opportunity to extract valuable video metadata, including details like the title, description, and tags. Additionally, you can identify the download link for the highest quality version available.

This process should result in acquiring a video asset along with its associated metadata, often structured in XML format. Once these initial steps are executed, you may offer the user the choice to remove the video entry from OVP Source.

Following this, the natural progression leads into the OVP Destination ingest process, aligning with their API documentation and specifications.



[[File:AVmigrationflow.png|OVP migration flow UML]] 



Purpose: The purpose of the document is to outline a generic blueprint for the migration of video content from any Source On-line Video Platform (OVP) to a Destination OVP. This blueprint provides a high-level overview of the key steps and considerations involved in such a migration process.

Context: In the context of digital content migration, this blueprint serves as a guide for initiating the transition from one OVP to another. It emphasizes the importance of retrieving video information from the source platform, including metadata extraction and potential deletion of content. The document sets the stage for a seamless migration process, aligning with the API documentation and specifications of the destination OVP.




Some On-line Video Platforms (OVPs) and the sources of their API documentation:

YouTube:
Source: [https://developers.google.com/youtube/registering_an_application]
Documentation URL: YouTube Data API Documentation
Description: YouTube provides a Data API that allows developers to access YouTube's features programmatically. It covers functionalities related to video uploads, playlists, video information retrieval, and more.

Vimeo:
Source: [https://developer.vimeo.com/api/reference]
Documentation URL: Vimeo API Documentation
Description: Vimeo's API documentation offers resources for integrating Vimeo's video hosting and sharing capabilities into applications. It provides access to features like video upload, video information retrieval, and advanced privacy controls.

Kaltura:
Source: [https://developer.kaltura.com/]
Documentation URL: Kaltura Developer Documentation
Description: Kaltura's Developer Documentation provides extensive resources for developers to leverage Kaltura's video platform capabilities. It covers video management, publishing, and monetization, as well as features like video uploading, player customization, and analytics.

Brightcove:
Source: [https://apis.support.brightcove.com/]
Documentation URL: Brightcove API Documentation
Description: Brightcove's API documentation provides comprehensive resources for developers to interact with the Brightcove platform programmatically. It includes features related to video publishing, content management, player customization, analytics, and monetization.

DLCM

2023-10-09T09:04:36Z

172.104.134.96:

{{Infobox tool
|purpose=DLCM is an archiving solution based on the OAIS model developed in Java by Swiss universities. It is also the technological stack of the OLOS service.
|homepage=https://gitlab.unige.ch/dlcm
|sourcecode=https://gitlab.unige.ch/dlcm
|license=GNU GPL v2.0
|formats_out=METS (Metadata Encoding and Transmission Standard), PREMIS (Preservation Metadata Implementation Strategies)
|function=Preservation System
}}
{{Infobox tool details}}
== Description ==


== User Experiences ==

DLCM

2023-10-09T09:04:15Z

172.104.134.96:

{{Infobox tool
|purpose=DLCM is an archiving solution based on the OAIS model (ISO 14721) developed in Java by Swiss universities. It is also the technological stack of the OLOS service.
|homepage=https://gitlab.unige.ch/dlcm
|sourcecode=https://gitlab.unige.ch/dlcm
|license=GNU GPL v2.0
|formats_out=METS (Metadata Encoding and Transmission Standard), PREMIS (Preservation Metadata Implementation Strategies)
|function=Preservation System
}}
{{Infobox tool details}}
== Description ==


== User Experiences ==

DLCM

2023-10-09T09:02:39Z

172.104.134.96:

{{Infobox tool
|purpose=DLCM is an archiving solution based on the OAIS model (ISO 14721) developed by Swiss universities. It is also the technological stack of the OLOS service.
|homepage=https://gitlab.unige.ch/dlcm
|sourcecode=https://gitlab.unige.ch/dlcm
|license=GNU GPL v2.0
|language=Java
|formats_out=METS (Metadata Encoding and Transmission Standard), PREMIS (Preservation Metadata Implementation Strategies)
|function=Preservation System
}}
{{Infobox tool details}}
== Description ==


== User Experiences ==

DLCM

2023-10-09T08:16:36Z

172.104.134.96: Created page with "{{Infobox tool |purpose=DLCM is an archiving solution based on the OAIS model (ISO 14721) developed by Swiss universities. It is also the technological stack of the OLOS servi..."

{{Infobox tool
|purpose=DLCM is an archiving solution based on the OAIS model (ISO 14721) developed by Swiss universities. It is also the technological stack of the OLOS service.
|homepage=https://gitlab.unige.ch/dlcm
|sourcecode=https://gitlab.unige.ch/dlcm
|license=GNU GPL v2.0
|formats_out=METS (Metadata Encoding and Transmission Standard), PREMIS (Preservation Metadata Implementation Strategies)
|function=Preservation System
}}
{{Infobox tool details}}
== Description ==


== User Experiences ==

Workflow:Deaccessioning data

2023-10-05T11:39:44Z

172.104.134.96: Procedure to remove datasets but retaining the PID

{{Infobox COW
|status=Production
|tools=DataCite
|input=Datasets for which there is reason to remove them from the collection.
|output=A tombstone page (or similar) to ensure the resolving of the PID of the dataset, leading to information that the dataset was removed.
|organisation=SEADDA community of archaeologists and digital specialists
|organisationurl=https://www.seadda.eu/
}}
==Workflow Description==



[[File:workflow.png|Textual description]] 



Datasets may need to be deaccessioned or deleted, depending on institutional policy, when:
* Errors are discovered in a published dataset that renders it unusable for research.
* If the content is unlawful or the dataset is fraudulent from a scientific point of view.
* If one or more of the authors or rights holders did not give permission for publication.
* If personal data appears to have been deposited without a legal ground such as permission of research subjects.
* If the content consists of personal data, and a research subject rightfully objects to the preservation of a digital object with an appeal to the GDPR (e.g. right to be forgotten; or revocation of informed consent).
* If there is a legally binding maximum preservation period for the content.

If there is sufficient ground to decide to deaccession a dataset, the deaccessioning is done by a Data Manager who will conduct an appraisal of the request.

In all cases where access to a published dataset is terminated, a notice will be added to the landing page of the Persistent Identifier (PID) associated with the dataset to indicate that the dataset is no longer available: this is called a 'tombstone page'. This is done to ensure that anyone that has citations to resources that have subsequently been removed are retained, and furthermore that the provenance of the deaccession or removal is recorded within all associated metadata.

==Purpose, Context and Content==


Within the SEADDA consortium, workflows for dealing with deaccessioning of datasets were examined from three long standing digital repositories for archaeological data: the Archaeology Data Station of Data Archiving and Networked Services (DANS), the Swedish National Data Service (SND), and the Archaeology Data Service (ADS).

==Evaluation/Review==


==Further Information==

https://docs.google.com/document/d/1mAL20vvhZnYkZuc_60awac5wc-kiaBzYZ_AJahS_nkM/edit?usp=sharing


[[Special:Contributions/172.104.134.96|172.104.134.96]] 11:39, 5 October 2023 (UTC)

Workflow:Assessing information

2023-10-05T11:20:26Z

172.104.134.96:

{{Infobox COW
|status=Production
|input=Submitted datasets.
|output=Assessed datasets. This workflow is part of the curation to ensure FAIR datasets within a Trusted Digital Repository
|organisation=SEADDA community of archaeologists and digital specialists
|organisationurl=https://www.seadda.eu/
}}
==Workflow Description==



[[File:workflow.png|Textual description]] 



A member of staff runs through a checklist to ensure that:
* The dataset confirms to the collection policy of the repository
* Data deposit contains no malware (all files).
* Digital objects are in correct formats (all files).
* Data deposit has collection-level metadata.
* All digital objects have core descriptive metadata (all files).
* Digital objects have additional technical metadata (all files).
* Digital objects can be opened, are valid, and can be reused (all files, representative sample for large datasets) .
* The data deposit has no sensitive data concerns (all files, representative sample for large datasets).
* Content is appropriate and complete (all files, representative sample for large datasets).
* The dataset is structured in a manner which is clear for the purposes of reusing data.

==Purpose, Context and Content==


Within the SEADDA consortium, workflows for dealing with assessing information were examined from three long standing digital repositories for archaeological data: the Archaeology Data Station of Data Archiving and Networked Services (DANS), the Swedish National Data Service (SND), and the Archaeology Data Service (ADS).

==Evaluation/Review==


==Further Information==

https://docs.google.com/document/d/1mAL20vvhZnYkZuc_60awac5wc-kiaBzYZ_AJahS_nkM/edit?usp=sharing


[[Special:Contributions/172.104.134.96|172.104.134.96]] 11:19, 5 October 2023 (UTC)

Workflow:Assessing information

2023-10-05T11:19:17Z

172.104.134.96: A workflow that is undertaken by repositories for all datasets to ensure relevance and understandability of data and metadata based on defined criteria.

{{Infobox COW
|status=Production
|input=Submitted datasets.
|output=Assessed datasets. This workflow is part of the curation to ensure FAIR datasets within a Trusted Digital Repository
|organisation=SEADDA community of archaeologists and digital specialists
|organisationurl=https://www.seadda.eu/
}}
==Workflow Description==



[[File:workflow.png|Textual description]] 



A member of staff runs through a checklist to ensure that:
- The dataset confirms to the collection policy of the repository
- Data deposit contains no malware (all files).
- Digital objects are in correct formats (all files).
- Data deposit has collection-level metadata.
- All digital objects have core descriptive metadata (all files).
- Digital objects have additional technical metadata (all files).
- Digital objects can be opened, are valid, and can be reused (all files, representative sample for large datasets) .
- The data deposit has no sensitive data concerns (all files, representative sample for large datasets).
- Content is appropriate and complete (all files, representative sample for large datasets).
- The dataset is structured in a manner which is clear for the purposes of reusing data.

==Purpose, Context and Content==


Within the SEADDA consortium, workflows for dealing with assessing information were examined from three long standing digital repositories for archaeological data: the Archaeology Data Station of Data Archiving and Networked Services (DANS), the Swedish National Data Service (SND), and the Archaeology Data Service (ADS).

==Evaluation/Review==


==Further Information==

https://docs.google.com/document/d/1mAL20vvhZnYkZuc_60awac5wc-kiaBzYZ_AJahS_nkM/edit?usp=sharing


[[Special:Contributions/172.104.134.96|172.104.134.96]] 11:19, 5 October 2023 (UTC)

CSV export form for Microsoft Access

2023-09-28T11:53:03Z

172.104.134.96: An Access form to export database tables to CSV

{{Infobox tool
|purpose=A Microsoft Access form to export all database tables to interoperable CSV files.
|homepage=https://dans.knaw.nl/en/file-formats/spreadsheets/csv/
|platforms=Microsoft Windows
|language=English; Dutch
|formats_in=MDB, ACCDB, XLS
|formats_out=CSV (Comma Separated Values)
|function=File Format Migration
|content=Database, Spreadsheet
}}
{{Infobox tool details}}
== Description ==

Directly exporting a data table from Microsoft Access to CSV often results in incorrect handling of certain values such as dates, integers with separators, memo fields and such. This tool consists of an Access database in MDB format which contains a single Form, which can be used to easily select and export all tables within the database to standardized CSV files, keeping all of the values intact.

The form can be copied from this MDB into another Access database, to use for exporting the tables from that database.

It is also possible to import an External data table or spreadsheet (such as an Excel sheet or a DBF file) into this database, then export that table to CSV.

Please take note of the fact that only the data tables are exported. If information on table attributes or table relations is present and needs to be retained, this information should be documented separately. Access provides the Database Tool 'Database Documenter' which can write this information into a PDF file.

When importing tables from spreadsheets, take note that the CSV exports will only keep the values from rows and columns and will not store characteristics such as coloring of cells, cell annotations or formulas.

Known issues and instructions how to resolve them are detailed in a readme file, provided with the tool.

This script was developed internally at DANS (Data Archiving and Networked Services), the Dutch national centre of expertise and repository for research data.

Workflow:Archival Forensics workflow (storage media deposit)

2023-05-26T11:59:10Z

172.104.134.96:

{{Infobox COW
|status=Experimental
|tools=Archivists' Toolkit, Audacity, BitCurator, Duke Data Accessioner, FTK (Forensic Toolkit), Karen's Directory Printer, TeraCopy, TreeSize, VLC Media Player, VirtualBox, WinMerge
|input=Request to forensically process a digital deposit (storage media) to the University of Glasgow Archives & Special Collections, as part of of the Digital Archiving workflow (see Further Information).
|output=A verified, authentic copy of storage media content exported as a logical or physical image file, with or without forensic processing.
|organisation=Archives and Special Collections (ASC), University of Glasgow
|organisationurl=https://www.gla.ac.uk/myglasgow/archivespecialcollections/
}}
==Workflow Description==



[[File:Archival-forensics-workglow_v0-1_web_sm.png|1000px|Archival forensics workflow produced by Archives and Special Collections at the University of Glasgow]] 



; START
: A request to forensically process a digital deposit (storage media) to the University of Glasgow Archives & Special Collections, as part of of the [https://coptr.digipres.org/index.php/Workflow:Digital_archiving_workflow_(high-level) Digital Archiving workflow.] 

; PREPARATION
: Obtain supporting resources and materials to forensically process digital storage media.
# Consult the physical conservation and preservation report, documenting all actions on the acquired media and produced during Acquisition in the [https://coptr.digipres.org/index.php/Workflow:Digital_archiving_workflow_(high-level) Digital Archiving workflow].
# Retrieve the unique accession number generated for the media to be processed from the Collections Management System. Use the accession number as reference in all forensic processing actions.
# Update the conservation and preservation logs on the Collections Management System relating to the storage media, including:
#* photographic records of the storage media before processing, clearly showing state, serial number(s) and any other relevant information recorded on the media (e.g. labels).
#* Documentation of media characteristics, such as technology, type, brand, model, serial number.
#* Documentation of any hardware setup or configuration necessary to process the storage medium.
# Proceed to Imaging.

; IMAGING
: Create an exact copy of storage media, encapsulating contents and structures in a single file (a disk image).
# Use write-blocking tools (software or hardware) to only permit read-only access to storage media, so as to avoid compromising the integrity of the data; and protect the data chain of custody.
# Use disk imaging software to generate a forensic image file, which can either be:
#* a physical image, which is a bit-by-bit (exact) copy of the storage medium and includes active (used) and free space. Any deleted data or file fragments will be copied into the image file.
#* A logical image, which captures active data on the device but not any deleted space, deleted files or fragments.
#* A selection of specific files and directories, also known as a targeted collection.
# Instruct the disk imaging software to create a complete file and directory listing; and verify the integrity of the generated image file by comparing hashes:
#* If verification fails and attempts at re-imaging are unsuccessful, create a "failed imaging" report in the Collections Management System logs.
#* If verification is successful, store the image in process store.
# Is further forensic processing and analysis required?
#* If no, submit the verified disk image to the [https://coptr.digipres.org/index.php/Workflow:Digital_archiving_workflow_(high-level) Digital Archiving workflow]. OR
#* If yes, proceed to Processing.

; PROCESSING
: Extract and manage information from the data in storage media, and make it available for analysis.
# Collate sources for processing, by selecting specific folders/files to review and - where appropriate - aggregating data from multiple storage media.
# Perform virus and malware detection checks on the collated sources.
# Use forensic software to identify and, if possible, remove irrelevant or redundant files from processing. Examples may include operating systems, system files, or user-defined files that have been deemed as irrelevant.
# Use forensic software to process the data, including hash generation for files; expanding compound files (e.g. zip archives); format identification and validation; creating search text indices; and preparing audiovisual, web and email data for analysis.
# Proceed to Analysis.

; ANALYSIS
: Use digital forensics methods to search, categorise, review, interpret and curate data in storage media, so as to aid selection and appraisal processes.
# Review the agreement(s) under which the records were donated, in order to identify permissible actions (e.g. whether restoring deleted files is allowed).
# Depending on the nature of the data and on archival needs, use forensic software to identify records of interest, and make them available for appraisal. Analysis methods may include:
#* Data carving, for restoring data that was deleted or lost from the file system.
#* Decrypting encrypted files and recovering passwords for password-protected files.
#* Viewing and exporting geolocation data from files that have geolocation information associated with them.
#* Analysing document content to explore terms/words of interest; and automate the identification of personal information, such as names, phone numbers, credit card and social security numbers.
#* Identifying the language in which documents are written.
#* Generating thumbnails from images and videos; and extracting metadata from multimedia files.
#* Flagging duplicate files.
#* Discovering information (including documents and email communications) relating to pre-defined lists of persons of interest.
# Once all analyses have been completed, consolidate the resulting data into an appropriate file/folder structure.
# Proceed to Exporting.

; EXPORTING
: Export the forensically analysed contents of storage media as logical disk images, alongside relevant processing reports, filters and labels.
# Export any custom filters and labels created to manage the data, which can be useful for other digital archiving processes. Filters help locate items of interest quickly; and labels allow for grouping files in customised ways (e.g. flagging content that requires archivist attention; or records associated with a specific individual).
# Export any reports generated during processing and analysis, such as file hashes, virus and malware detection reports, search index terms and geolocation data.
# Export the forensically curated contents of processed storage media into a logical disk image.
# Submit the logical disk image to the [https://coptr.digipres.org/index.php/Workflow:Digital_archiving_workflow_(high-level) Digital Archiving workflow].

==Purpose, Context and Content==

The workflow is meant to describe the steps and processes involved in an archival forensics examination of digital records submitted in storage media to University Archives at the University of Glasgow. Although the workflow can operate as stand-alone, it has been designed to align with and extend the [https://coptr.digipres.org/index.php/Workflow:Digital_archiving_workflow_(high-level) Digital Archiving workflow].

DiPS (Digital Preservation Solution)

2023-04-24T09:57:27Z

172.104.134.96:

{{Infobox tool
|purpose=DiPS (OAIS compliant '''Di'''gital '''P'''reservation '''S'''olution)
|homepage=https://go.sergroup.com/dips-kommunal
|platforms=Linux and Windows as well as the common database systems
|language=Java 8 / 11
|formats_in=e.g. XML, XDOMEA, XPSR
|formats_out=e.g. XML, XDOMEA, PREMIS
|function=Access, Active Data Storage, File Format Identification, File Format Migration, File Management, Metadata Extraction, Preservation System, Secure Deletion, Service, Storage, Transfer, Validation, Workflow
|content=Audio, Binary Data, Container, Document, Image, Metadata, Video
}}
{{Infobox tool details}}
== Description ==
DiPS is an OAIS compliant long-term archive solution based on the ECM systems from SER Group ([https://www.sergroup.com/en/ sergroup.com]).
The long-term data is stored as XMLs within the AIPs and managed in the ECM index management. It uses PREMIS to store technical object information (format information, hash values, file sizes, etc.), events (pre-ingest, ingest, conversions, etc.), agents (natural persons and systems) and relationships between objects. The preservation metadata is acquired both from internal mechanisms and from external sources.
Due to the extensive use of Java and standardized APIs it can be extended and customized in many ways.

Furthermore the system is based on:
* DROID and custom developments for format recognition
* JHOVE and a number of custom modules for format validation
* Custom development and free tools (including ffmpeg/ffprobe) for metadata extraction from content objects
* Custom development and free tools for format conversion

== Stakeholder/Audience ==
The primary focus are (long-term) archives but can also be used in other environments with appropriate extensions (e.g. Geographic Information Systems).
For example, it is offered as a service in the digital archive network "DiPS.kommunal" in North Rhine-Westphalia, Germany ([http://www.danrw.de danrw.de] / [https://www.danrw.de/ueber-das-da-nrw/da-nrw-ein-loesungsverbund details])

DiPS (Digital Preservation Solution)

2023-04-24T09:54:12Z

172.104.134.96:

DiPS (Digital Preservation Solution)

2023-04-24T09:51:17Z

172.104.134.96:

{{Infobox tool
|purpose=DiPS (OAIS compliant '''Di'''gital '''P'''reservation '''S'''olution)
|homepage=https://go.sergroup.com/dips-kommunal
|platforms=Linux and Windows as well as the common database systems
|language=Java 8 / 11
|formats_in=e.g. XML, XDOMEA, XPSR
|formats_out=e.g. XML, XDOMEA, PREMIS
|function=Access, Active Data Storage, File Format Identification, File Format Migration, File Management, Metadata Extraction, Preservation System, Secure Deletion, Service, Storage, Transfer, Validation, Workflow
|content=Audio, Binary Data, Container, Document, Image, Metadata, Video
}}
{{Infobox tool details}}
== Description ==
DiPS is an OAIS compliant long-term archive solution based on the ECM systems from SER Group ([https://www.sergroup.com/en/ sergroup.com]).
The long-term data is stored as XMLs within the AIPs and managed in the ECM index management. It uses PREMIS to store technical object information (format information, hash values, file sizes, etc.), events (pre-ingest, ingest, conversions, etc.), agents (natural persons and systems) and relationships between objects.
Due to the extensive use of Java and standardized APIs it can be extended and customized in many ways.

Furthermore the system is based on:
* DROID and custom developments for format recognition
* JHOVE and a number of custom modules for format validation,
* Custom development and free tools (including ffmpeg/ffprobe) for metadata extraction from content objects
* Custom development and free tools for format conversion

== Stakeholder/Audience ==
The primary focus are (long-term) archives but can also be used in other environments with appropriate extensions (e.g. Geographic Information Systems)
For example, it is offered as a service in the digital archive network "DiPS.kommunal" in North Rhine-Westphalia, Germany ([http://www.danrw.de danrw.de])

DiPS (Digital Preservation Solution)

2023-04-24T09:49:06Z

172.104.134.96:

{{Infobox tool
|purpose=DiPS (OAIS compliant '''Di'''gital '''P'''reservation '''S'''olution)
|homepage=https://go.sergroup.com/dips-kommunal
|platforms=Linux and Windows as well as the common database systems
|language=Java 8 / 11
|formats_in=e.g. XML, XDOMEA, XPSR
|formats_out=e.g. XML, XDOMEA, PREMIS
|function=Access, Active Data Storage, File Format Identification, File Format Migration, File Management, Metadata Extraction, Preservation System, Secure Deletion, Service, Storage, Transfer, Validation, Workflow
|content=Audio, Binary Data, Container, Document, Image, Metadata, Video
}}
{{Infobox tool details}}
== Description ==
DiPS is an OAIS compliant long-term archive solution based on the ECM systems from SER Group ([https://www.sergroup.com/en/ sergroup.com]).
The long-term data is stored as XMLs within the AIPs and managed in the ECM index management. It uses PREMIS to store technical object information (format information, hash values, file sizes, etc.), events (pre-ingest, ingest, conversions, etc.), agents (natural persons and systems) and relationships between objects.
Due to the extensive use of Java and standardized APIs it can be extended and customized in many ways.

== Stakeholder/Audience ==
The primary focus are (long-term) archives but can also be used in other environments with appropriate extensions (e.g. Geographic Information Systems)
For example, it is offered as a service in the digital archive network "DiPS.kommunal" in North Rhine-Westphalia, Germany ([http://www.danrw.de danrw.de])

DiPS (Digital Preservation Solution)

2023-04-24T09:45:30Z

172.104.134.96:

DiPS (Digital Preservation Solution)

2023-04-24T09:36:55Z

172.104.134.96:

DiPS (Digital Preservation Solution)

2023-04-24T09:36:09Z

172.104.134.96:

DiPS (Digital Preservation Solution)

2023-04-24T09:35:16Z

172.104.134.96:

DiPS (Digital Preservation Solution)

2023-04-24T09:33:08Z

172.104.134.96:

DiPS (Digital Preservation Solution)

2023-04-24T08:54:01Z

172.104.134.96:

{{Infobox tool
|purpose=DiPS (OAIS compliant '''Di'''gital '''P'''reservation '''S'''olution)
|homepage=https://go.sergroup.com/dips-kommunal
|formats_in=XML, XDOMEA, XPSR
|formats_out=XML, XDOMEA, PREMIS
|function=Access, Active Data Storage, File Format Identification, File Format Migration, File Management, Metadata Extraction, Preservation System, Secure Deletion, Service, Storage, Transfer, Validation, Workflow
|content=Audio, Binary Data, Container, Document, Image, Metadata, Video
}}
{{Infobox tool details}}
== Description ==
DiPS is an OAIS compliant long-term archive solution based on the ECM system from SER Group.
It uses PREMIS to store technical object information (format information, hash values, file sizes, etc.), events (pre-ingest, ingest, conversions, etc.), agents (natural persons and systems) and relationships between objects.

DiPS (Digital Preservation Solution)

2023-04-24T08:53:00Z

172.104.134.96:

{{Infobox tool
|purpose=DiPS (OAIS compliant '''Di'''gital '''P'''reservation '''S'''olution)
|homepage=https://go.sergroup.com/dips-kommunal
|formats_in=XML, XDOMEA
|formats_out=XML
|function=Access, Active Data Storage, File Format Identification, File Format Migration, File Management, Metadata Extraction, Preservation System, Secure Deletion, Service, Storage, Transfer, Validation, Workflow
|content=Audio, Binary Data, Container, Document, Image, Metadata, Video
}}
{{Infobox tool details}}
== Description ==
DiPS is an OAIS compliant long-term archive solution based on the ECM system from SER Group.
It uses PREMIS to store technical object information (format information, hash values, file sizes, etc.), events (pre-ingest, ingest, conversions, etc.), agents (natural persons and systems) and relationships between objects.

DiPS (Digital Preservation Solution)

2023-04-24T08:52:06Z

172.104.134.96:

{{Infobox tool
|purpose=DiPS (OAIS compliant digital reservation solution)
|homepage=https://go.sergroup.com/dips-kommunal
|formats_in=XML, XDOMEA
|formats_out=XML
|function=Access, Active Data Storage, File Format Identification, File Format Migration, File Management, Metadata Extraction, Preservation System, Secure Deletion, Service, Storage, Transfer, Validation, Workflow
|content=Audio, Binary Data, Container, Document, Image, Metadata, Video
}}
{{Infobox tool details}}
== Description ==
DiPS is an OAIS compliant long-term archive solution based on the ECM system from SER Group.
It uses PREMIS to store technical object information (format information, hash values, file sizes, etc.), events (pre-ingest, ingest, conversions, etc.), agents (natural persons and systems) and relationships between objects.

DiPS (Digital Preservation Solution)

2023-04-24T08:51:04Z

172.104.134.96:

{{Infobox tool
|purpose=DiPS is an OAIS compliant long-term archive solution based on the ECM system from SER Group.
|homepage=https://go.sergroup.com/dips-kommunal
|formats_in=XML, XDOMEA
|formats_out=XML
|function=Access, Active Data Storage, File Format Identification, File Format Migration, File Management, Metadata Extraction, Preservation System, Secure Deletion, Service, Storage, Transfer, Validation, Workflow
|content=Audio, Binary Data, Container, Document, Image, Metadata, Video
}}
{{Infobox tool details}}
== Description ==
DiPS is an OAIS compliant long-term archive solution based on the ECM system from SER Group.
It uses PREMIS to store technical object information (format information, hash values, file sizes, etc.), events (pre-ingest, ingest, conversions, etc.), agents (natural persons and systems) and relationships between objects.

DiPS (Digital Preservation Solution)

2023-04-24T08:49:20Z

172.104.134.96: Created page with "{{Infobox tool |purpose=DiPS is an OAIS compliant long-term archive solution based on the ECM system from SER Group. |homepage=https://go.sergroup.com/dips-kommunal |formats_i..."

{{Infobox tool
|purpose=DiPS is an OAIS compliant long-term archive solution based on the ECM system from SER Group.
|homepage=https://go.sergroup.com/dips-kommunal
|formats_in=XML, XDOMEA
|formats_out=XML
|function=Access, Active Data Storage, File Format Identification, File Format Migration, File Management, Metadata Extraction, Preservation System, Secure Deletion, Service, Storage, Transfer, Validation, Workflow
|content=Audio, Binary Data, Container, Document, Image, Metadata, Video
}}
{{Infobox tool details}}
== Description ==


== User Experiences ==

Fq

2023-01-02T09:36:55Z

172.104.134.96:

{{Infobox tool
|purpose=Tool, language and decoders for working with binary data.
|homepage=https://github.com/wader/fq
|sourcecode=https://github.com/wader/fq
|license=MIT and BSD
|cost=Free and open source
|platforms=macOS, Linux and Winodows
|language=golang
|formats_in=CSV (Comma Separated Values), XML, PNG, TIFF, SVG, JPEG, GIF, JSON, YAML, TOML, MP4, MP3, Matroka
|formats_out=XML, JSON, YAML, TOML, CSV
|function=Access, Validation, Binary & Hexidecimal Editing, Discovery, Repair, Quality Assurance, Policy, File Format Identification, File Recovery, Forensic, Metadata Extraction
|content=Binary Data, Container, Metadata
}}
{{Infobox tool details}}
== Description ==

fq is inspired by the well known jq tool and language and allows you to work with binary formats the same way you would using jq. In addition it can present data like a hex viewer, transform, slice and concatenate binary data. It also supports nested formats and has an interactive REPL with auto-completion.

It was originally designed to query, inspect and debug media codecs and containers like mp4, flac, mp3, jpeg. But has since then been extended to support a variety of formats like executables, packet captures (with TCP reassembly) and serialization formats like JSON, YAML, XML, ASN1 BER, Avro, CBOR, protobuf. In addition it also has functions to work with URL:s, convert to/from hex, number bases, search for things etc.

In summary it aims to be jq, hexdump, dd and gdb for files combined into one.

== User Experiences ==

* [https://www.youtube.com/watch?v=GJOq_b0eb-s Binary Tools Summit 2022]
* [https://www.youtube.com/watch?v=-Pwt5KL-xRs&t=1450s NTTW (No Time To Wait) 2022]

= Development Activity =

All development activity is visible on GitHub: https://github.com/wader/fq/commits

=== Release Feed ===
Below are the the latest FQ releases:
<rss max=5>https://github.com/wader/fq/releases.atom</rss>

Bad Peggy

2022-11-30T09:40:15Z

172.104.134.96: corrected typo

{{Infobox tool
|image=BadPeggy.png
|purpose=Scans for damaged images and photos.
|homepage=https://www.coderslagoon.com/#/product/badpeggy
|license=GPLv3
|platforms=Windows, Linux, OSX
|formats_in=JPEG, PNG, BMP, GIF
|function=Quality Assurance, Validation
|content=Image
}}
{{Infobox tool details}}
== Description ==

Bad Peggy scans images ([[JPEG]], [[PNG]], [[BMP]], [[GIF]]) for damages and other blemishes, and shows the results and files instantly. It enables you to find such broken files quickly, inspect and then either delete or move them to a different location.

Requires Java 6 or higher.

Licensed under the GPLv3.

Quoted from the documentation:
"Bad Beggy uses the Java Image IO (JIIO) library to examine image files. Its decoder emits warnings and errors while an image gets loaded. Thus the results do depend on it being up-to-date and also its changes in functionality. Bad Peggy checks though on startup if in general, well-known errors in images do get detected, i.e. if JIIO is still functioning in detecting damaged images as expected. What "damaged" truly means depends and can be
*small difference from the official format, e.g. extra data appended after the actual image.
*non-critical issues like unknown values, which do not affect displaying the image at all.
*minor damage which only disturb smaller parts of the image.
*major damage, which causes the display to be corrupted after a particular position.
*completely truncated or i.e. incomplete images.
*errors at the beginning of the files, so that decoding can't even commence.
*files with are not images at all, but accidentally carry the file extension.
*image files which don't get recognized b the JIIO, but can be processed by other image viewers, e.g. if additional information is stored before the *image data starts (which smarter or more aggressive decoders then skip).
*an image which looks damaged because it got loaded as such, and saved again in another application - and thus is structurally fine.
*an image which is logically damaged but does not cause complains by the JIIO, although the flaws are clearly visible - this is one of the most problematic cases, since such files won't be detected by Bad Peggy - detection for such problems is difficult, you can compare this with a text editor loading and displaying a file with the word "text pr{cessor" in the, where the 'a' to '{' change was caused by a faulty transmission but the text *still makes sense to the editor itself.
In general it is not recommended to just discard every image reported as damaged but to check out if repairing or re-saving the file in other applications into a generally valid image format is possible."

== User Experiences ==

* '''KOST-CECO:''' Used in [[KOST-Val]] for the JPEG validation module. [[KOST-Val]] evaluates the error message "Not a JPEG file" further.
* '''Error detection of JPEG files with JHOVE and Bad Peggy:''' http://openpreservation.org/blog/2016/11/29/jpegvalidation/ (December 2016)

* To validate other file formats than JPEG, make sure the file extension (e. g. gif, png, bmp) appears on the List "options" -> "file extension". If it does not appear there, maybe you should re-download directly from '''Coderslagoon''' https://www.coderslagoon.com/#/product/badpeggy

== Development Activity ==


New in Version 2.0:
* Support for PNG, BMP and GIF images.
* Simplified status bar.
* Visual error differentiation changed to grayscale.
* Error differentiation now in done in gray tones.
* Message box button text is now translated.
* Minor bug fixes and cosmetic changes.

Bad Peggy Sources: https://www.coderslagoon.com/files/badpeggy20_src.tar.xz

Bagger

2022-11-25T14:28:19Z

172.104.134.96: /* Activity Feed */

{{Infobox tool
|purpose=GUI application to facilitate the creation and verification of [[BagIt]] bags.
|homepage=https://github.com/LibraryOfCongress/bagger
|license=Open License, but relies on several Apache based and GNU licensed components.
|function=Fixity, Transfer
}}
{{Infobox tool details
|ohloh_id=Bagger
}}
= Description =
The [[BagIt]] specification is a hierarchical file packaging format for the creation of standardised digital containers called 'bags,' which are used for storing and transferring digital content. Derived from work by the Library of Congress and the California Digital Library, a bag consists of a ‘payload’ - the digital content - and ‘tags' - metadata files to document the storage and transfer of the bag. There are a number of [http://sourceforge.net/projects/loc-xferutils/ Bagit-specific tools] to ease bag creation, including the BagIt Library, a Java-based software library to support the creation, manipulation, and validation of bags. For those less comfortable with command-line interface, the Bagger application provides a graphical user interface to the BagIt Library.
====Provider====
The United States Library of Congress, and the National Digital Information Infrastructure and Preservation Program (NDIIPP)
====Licensing and cost====
[http://www.linfo.org/bsdlicense.html BSD License] - free.  The BagIt Library is public domain.
====Development activity====
Bagger 2.1.2 was released in February 2012. BagIt Library 4.1 was released in January 2012.
The Library of Congress website implies ongoing development of the Transfer Utilities. 
====Platform and interoperability====
Both the BagIt Library and Bagger require Java 6.
====Functional notes====
Bags contain at minimum three elements: a ‘payload’ and at least two ‘tags.’ The payload consists of the content being preserved. The first tag is a manifest itemising the files making up the content along with their checksums; the second is a bagit.txt file identifying the container as a bag and giving the version of the specification used and the character encoding of the tags.  The specification additionally allows for several optional tags.  
====Documentation and user support====
Documentation is extremely sparse, primarily consisting of README files detailing release notes. The [http://www.digitalpreservation.gov/documents/bagitspec.pdf BagIt specification itself ]can be found through the Library of Congress website.
It appears that the main user support consists of a mailing list hosted by Sourceforge; however, the list archive only shows 11 posts for 2011.
====Usability====
The BagIt Library uses a command-line interface, while Bagger provides a graphical user interface. No installation is required; the tools can simply be downloaded and run, although it may not be immediately clear to users how to do so.
====Expertise required====
BagIt is designed to create a common language for users exchanging digital materials, essentially negating the need for expertise about others’ protocols. However, for configuration, familiarity with one’s own repository’s technical protocols is essential.
====Standards compliance====
The BagIt specification is an Internet Engineering Task Force (IETF) internet draft.
====Influence and take-up====
The BagIt specification has become widely accepted in the preservation community, and is used by the Library of Congress, Chronopolis, and The Stanford Digital Repository, among others. The Transfer Utilities have been downloaded nearly 4000 times from Sourceforge.

= User Experiences =

= Development Activity =

=== Activity Feed===
Link to any RSS feed that is updated when issue or code updates occur, if any, e.g:
<rss max=7>https://github.com/LibraryOfCongress/bagger/commits/master.atom</rss>

=== Release Feed ===

Siegfried

2022-11-25T14:24:46Z

172.104.134.96: /* Activity Feed */

{{Infobox tool
|purpose=A PRONOM based, command line, file format identification tool using Aho Corasick matching and no buffer limits.
|homepage=http://www.itforarchivists.com/siegfried
|license=Apache License 2.0
|function=File Format Identification
}}
{{Infobox tool details}}
== Description ==

Siegried is a file format identification tool that, like DROID and Fido, is based on PRONOM. However, it uses a different pattern matching algorithm that offers different strengths and weaknesses to those other PRONOM based tools. A detailed description of the tool and why it was created can be found in [http://www.openplanetsfoundation.org/blogs/2014-09-27-siegfried-pronom-based-file-format-identification-tool this blog post].
Besides, there is a more detailed description in terms of functionality on the [https://github.com/richardlehane/siegfried Github Page of Siegfried].

Siegfried was first publicly released the 28th february 2014 in version 0.1.

== User Experiences ==

* '''ZBW:'''
** The command line tool is very easy to handle. The default output is yaml. This can be changed to csv or json.
***Usual command: sf file.ext (will output in yaml)
***Change output to csv: sf -csv file.ext
***Change output to json: sf -json file.ext
**It is also possible to save the output in an external file:
***sf file.ext >output.yml
***sf -csv file.ext >output.csv
***sf -json file.ext >output.json

== Development Activity ==

All development activity is visible on GitHub: http://github.com/richardlehane/siegfried/commits

=== Release Feed ===
Below the last 3 release feeds:
<rss max=3>https://github.com/richardlehane/siegfried/releases.atom</rss>

=== Activity Feed ===
Below the last 5 commits:
<rss max=5>https://github.com/richardlehane/siegfried/commits/main.atom</rss>

Rosetta

2022-11-13T17:42:19Z

172.104.134.96: /* Description */

{{Infobox tool
|purpose=Ex Libris Rosetta enables institutions to preserve and provide access to the collections in their care.
|homepage=http://www.exlibrisgroup.com/category/RosettaOverview
|license=Commercially licensed product.
|formats_in=METS (Metadata Encoding and Transmission Standard), PREMIS (Preservation Metadata Implementation Strategies)
|formats_out=METS (Metadata Encoding and Transmission Standard), PREMIS (Preservation Metadata Implementation Strategies)
|function=Access, File Format Migration, Metadata Processing, Preservation System
}}
{{Infobox tool details}}
== Description ==
Designed in collaboration with the National Library of New Zealand and reviewed by an international peer group of recognized leaders and innovators, Ex Libris Rosetta enables institutions to preserve and provide access to the collections in their care, now and in the future.

Rosetta is a complete preservation solution. Its focus is to archive and preserve the digitized and born digital materials stored at academic and memory institutions like libraries and archives, research organizations, and government institutions.
It aims to ensure data integrity and access over time for the archived digital data.

Rosetta supports the acquisition, validation, ingest, storage, preservation, and dissemination of digital objects that are in various formats and originate from many sources.

Rosetta uses standards like Premis and METS. The METS profile of Ex Libris is published and open. The data model of Rosetta is as follows:

* intellectual entity (coherent set of content, the whole unit like e. g. a digitized book)
* representation (the set of files, including all the metadata)
* file
* bit-stream

Tools integrated in Rosetta are as follows:
* BIRT (open source, eclipse-based Business Intelligence and Reporting Tools reporting system)
* thanks to a SDK, Rosetta users can easily build their own tools, submission applications and plug-ins
* Pronom connection

Rosetta is an OAIS-compliant solution.

== User Experiences ==


== Development Activity ==

Rosetta

2022-11-13T17:40:35Z

172.104.134.96: /* Description */

{{Infobox tool
|purpose=Ex Libris Rosetta enables institutions to preserve and provide access to the collections in their care.
|homepage=http://www.exlibrisgroup.com/category/RosettaOverview
|license=Commercially licensed product.
|formats_in=METS (Metadata Encoding and Transmission Standard), PREMIS (Preservation Metadata Implementation Strategies)
|formats_out=METS (Metadata Encoding and Transmission Standard), PREMIS (Preservation Metadata Implementation Strategies)
|function=Access, File Format Migration, Metadata Processing, Preservation System
}}
{{Infobox tool details}}
== Description ==
Designed in collaboration with the National Library of New Zealand and reviewed by an international peer group of recognized leaders and innovators, Ex Libris Rosetta enables institutions to preserve and provide access to the collections in their care, now and in the future.

Rosetta is a complete preservation solution. Its focus is to archive and preserve the digitized and born digital materials stored at academic and memory institutions like libraries and archives, research organizations and government institutions.
It aims to ensure data integrity and access over time for the archived digital data.

Rosetta supports the acquisition, validation, ingest, storage, preservation and dissemination of digitale objects that are in various formats and originate from many sources.

Rosetta uses standards like Premis and Mets. The METS profile of Ex Libris is published and open. The data model of Rosetta is as following:

* intellectual entity (coherent set of content, the whole unit like e. g. a digitized book)
* representation (the set of files, including all the metadata)
* file
* bit-stream

Tools integrated in Rosetta
* BIRT (open source, eclipse-based Business Intelligence and Reporting Tools reporting system)
* thanks to a SDK, Rosetta users can easily build their own tools, submission applications and plug-ins
* Pronom connection

Rosetta is an OAIS-compliant solution.

== User Experiences ==


== Development Activity ==

AudiAnnotate

2022-11-02T20:56:09Z

172.104.134.96:

{{Infobox tool
|purpose=To make audio and its interpretations more discoverable and usable by extending the use of the newest IIIF (International Image Interoperability Framework) standard for audio with the development of the AudiAnnotate web application, documented workflows and workshops that will facilitate the use of existing best-of-breed, open source tools for audio annotation
(Sonic Visualiser), for public code and document repositories (GitHub), and audio presentation (Universal Viewer) to produce, publish, and sustain shareable W3C Web Annotations for individual and collaborative audio projects.
|homepage=http://audiannotate.brumfieldlabs.com
|cost=None
|function=Academic Social Networking, Access, Annotation, Personal Archiving, Preservation System, Service, Version Control, Workflow, Metadata Processing, Rendering, Discovery, Persistent Identification, Managing Active Research Data
|content=Audio, Video
}}
{{Infobox tool details}}
== Description ==
The AudiAnnotate project originates from the premise that facilitating the annotation of audio collections will accelerate access to, promote scholarship with, and extend our understanding of important audio collections, some of which may be currently inaccessible and others which could potentially be lost forever. Audio collections are not discoverable without annotations. If we cannot discover an audio file, we will not use it in scholarship. If we do not use audio collections, libraries and archives that hold massive collections of audio recordings from a diverse range of bygone timeframes, cultures, and contexts will not preserve them.

Broadly speaking, the application and workflows that we will develop in the AudiAnnotate project will help users to translate their own analyses of audio recordings into media annotations that will be publishable as easy-to-maintain, static, W3C Web Annotations associated with IIIF manifests and hosted in a GitHub repository that are viewable through presentation software such as Universal Viewer.

In response to the need for a workflow that supports IIIF manifest creation, collaborative editing, flexible modes of presentation, and permissions control, the AudiAnnotate project is developing AWE, a documented workflow using the recently adopted IIIF standard for AV materials that will help libraries, archives, and museums (LAMs), scholars, and the public access and use AV cultural heritage items. We will achieve this goal by connecting existing best-of-breed, open source tools for AV management (Aviary), annotation (such as Audacity and OHMS), public code and document repositories (GitHub), and the AudiAnnotate web application for creating and sharing IIIF manifests and annotations. Usually limited by proprietary software and LAM systems with restricted access to AV, users will use AWE as a complete sequence of tools and transformations for accessing, identifying, annotating, and sharing AWE “projects” such as singular pages or multi-page exhibits or editions with AV materials. LAMs will benefit from AWE as it facilitates metadata generation, is built on W3C web standards in IIIF for sharing online scholarship, and generates static web pages that are lightweight and easy to preserve and harvest. AWE represents a new kind of AV ecosystem where the exchange is opened between institutional repositories, annotation software, online repositories and publication platforms, and all kinds of users.

The AudiAnnotate Project has been awarded a 2019 Digital Extension Grant from the American Council of Learned Societies. AWE has been generously funded by the Andrew W. Mellon Foundation.

== User Experiences ==

AudiAnnotate

2022-11-02T20:50:32Z

172.104.134.96:

{{Infobox tool
|purpose=To make audio and its interpretations more discoverable and usable by extending the use of the newest IIIF (International Image Interoperability Framework) standard for audio with the development of the AudiAnnotate web application, documented workflows and workshops that will facilitate the use of existing best-of-breed, open source tools for audio annotation
(Sonic Visualiser), for public code and document repositories (GitHub), and audio presentation (Universal Viewer) to produce, publish, and sustain shareable W3C Web Annotations for individual and collaborative audio projects.
|homepage=http://audiannotate.brumfieldlabs.com
|cost=None
|function=Access, Annotation, Preservation System, Workflow, Academic Social Networking, Personal Archiving, Service, Version Control
|content=Audio, Video
}}
{{Infobox tool details}}
== Description ==
The AudiAnnotate project originates from the premise that facilitating the annotation of audio collections will accelerate access to, promote scholarship with, and extend our understanding of important audio collections, some of which may be currently inaccessible and others which could potentially be lost forever. Audio collections are not discoverable without annotations. If we cannot discover an audio file, we will not use it in scholarship. If we do not use audio collections, libraries and archives that hold massive collections of audio recordings from a diverse range of bygone timeframes, cultures, and contexts will not preserve them.

Broadly speaking, the application and workflows that we will develop in the AudiAnnotate project will help users to translate their own analyses of audio recordings into media annotations that will be publishable as easy-to-maintain, static, W3C Web Annotations associated with IIIF manifests and hosted in a GitHub repository that are viewable through presentation software such as Universal Viewer.

In response to the need for a workflow that supports IIIF manifest creation, collaborative editing, flexible modes of presentation, and permissions control, the AudiAnnotate project is developing AWE, a documented workflow using the recently adopted IIIF standard for AV materials that will help libraries, archives, and museums (LAMs), scholars, and the public access and use AV cultural heritage items. We will achieve this goal by connecting existing best-of-breed, open source tools for AV management (Aviary), annotation (such as Audacity and OHMS), public code and document repositories (GitHub), and the AudiAnnotate web application for creating and sharing IIIF manifests and annotations. Usually limited by proprietary software and LAM systems with restricted access to AV, users will use AWE as a complete sequence of tools and transformations for accessing, identifying, annotating, and sharing AWE “projects” such as singular pages or multi-page exhibits or editions with AV materials. LAMs will benefit from AWE as it facilitates metadata generation, is built on W3C web standards in IIIF for sharing online scholarship, and generates static web pages that are lightweight and easy to preserve and harvest. AWE represents a new kind of AV ecosystem where the exchange is opened between institutional repositories, annotation software, online repositories and publication platforms, and all kinds of users.

The AudiAnnotate Project has been awarded a 2019 Digital Extension Grant from the American Council of Learned Societies. AWE has been generously funded by the Andrew W. Mellon Foundation.

== User Experiences ==

AudiAnnotate

2022-11-02T19:43:38Z

172.104.134.96:

{{Infobox tool
|purpose=To make audio and its interpretations more discoverable and usable by extending the use of the newest IIIF (International Image Interoperability Framework) standard for audio with the development of the AudiAnnotate web application, documented workflows and workshops that will facilitate the use of existing best-of-breed, open source tools for audio annotation
(Sonic Visualiser), for public code and document repositories (GitHub), and audio presentation (Universal Viewer) to produce, publish, and sustain shareable W3C Web Annotations for individual and collaborative audio projects.
|homepage=http://audiannotate.brumfieldlabs.com
|cost=None
|function=Access, Annotation, Preservation System, Workflow
|content=Audio, Video
}}
{{Infobox tool details}}
== Description ==
The AudiAnnotate project originates from the premise that facilitating the annotation of audio collections will accelerate access to, promote scholarship with, and extend our understanding of important audio collections, some of which may be currently inaccessible and others which could potentially be lost forever. Audio collections are not discoverable without annotations. If we cannot discover an audio file, we will not use it in scholarship. If we do not use audio collections, libraries and archives that hold massive collections of audio recordings from a diverse range of bygone timeframes, cultures, and contexts will not preserve them.

Broadly speaking, the application and workflows that we will develop in the AudiAnnotate project will help users to translate their own analyses of audio recordings into media annotations that will be publishable as easy-to-maintain, static, W3C Web Annotations associated with IIIF manifests and hosted in a GitHub repository that are viewable through presentation software such as Universal Viewer.

In response to the need for a workflow that supports IIIF manifest creation, collaborative editing, flexible modes of presentation, and permissions control, the AudiAnnotate project is developing AWE, a documented workflow using the recently adopted IIIF standard for AV materials that will help libraries, archives, and museums (LAMs), scholars, and the public access and use AV cultural heritage items. We will achieve this goal by connecting existing best-of-breed, open source tools for AV management (Aviary), annotation (such as Audacity and OHMS), public code and document repositories (GitHub), and the AudiAnnotate web application for creating and sharing IIIF manifests and annotations. Usually limited by proprietary software and LAM systems with restricted access to AV, users will use AWE as a complete sequence of tools and transformations for accessing, identifying, annotating, and sharing AWE “projects” such as singular pages or multi-page exhibits or editions with AV materials. LAMs will benefit from AWE as it facilitates metadata generation, is built on W3C web standards in IIIF for sharing online scholarship, and generates static web pages that are lightweight and easy to preserve and harvest. AWE represents a new kind of AV ecosystem where the exchange is opened between institutional repositories, annotation software, online repositories and publication platforms, and all kinds of users.

The AudiAnnotate Project has been awarded a 2019 Digital Extension Grant from the American Council of Learned Societies. AWE has been generously funded by the Andrew W. Mellon Foundation.

== User Experiences ==

AudiAnnotate

2022-11-02T19:42:48Z

172.104.134.96: summary

{{Infobox tool
|purpose=To make audio and its interpretations more discoverable and usable by extending the use of the newest IIIF (International Image Interoperability Framework) standard for audio with the development of the AudiAnnotate web application, documented workflows and workshops that will facilitate the use of existing best-of-breed, open source tools for audio annotation
(Sonic Visualiser), for public code and document repositories (GitHub), and audio presentation (Universal Viewer) to produce, publish, and sustain shareable W3C Web Annotations for individual and collaborative audio projects.
|homepage=http://audiannotate.brumfieldlabs.com
|cost=0
|function=Access, Annotation, Preservation System, Workflow
|content=Audio, Video
}}
{{Infobox tool details}}
== Description ==
The AudiAnnotate project originates from the premise that facilitating the annotation of audio collections will accelerate access to, promote scholarship with, and extend our understanding of important audio collections, some of which may be currently inaccessible and others which could potentially be lost forever. Audio collections are not discoverable without annotations. If we cannot discover an audio file, we will not use it in scholarship. If we do not use audio collections, libraries and archives that hold massive collections of audio recordings from a diverse range of bygone timeframes, cultures, and contexts will not preserve them.

Broadly speaking, the application and workflows that we will develop in the AudiAnnotate project will help users to translate their own analyses of audio recordings into media annotations that will be publishable as easy-to-maintain, static, W3C Web Annotations associated with IIIF manifests and hosted in a GitHub repository that are viewable through presentation software such as Universal Viewer.

In response to the need for a workflow that supports IIIF manifest creation, collaborative editing, flexible modes of presentation, and permissions control, the AudiAnnotate project is developing AWE, a documented workflow using the recently adopted IIIF standard for AV materials that will help libraries, archives, and museums (LAMs), scholars, and the public access and use AV cultural heritage items. We will achieve this goal by connecting existing best-of-breed, open source tools for AV management (Aviary), annotation (such as Audacity and OHMS), public code and document repositories (GitHub), and the AudiAnnotate web application for creating and sharing IIIF manifests and annotations. Usually limited by proprietary software and LAM systems with restricted access to AV, users will use AWE as a complete sequence of tools and transformations for accessing, identifying, annotating, and sharing AWE “projects” such as singular pages or multi-page exhibits or editions with AV materials. LAMs will benefit from AWE as it facilitates metadata generation, is built on W3C web standards in IIIF for sharing online scholarship, and generates static web pages that are lightweight and easy to preserve and harvest. AWE represents a new kind of AV ecosystem where the exchange is opened between institutional repositories, annotation software, online repositories and publication platforms, and all kinds of users.

The AudiAnnotate Project has been awarded a 2019 Digital Extension Grant from the American Council of Learned Societies. AWE has been generously funded by the Andrew W. Mellon Foundation.

== User Experiences ==

EPADD

2022-10-24T04:35:20Z

172.104.134.96:

{{Infobox tool
|image=Epadd_logo_orig.png
|purpose=ePADD is a software package developed by Stanford University's Special Collections & University Archives that supports archival processes around the appraisal, ingest, processing, discovery, and delivery of email archives.
|homepage=https://library.stanford.edu/projects/epadd
|license=Apache 2.0
|platforms=Java
|Wikidata ID=Q59652265
|formats_out=PREMIS (Preservation Metadata Implementation Strategies)
|function=Access, Appraisal, Content Profiling, Metadata Extraction, Metadata Processing
|content=Email
}}
{{Infobox tool details
|ohloh_id=epadd
}}




== Description ==

From the [https://github.com/ePADD/epadd project Github page]:
"ePADD is a software package developed by Stanford University's Special Collections & University Archives that supports archival processes around the appraisal, ingest, processing, discovery, and delivery of email archives.

The software is comprised of four modules:

* '''Appraisal:''' Allows donors, dealers, and curators to easily gather and review email archives prior to transferring those files to an archival repository.

* '''Processing:''' Provides archivists with the means to arrange and describe email archives.

* '''Discovery:''' Provides the tools for repositories to remotely share a redacted view of their email archives with users through a web server discovery environment. (Note that this module is downloaded separately).

* '''Delivery:''' Enables archival repositories to provide moderated full-text access to unrestricted email archives within a reading room environment."

From the [https://library.stanford.edu/projects/epadd/ Project page]:

'''ePADD Technical Information'''

ePADD is written in Java and Javascript and powered by Apache Tomcat (v7.0) using Java EE Servlet API (v3.x) and Java Mail (v1.4.2). Text and metadata extraction, indexing and retrieval is performed by Apache Lucene (v4.7) and Apache Tika (v1.8). Charting and visualization is supported using the D3-based reusable chart library (v0.4.10). Oracle's Java Application Bundler and Launch4J are used for packaging on Mac and Windows platforms respectively. Other Java libraries from Apache (Lang, commons, CLI, IO, logging, etc.) are also used. JSON formatting is performed with the libraries org.json and Gson.

ePADD has implemented its own natural language processing (NLP) toolkit which is used for named entity extraction, disambiguation and other tasks. This toolkit supplants the Apache OpenNLP used in earlier beta versions of the ePADD software. We continue to use Muse as an internal library within ePADD. However, the Apache OpenNLP proved insufficient for our needs (at least for name recognition), and after various rounds of customization, we built our own named entity recognizer. This toolkit uses external datasets such as Wikipedia/DBpedia, Freebase, Geonames, OCLC FAST and LC Subject Headings/LC Name Authority File.

The project is developed with IDEs like IntelliJ Idea and Eclipse, built with Apache Maven, Ant, and custom shell scripts, and tracked using Git for source control and issue tracking. The ePADD software client is browser-based and compatible with Chrome and Firefox. It is optimized for Windows 7 SP1/10, OSX 10.12/10.13, and Ubuntu 16.04 machines, using Java 8.

== User Experiences ==

* On migrating from different email formats before ingest to ePADD https://groups.google.com/forum/#!topic/digital-curation/srt-oIVwAGU
* [https://twitter.com/e_padd?lang=en ePADD on Twitter]
* [http://library.stanford.edu/blogs/special-collections-unbound/2018/07/epadd-60-beta-released ePADD 6.0 beta released!]
* [https://docs.google.com/document/d/1CVIpWK5FNs5KWVHgvtWTa7u0tZjUrFrBHq6_6ZJVfEA ePADD User Guide]
* [https://docs.google.com/document/d/10U9Hxh9MS9C9bS8M7uYuXk5m7EBFpgd0yiwCcOSM6D ePADD Shared Discovery Module Website Collection Contributor Guide]
* [https://library.stanford.edu/projects/epadd/presentations-publications Full list of presentations and publications]

== Development Activity ==

All development activity is visible on GitHub: http://github.com/ePADD/epadd/commits

=== Release Feed ===
Below the last 3 release feeds:
<rss max=3>https://github.com/ePADD/epadd/releases.atom</rss>

=== Activity Feed ===
Below the last 5 commits:
<rss max=5>https://github.com/ePADD/epadd/commits/master.atom</rss>

Minimum Preservation Tool

2022-09-15T19:13:44Z

172.104.134.96:

{{Infobox tool
|image=MPT_horizontal_no_logo.png
|purpose=The Minimum Preservation Tool (MPT) can be used to create an interim preservation storage environment for files awaiting preservation in a longer term repository solution. It supports checksum generation, fixity checking, and replication across two or more storage nodes.
|homepage=https://github.com/britishlibrary/mpt
|license=http://www.apache.org/licenses/LICENSE-2.0
|function=Fixity, Preservation System
}}
{{Infobox tool details}}
== Description ==
The MPT was designed to make use of existing network storage and compute resources available at most institutions. It is intended to provide greater protection to digital content than a standard network storage offering. The key differences between an MPT solution and standard network storage are replication of content across two or more storage “nodes” and regular fixity checking (checksum validation) of all files on all nodes to help ensure content remains authentic and unchanged.

Papers & Posters:
Back to Basics: The Minimum Preservation Tool (iPRES2021) - short paper. Pennock, Maureen; Beaman, John; May, Peter; Davies, Kevin
https://zenodo.org/record/5788586#.YyN4Q3bMJhE

Upscaling the MPT (iPRES2022)- poster. May, Peter; Davies, Kevin (forthcoming)

== User Experiences ==
Introduction to the MPT https://www.dpconline.org/blog/minimum-preservation-tool-mpt

== Development Activity ==
https://github.com/britishlibrary/mpt - Main site

https://github.com/anjackson/mpt/releases/tag/v1.1.6-UI - Experimental MPT user interface from @anjacks0n

SIARD

2022-09-13T13:37:31Z

172.104.134.96:

{{Infobox format
|Wikidata ID=Q2206173
|File formats wiki ID=SIARD
|PRONOM PUID=fmt/161, fmt/995, fmt/1196
}}