Ingest

From COPTR
Jump to navigation Jump to search
Lifecycle stage definition: Functions that support the DCC Lifecycle stage defined as "Transfer data to an archive, repository, data centre or other custodian. Adhere to documented guidance, policies or legal requirements."
Lifecycle order: 2

Functions within this lifecycle stage

FuntionDefinition
Dependency AnalysisTools for identifying essential information that resides externally to a digital object, or for identifying dependent processes such as which DLLs are required by a Windows process.
Encryption DetectionTools that support the detection of encryption or password protection in files.
File Format IdentificationTools that enable the automatic identification of the file format of a particular file, typically by examining characteristic codes (often termed file format magic) in the file header.
FixityTools that support the verification of file fixity, typically through the generation and validation of checksum based manifests.
Metadata ExtractionTools that support the extraction of metadata from files.
Metadata ProcessingTools that support the processing or management of metadata.
Persistent IdentificationTools that support the unique and persistent identification of files or intellectual entities.
Quality AssuranceTools that support quality checking of digital resources, identifying damaged, incomplete or low quality data. Typically used to identify damage introduced via processes such as format migration or digitisation.
ValidationTools that support the validation of digital files, typically against a file format specification.

Tools for this lifecycle stage

ToolFunctionPurpose
3-Heights(TM) PDF ValidatorValidation3-Heights(TM) PDF Validator from PDF-Tools AG.
7-ZipRendering
Transfer
Fixity
7-Zip is a file archiver with a high compression ratio, and encryption and fixity check capabilities
7trainMetadata ProcessingXSLT 2.0 tool for generating METS files from XML input
ACE (Audit Control Environment)FixityThe Auditing Control Environment is a mature set of software designed to help libraries and archives prove their holdings are intact and trustworthy.
ADC TestValidationTests and reports on audio analog-to-digital converters
ARK plugin for OmekaPersistent IdentificationGenerating and resolving ARK identifiers for resources in Omeka
AVP FixityFixityFixity monitoring for digital collections
Aaru Data Preservation SuiteBackup
Disk Imaging
Metadata Extraction
Media dump software and disc image manager
Adobe Photoshop ElementsMetadata Extraction
Metadata Processing
Personal Archiving
A commercial image editor with a metadata module (Organizer).
Apache PDFBoxMetadata Extraction
Repair
File Format Migration
Validation
Encryption Detection
JAVA PDF library for creation, manipulation, validation and content extraction of PDF documents
Apache POI - the Java API for Microsoft DocumentsFile Format Migration
Metadata Extraction
Encryption Detection
The Apache POI Project's mission is to create and maintain Java APIs for manipulating various file formats based upon the Office Open XML standards (OOXML) and Microsoft's OLE 2 Compound Document format (OLE2).
Apache TikaMetadata Extraction
File Format Identification
Text Extraction
Embedded File Extraction
Java based tool for identifying file formats using signatures and extracting metadata and text content from documents.
Archive::BagItFixity
File Copy
BagIt API for Perl
ArchivesSpaceAccess
Metadata Processing
ArchivesSpace is the next-generation web-based archives information management system, designed by archivists and supported by diverse archival repositories.
Archivists' ToolkitMetadata ProcessingAn open source archival data management system to provide broad, integrated support for the management of archives
ArchonAccess
Metadata Processing
Archon automatically publishes archival descriptive information and digital archival objects in a user-friendly website.
Ark servicePersistent IdentificationARK identifiers generator in python
AsTiffTagViewerQuality AssuranceAsTiffTagViewer is a TIFF Tag Viewer application.
BIL (BagIt Library)Fixity
File Copy
BagIt Library is a Java software library that supports the creation, manipulation and validation of bags.
BWF MetaEditMetadata Extraction
Metadata Processing
Validation
BWF MetaEdit permits embedding, validating, and exporting of metadata in Broadcast WAVE Format (BWF) files.
Bad PeggyValidation
Quality Assurance
Scans for damaged images and photos.
BagIt Transfer UtilitiesFixity
File Copy
BagIt transfer Utilities are a collection of tools developed for the purpose of validation and transfer of bags.
BaggerFixity
Transfer
GUI application to facilitate the creation and verification of BagIt bags.
BitCuratorFile Management
Fixity
Metadata Extraction
Metadata Processing
Quality Assurance
Validation
Workflow
The BitCurator Environment is an Ubuntu Linux distribution geared to the needs of archivists and librarians. It includes a suite of open source digital forensics and data analysis tools to help collecting institutions process born-digital materials.
BnL Mets ExporterMetadata ProcessingCommand Line Interface (CLI) to export METS/ALTO documents to other formats.
BrunnhildeMetadata Extraction
Content Profiling
Siegfried-based characterization of directories and disk images
C3POContent Profiling
Metadata Extraction
C3PO is a content profiling tool for visualization and preservation analysis
CSV ValidatorMetadata Processing
Validation
Validation of CSV files against user-defined schema
Checkit tiffQuality Assurance
Validation
a tool to validate TIFF files against given configuration profile
Checksum (by Corz)FixityFast hashing tool using a GUI interface
Cksum Unix commandFixitycksum computes a cyclic redundancy check (CRC) checksum for each given file, or standard input if none are given
ClocFile Format IdentificationCloc (Count Lines of Code) serves not only to count the lines of Code,but also guesses the programming language, thus can be used to identify files. It is a command line tool which is easy to use.
CollectusMetadata ProcessingThe UVa Library's Collectus digital object collector tool allows users to to collect image or text objects from a repository.
ContextMinerMetadata Processing
Web Capture
ContextMiner is a framework to collect, analyze, and present the contextual information along with the data.
Crazy-fast-image-scanFile Format Identification
Content Profiling
Forensic
A script to scan media very quickly to find out what kind of content it contains
Curator's WorkbenchMetadata ProcessingCurator's Workbench is a tool that automates and streamlines the process of preparing collections of digital materials for submission to a repository
CyberChefEncryption Detection
File Management
Decryption
Metadata Extraction
Personal Archiving
Binary & Hexidecimal Editing
Discovery
A forensic tool with workflow capabilities to analyse files and containers
DART (Digital Archivist's Resource Tool)Storage
File Management
Fixity
Transfer
Provides both a GUI and a command-line interface for packaging files and uploading them to remote repositories.
DIMAGMetadata Extraction
Preservation System
Access
File Format Migration
Storage
Workflow
A software suite supporting archives with preservation of digital information for eternity
DIMAG IngestListMetadata Extraction
Transfer
Accompanies ingest process from donor to archive, logs process steps.
DPF ManagerValidationA TIFF validity checker
DROID (Digital Record Object Identification)File Format Identification
Metadata Extraction
DROID (Digital Record Object Identification) is a software tool developed to perform automated batch identification of file formats.
DROID Siegfried Sqlite Analysis EngineMetadata Extraction
Content Profiling
De-Duplication
Analysis and automatic generation of summary information from DROID output
DUMPBIN UtilityMetadata Extraction
File Format Identification
The DUMPBIN utility, which is provided with the 32-bit version of Microsoft Visual C++, combines the abilities of the LINK, LIB, and EXEHDR utilities.
DV AnalyzerQuality Assurance
Metadata Processing
DV Analyzer is a technical quality control and reporting tool that examines DV streams in order to report errors in the tape-to-file transfer process.
DVRescueQuality Assurance
Metadata Processing
DVRescue is archivist-made software that supports data migration from DV tapes into digital files suitable for long-term preservation.
DataCitePersistent Identification
Managing Active Research Data
Citation and Impact Tracking
DataCite works with data centres to assign persistent identifiers to datasets using the Digital Object Identifier (DOI) infrastructure.
DbDIPviewAccess
Quality Assurance
Database Representation Information packaged and used for quick and user-friendly access to information stored in databases. Different combinations of Content Data Objects are supported by automated deployment mechanism.
Dependency Discovery ToolDependency AnalysisThe Dependency Discovery Tool searches through binary office files (.doc, .xls and .ppt) and tries to find any documents or files that are linked to the document.
Directory List & PrintMetadata Extraction
File Management
A universal metadata extractor
DiskFormatIDDisk Imaging
File Format Identification
Identify floppy disk formats from kryoflux stream files
DisktypeMetadata Extraction
Disk Imaging
Tool for detecting the content format of a disk or disk image. It knows about common file systems, partition tables, and boot codes.
DocuTeam PackerMetadata ProcessingCreates and edits SIPs
DocworksOCR
Workflow
Quality Assurance
Document digitization workflow software
Double CommanderFixity
File Copy
Batch Rename
File Management
De-Duplication
Open source file manager with two panels side by side
Duke Data AccessionerFile Copy
File Format Identification
Metadata Extraction
Transfer
Validation
Data Accessioner provides a graphical user interface to aid in migrating data from physical media to a dedicated file server, documenting the process and using MD5 checksums to identify any errors introduced in transfer.
EMET (Embedded Metadata Extraction Tool)Metadata ExtractionEMET is a stand-alone tool designed to extract metadata embedded in JPEG and TIFF files.
EPADDMetadata Processing
Metadata Extraction
Content Profiling
Access
Appraisal
ePADD is a software package developed by Stanford University's Special Collections & University Archives that supports archival processes around the appraisal, ingest, processing, discovery, and delivery of email archives.
EXE ExplorerMetadata ExtractionEXE Explorer reads and displays executable file properties and structure.
EXIF to DC XML normaliserFile Format Migration
Metadata Extraction
Extract EXIF data and normalise it to DC XML.
EZARKPersistent IdentificationARK identifiers management tool and sub-publishers registry
EZIDPersistent IdentificationEZID (easy-eye-dee) makes it easy to create and manage unique, persistent identifiers.
Easy CD-DA ExtractorFile Format Migration
Metadata Extraction
Disk Imaging
Easy CD-DA Extractor is CD Ripper, Music Converter, Audio Converter, Metadata Editor, and CD/DVD burning software.
EchoDep Hub and Spoke Framework Tool SuiteMetadata ProcessingTool suite to manage digital content in multiple repository systems.
EmbARCMetadata Processing
Quality Assurance
internal file metadata management including embedding and validation
EpubCheckValidation
Metadata Extraction
Encryption Detection
Validator for EPUB files
Exact Audio CopyFile Format Migration
Metadata Extraction
Disk Imaging
Exact Audio Copy is an audio grabber for audio CDs using standard CD and DVD-ROM drives on Windows only.
ExactFileFixityMaking sure that what you hash is what you get
ExempiMetadata Extraction
Metadata Processing
Exempi is a library for handling XMP metadata, based on the Adobe XMP SDK
ExifToolMetadata Processing
Metadata Extraction
Repair
Properties extraction, identification, metadata editing
Exiv2Metadata ProcessingExiv2 is a C++ library and a command line utility to manage image metadata.
FCIVFixity
Transfer
Generates and compares MD5 values stored in an XML file.
FFAStransMetadata Extraction
File Format Identification
File Format Migration
Quality Assurance
Workflow
Planning
Task automation engine, mostly used in audio and video visual content management.
FIDO (Format Identification for Digital Objects)Metadata Extraction
File Format Identification
A PRONOM based, command line, file format identification tool written in Python
FITS (File Information Tool Set)File Format Identification
Validation
Metadata Extraction
Encryption Detection
FITS allows data curators to identify, validate, and extract technical metadata for the objects in their digital repository.
File Analyzer and Metadata Harvester V2File Management
Fixity
Metadata Extraction
Metadata Processing
Quality Assurance
Validation
Workflow
The File Analyzer is a general purpose desktop (and command line) tool designed to automate simple, file-based operations. The File Analyzer assembles a toolkit of tasks a user can perform. The tasks that have been written into the File Analyzer code base have been optimized for use by libraries, archives, and other cultural heritage institutions.
File Format Identification PronomFile Format IdentificationPerl API to analyze and handle droid (PRONOM) signatures
FileAlyzerMetadata ExtractionFileAlyzer allows a basic analysis of files (showing file properties and file contents in hex dump form) and is able to interpret common file contents like resources structures (like text, graphics, HTML, media and PE).
FileVerifier++Fixity
De-Duplication
Windows utility for verifying file contents
Fine Free File CommandFile Format IdentificationThis is the home page for the open source implementation of the file(1) command that ships with every free operating system (OpenBSD, Linux, NetBSD, FreeBSD, etc.
FingerdetQuality AssuranceQA tool for detecting fingers on digitised pages
FixiFixityFixi is a command-line utility that indexes, verifies, and updates checksum information for collections of files.
Fixity ProFixityFixity Pro is a desktop application for Windows and Mac that provides simple automated monitoring and reporting on the data integrity of your files that are stored on your computer, removable storage devices, and mounted network storage locations. Use Fixity Pro to schedule routine scans that will tell you if your files have been changed and if any files have been added, removed, or moved/renamed since the last scan that was performed.
FlintValidation
Encryption Detection
Validates a file against a policy, using common validation tools
FreeCommanderFile Management
De-Duplication
Fixity
File Copy
Split-screen file manager with desirable extras
GNU DiffutilsDe-Duplication
Quality Assurance
GNU Diffutils is a package of several programs related to finding differences between files.
GNU libextractorMetadata ExtractionGNU libextractor is a library used to extract meta data from files of arbitrary type.
GeosetterMetadata Extraction
Metadata Processing
Personal Archiving
A tool that sets coordinates and edits all kind of embedded image metadata.
GetID3()Metadata ExtractionExtracts technical and embedded descriptive metadata from common multimedia file formats.
GoobiWorkflow
OCR
Planning
Quality Assurance
Workflow Management Tool
GreensPersistent IdentificationARK identifiers minter and resolver
GreenstoneAccess
Metadata Processing
A suite of software for building and distributing digital library collections
GumshoeMetadata Processing
Forensic
Search interface for metadata extracted from forensic disk images.
Gvfs-infoFile Format Identificationgvfs-info - print information about files and directories
ICA-AtoMMetadata ProcessingICA-AtoM allows organisations to create standards-based descriptions of their archival holdings and subsequently publish them to the Web.
IMacrosQuality Assurance
Web Capture
iMacros makes it easy to test web-based applications.
ITextMetadata ExtractionPDF library for manipulation, content extraction and creation
ImageVerifierMetadata Processing
Validation
Quality Assurance
ImageVerifier (IV for short) traverses a hierarchy of folders looking for image files to verify. It can verify TIFFs, JPEGs. PSDs, DNGs, and non-DNG raws (e.g., NEF, CR2).
InBoxerMetadata Extraction
Metadata Processing
Access
InBoxer is a next generation email archiving, IM archiving, e-discovery, and policy management system.
Index.dat Analyzer v2.5Metadata Extraction
Forensic
Index.dat Analyzer is a tool to view, examine and delete contents of index.dat files.
... further results