Ingest
Revision as of 10:00, 9 June 2021 by Prwheatley (talk | contribs)
Functions within this lifecycle stage
| Funtion | Definition |
|---|---|
| Dependency Analysis | Tools for identifying essential information that resides externally to a digital object, or for identifying dependent processes such as which DLLs are required by a Windows process. |
| Encryption Detection | Tools that support the detection of encryption or password protection in files. |
| File Format Identification | Tools that enable the automatic identification of the file format of a particular file, typically by examining characteristic codes (often termed file format magic) in the file header. |
| Fixity | Tools that support the verification of file fixity, typically through the generation and validation of checksum based manifests. |
| Metadata Extraction | Tools that support the extraction of metadata from files. |
| Metadata Processing | Tools that support the processing or management of metadata. |
| Persistent Identification | Tools that support the unique and persistent identification of files or intellectual entities. |
| Quality Assurance | Tools that support quality checking of digital resources, identifying damaged, incomplete or low quality data. Typically used to identify damage introduced via processes such as format migration or digitisation. |
| Validation | Tools that support the validation of digital files, typically against a file format specification. |
Tools for this lifecycle stage
| Tool | Function | Purpose |
|---|---|---|
| 3-Heights(TM) PDF Validator | Validation | 3-Heights(TM) PDF Validator from PDF-Tools AG. |
| 7-Zip | Rendering Transfer Fixity | 7-Zip is a file archiver with a high compression ratio, and encryption and fixity check capabilities |
| 7train | Metadata Processing | XSLT 2.0 tool for generating METS files from XML input |
| ACE (Audit Control Environment) | Fixity | The Auditing Control Environment is a mature set of software designed to help libraries and archives prove their holdings are intact and trustworthy. |
| ADC Test | Validation | Tests and reports on audio analog-to-digital converters |
| ALTAG3D | File Management Storage Metadata Extraction Personal Archiving | An open source archive software |
| ARK plugin for Omeka | Persistent Identification | Generating and resolving ARK identifiers for resources in Omeka |
| AVP Fixity | Fixity | Fixity monitoring for digital collections |
| Aaru Data Preservation Suite | Backup Disk Imaging Metadata Extraction | Media dump software and disc image manager |
| Adobe Photoshop Elements | Metadata Extraction Metadata Processing Personal Archiving | A commercial image editor with a metadata module (Organizer). |
| Apache PDFBox | Metadata Extraction Repair File Format Migration Validation Encryption Detection | JAVA PDF library for creation, manipulation, validation and content extraction of PDF documents |
| Apache POI - the Java API for Microsoft Documents | File Format Migration Metadata Extraction Encryption Detection | The Apache POI Project's mission is to create and maintain Java APIs for manipulating various file formats based upon the Office Open XML standards (OOXML) and Microsoft's OLE 2 Compound Document format (OLE2). |
| Apache Tika | Metadata Extraction File Format Identification Text Extraction Embedded File Extraction | Java based tool for identifying file formats using signatures and extracting metadata and text content from documents. |
| Archive::BagIt | Fixity File Copy | BagIt API for Perl |
| ArchivesSpace | Access Metadata Processing | ArchivesSpace is the next-generation web-based archives information management system, designed by archivists and supported by diverse archival repositories. |
| Archivists' Toolkit | Metadata Processing | An open source archival data management system to provide broad, integrated support for the management of archives |
| Archon | Access Metadata Processing | Archon automatically publishes archival descriptive information and digital archival objects in a user-friendly website. |
| Ark service | Persistent Identification | ARK identifiers generator in python |
| AsTiffTagViewer | Quality Assurance | AsTiffTagViewer is a TIFF Tag Viewer application. |
| AudiAnnotate | Access Annotation Preservation System Workflow Academic Social Networking Personal Archiving Service Version Control Metadata Processing Rendering Discovery Persistent Identification Managing Active Research Data | To make audio and its interpretations more discoverable and usable by extending the use of the newest IIIF (International Image Interoperability Framework) standard for audio with the development of the AudiAnnotate web application, documented workflows and workshops that will facilitate the use of existing best-of-breed, open source tools for audio annotation (Sonic Visualiser), for public code and document repositories (GitHub), and audio presentation (Universal Viewer) to produce, publish, and sustain shareable W3C Web Annotations for individual and collaborative audio projects. |
| BIL (BagIt Library) | Fixity File Copy | BagIt Library is a Java software library that supports the creation, manipulation and validation of bags. |
| BWF MetaEdit | Metadata Extraction Metadata Processing Validation | BWF MetaEdit permits embedding, validating, and exporting of metadata in Broadcast WAVE Format (BWF) files. |
| Bad Peggy | Validation Quality Assurance | Scans for damaged images and photos. |
| BagIt Transfer Utilities | Fixity File Copy | BagIt transfer Utilities are a collection of tools developed for the purpose of validation and transfer of bags. |
| Bagger | Fixity Transfer | GUI application to facilitate the creation and verification of BagIt bags. |
| BitCurator | File Management Fixity Metadata Extraction Metadata Processing Quality Assurance Validation Workflow | The BitCurator Environment is an Ubuntu Linux distribution geared to the needs of archivists and librarians. It includes a suite of open source digital forensics and data analysis tools to help collecting institutions process born-digital materials. |
| BnL Mets Exporter | Metadata Processing | Command Line Interface (CLI) to export METS/ALTO documents to other formats. |
| Brunnhilde | Metadata Extraction Content Profiling Appraisal | Siegfried-based characterization of directories and disk images |
| C3PO | Content Profiling Metadata Extraction | C3PO is a content profiling tool for visualization and preservation analysis |
| CSV Validator | Metadata Processing Validation | Validation of CSV files against user-defined schema |
| Checkit tiff | Quality Assurance Validation | a tool to validate TIFF files against given configuration profile |
| Checksum (by Corz) | Fixity | Fast hashing tool using a GUI interface |
| Cksum Unix command | Fixity | cksum computes a cyclic redundancy check (CRC) checksum for each given file, or standard input if none are given |
| Cloc | File Format Identification | Cloc (Count Lines of Code) serves not only to count the lines of Code,but also guesses the programming language, thus can be used to identify files. It is a command line tool which is easy to use. |
| CloudCompare | File Format Migration Multi Format Rendering Metadata Extraction | CloudCompare is a tool for editing and processing 3D point clouds and triangular meshes. |
| Collectus | Metadata Processing | The UVa Library's Collectus digital object collector tool allows users to to collect image or text objects from a repository. |
| ContextMiner | Metadata Processing Web Capture | ContextMiner is a framework to collect, analyze, and present the contextual information along with the data. |
| Crazy-fast-image-scan | File Format Identification Content Profiling Forensic | A script to scan media very quickly to find out what kind of content it contains |
| Curator's Workbench | Metadata Processing | Curator's Workbench is a tool that automates and streamlines the process of preparing collections of digital materials for submission to a repository |
| CyberChef | Encryption Detection File Management Decryption Metadata Extraction Personal Archiving Binary & Hexidecimal Editing Discovery | A forensic tool with workflow capabilities to analyse files and containers |
| DART (Digital Archivist's Resource Tool) | Storage File Management Fixity Transfer | Provides both a GUI and a command-line interface for packaging files and uploading them to remote repositories. |
| DBPTK Developer | Validation File Format Migration | DBPTK Developer - library and command-line tool for exection of database preservation actions |
| DIMAG | Metadata Extraction Preservation System Access File Format Migration Storage Workflow | A software suite supporting archives with preservation of digital information for eternity |
| DIMAG IngestList | Metadata Extraction Transfer | Accompanies ingest process from donor to archive, logs process steps. |
| DNS | Metadata Processing File Format Migration Validation Preservation System | DNS - DA NRW Software Suite |
| DPF Manager | Validation | A TIFF validity checker |
| DROID (Digital Record Object Identification) | File Format Identification Metadata Extraction | DROID (Digital Record Object Identification) is a software tool developed to perform automated batch identification of file formats. |
| DUMPBIN Utility | Metadata Extraction File Format Identification | The DUMPBIN utility, which is provided with the 32-bit version of Microsoft Visual C++, combines the abilities of the LINK, LIB, and EXEHDR utilities. |
| DV Analyzer | Quality Assurance Metadata Processing | DV Analyzer is a technical quality control and reporting tool that examines DV streams in order to report errors in the tape-to-file transfer process. |
| DVRescue | Quality Assurance Metadata Processing | DVRescue is archivist-made software that supports data migration from DV tapes into digital files suitable for long-term preservation. |
| DataCite | Persistent Identification Managing Active Research Data Citation and Impact Tracking | DataCite works with data centres to assign persistent identifiers to datasets using the Digital Object Identifier (DOI) infrastructure. |
| DbDIPview | Access Quality Assurance Redaction | Framework for packaging the database Representation Information and pre-configured user-friendly access. Different combinations of Content Data Objects are supported by an automated deployment mechanism. Enables access to the archived databases in the archive reading room for non-technical users. |
| Demystify | Metadata Extraction Content Profiling De-Duplication | Format Identification Analysis and Reporting |
| Dependency Discovery Tool | Dependency Analysis | The Dependency Discovery Tool searches through binary office files (.doc, .xls and .ppt) and tries to find any documents or files that are linked to the document. |
| Developer Tools in QA: Novice's Toolkit | Quality Assurance | A collaborative document which non-developers can adapt to record QA methods using built-in browser developer tools. |
| DiPS (Digital Preservation Solution) | Access Active Data Storage File Format Identification File Format Migration File Management Metadata Extraction Preservation System Secure Deletion Service Storage Transfer Validation Workflow | DiPS (OAIS compliant Digital Preservation Solution) |
| Directory List & Print | Metadata Extraction File Management | A universal metadata extractor |
| Directory Report | Metadata Processing | Show disk usage, directory printer, find duplicate files, rename files, show file CRC and maintain your files - all in 1 tool |
| DiskFormatID | Disk Imaging File Format Identification | Identify floppy disk formats from kryoflux stream files |
| Disktype | Metadata Extraction Disk Imaging | Tool for detecting the content format of a disk or disk image. It knows about common file systems, partition tables, and boot codes. |
| Docuteam packer | Appraisal Data capture and Deposit File Management Fixity Metadata Processing | Creates and edits SIPs |
| Docworks | OCR Workflow Quality Assurance | Document digitization workflow software |
| Double Commander | Fixity File Copy Batch Rename File Management De-Duplication | Open source file manager with two panels side by side |
| Duke Data Accessioner | File Copy File Format Identification Metadata Extraction Transfer Validation | Data Accessioner provides a graphical user interface to aid in migrating data from physical media to a dedicated file server, documenting the process and using MD5 checksums to identify any errors introduced in transfer. |
| EMET (Embedded Metadata Extraction Tool) | Metadata Extraction | EMET is a stand-alone tool designed to extract metadata embedded in JPEG and TIFF files. |
| EPADD | Metadata Processing Metadata Extraction Content Profiling Access Appraisal | ePADD is a software package developed by Stanford University's Special Collections & University Archives that supports archival processes around the appraisal, ingest, processing, discovery, and delivery of email archives. |
| EXE Explorer | Metadata Extraction | EXE Explorer reads and displays executable file properties and structure. |
| EXIF to DC XML normaliser | File Format Migration Metadata Extraction | Extract EXIF data and normalise it to DC XML. |
| EZARK | Persistent Identification | ARK identifiers management tool and sub-publishers registry |
| EZID | Persistent Identification | EZID (easy-eye-dee) makes it easy to create and manage unique, persistent identifiers. |
| Easy CD-DA Extractor | File Format Migration Metadata Extraction Disk Imaging | Easy CD-DA Extractor is CD Ripper, Music Converter, Audio Converter, Metadata Editor, and CD/DVD burning software. |
| EchoDep Hub and Spoke Framework Tool Suite | Metadata Processing | Tool suite to manage digital content in multiple repository systems. |
| EmbARC | Metadata Processing Quality Assurance | internal file metadata management including embedding and validation |
| EpubCheck | Validation Metadata Extraction Encryption Detection | Validator for EPUB files |
| Exact Audio Copy | File Format Migration Metadata Extraction Disk Imaging | Exact Audio Copy is an audio grabber for audio CDs using standard CD and DVD-ROM drives on Windows only. |
| ExactFile | Fixity | Making sure that what you hash is what you get |
| Exempi | Metadata Extraction Metadata Processing | Exempi is a library for handling XMP metadata, based on the Adobe XMP SDK |
| ExifTool | Metadata Processing Metadata Extraction Repair | Properties extraction, identification, metadata editing |
| Exiv2 | Metadata Processing | Exiv2 is a C++ library and a command line utility to manage image metadata. |
| FCIV | Fixity Transfer | Generates and compares MD5 values stored in an XML file. |
| FFAStrans | Metadata Extraction File Format Identification File Format Migration Quality Assurance Workflow Planning | Task automation engine, mostly used in audio and video visual content management. |
| FIDO (Format Identification for Digital Objects) | Metadata Extraction File Format Identification | A PRONOM based, command line, file format identification tool written in Python |
| FITS (File Information Tool Set) | File Format Identification Validation Metadata Extraction Encryption Detection | FITS allows data curators to identify, validate, and extract technical metadata for the objects in their digital repository. |
| File Analyzer and Metadata Harvester V2 | File Management Fixity Metadata Extraction Metadata Processing Quality Assurance Validation Workflow | The File Analyzer is a general purpose desktop (and command line) tool designed to automate simple, file-based operations. The File Analyzer assembles a toolkit of tasks a user can perform. The tasks that have been written into the File Analyzer code base have been optimized for use by libraries, archives, and other cultural heritage institutions. |
| File Format Identification Pronom | File Format Identification | Perl API to analyze and handle droid (PRONOM) signatures |
| FileAlyzer | Metadata Extraction | FileAlyzer allows a basic analysis of files (showing file properties and file contents in hex dump form) and is able to interpret common file contents like resources structures (like text, graphics, HTML, media and PE). |
| FileTrove | Metadata Extraction | FileTrove indexes files and creates metadata from them. The single binary application walks a directory tree and identifies all regular files by type with Siegfried. |
| FileVerifier++ | Fixity De-Duplication | Windows utility for verifying file contents |
| Filestar | File Format Migration Metadata Extraction File Format Identification | Universal file converter for 900+ file types. |
| Fine Free File Command | File Format Identification | This is the home page for the open source implementation of the file(1) command that ships with every free operating system (OpenBSD, Linux, NetBSD, FreeBSD, etc. |
| Fingerdet | Quality Assurance | QA tool for detecting fingers on digitised pages |
| Fixi | Fixity | Fixi is a command-line utility that indexes, verifies, and updates checksum information for collections of files. |
| Fixity Pro | Fixity | Fixity Pro is a desktop application for Windows and Mac that provides simple automated monitoring and reporting on the data integrity of your files that are stored on your computer, removable storage devices, and mounted network storage locations. Use Fixity Pro to schedule routine scans that will tell you if your files have been changed and if any files have been added, removed, or moved/renamed since the last scan that was performed. |
| Flint | Validation Encryption Detection | Validates a file against a policy, using common validation tools |
| Fq | Access Validation Binary & Hexidecimal Editing Discovery Repair Quality Assurance Policy File Format Identification File Recovery Forensic Metadata Extraction | Tool, language and decoders for working with binary data. |
| FreeCommander | File Management De-Duplication Fixity File Copy | Split-screen file manager with desirable extras |
| GNU Diffutils | De-Duplication Quality Assurance | GNU Diffutils is a package of several programs related to finding differences between files. |
| GNU libextractor | Metadata Extraction | GNU libextractor is a library used to extract meta data from files of arbitrary type. |
| Geosetter | Metadata Extraction Metadata Processing Personal Archiving | A tool that sets coordinates and edits all kind of embedded image metadata. |
| GetID3() | Metadata Extraction | Extracts technical and embedded descriptive metadata from common multimedia file formats. |
| ... further results | ||