Ingest
Revision as of 10:00, 9 June 2021 by Prwheatley (talk | contribs)
Functions within this lifecycle stage
| Funtion | Definition |
|---|---|
| Dependency Analysis | Tools for identifying essential information that resides externally to a digital object, or for identifying dependent processes such as which DLLs are required by a Windows process. |
| Encryption Detection | Tools that support the detection of encryption or password protection in files. |
| File Format Identification | Tools that enable the automatic identification of the file format of a particular file, typically by examining characteristic codes (often termed file format magic) in the file header. |
| Fixity | Tools that support the verification of file fixity, typically through the generation and validation of checksum based manifests. |
| Metadata Extraction | Tools that support the extraction of metadata from files. |
| Metadata Processing | Tools that support the processing or management of metadata. |
| Persistent Identification | Tools that support the unique and persistent identification of files or intellectual entities. |
| Quality Assurance | Tools that support quality checking of digital resources, identifying damaged, incomplete or low quality data. Typically used to identify damage introduced via processes such as format migration or digitisation. |
| Validation | Tools that support the validation of digital files, typically against a file format specification. |
Tools for this lifecycle stage
| Tool | Function | Purpose |
|---|---|---|
| 3-Heights(TM) PDF Validator | Validation | 3-Heights(TM) PDF Validator from PDF-Tools AG. |
| 7-Zip | Fixity Rendering Transfer | 7-Zip is a file archiver with a high compression ratio, and encryption and fixity check capabilities |
| 7train | Metadata Processing | XSLT 2.0 tool for generating METS files from XML input |
| ACE (Audit Control Environment) | Fixity | The Auditing Control Environment is a mature set of software designed to help libraries and archives prove their holdings are intact and trustworthy. |
| ADC Test | Validation | Tests and reports on audio analog-to-digital converters |
| ALTAG3D | File Management Metadata Extraction Personal Archiving Storage | An open source archive software |
| ARK plugin for Omeka | Persistent Identification | Generating and resolving ARK identifiers for resources in Omeka |
| AVP Fixity | Fixity | Fixity monitoring for digital collections |
| Aaru Data Preservation Suite | Backup Disk Imaging Metadata Extraction | Media dump software and disc image manager |
| Adobe Photoshop Elements | Metadata Extraction Metadata Processing Personal Archiving | A commercial image editor with a metadata module (Organizer). |
| Apache PDFBox | Validation Encryption Detection File Format Migration Metadata Extraction Repair | JAVA PDF library for creation, manipulation, validation and content extraction of PDF documents |
| Apache POI - the Java API for Microsoft Documents | Encryption Detection File Format Migration Metadata Extraction | The Apache POI Project's mission is to create and maintain Java APIs for manipulating various file formats based upon the Office Open XML standards (OOXML) and Microsoft's OLE 2 Compound Document format (OLE2). |
| Apache Tika | File Format Identification Metadata Extraction Text Extraction Embedded File Extraction | Java based tool for identifying file formats using signatures and extracting metadata and text content from documents. |
| Archifiltre-Mails | Annotation Data capture and Deposit Metadata Processing Transfer Appraisal | Archifiltre-Mails connects to email containers and visualizes their content, helping you in exploring and adding metadata. |
| Archive::BagIt | Fixity File Copy | BagIt API for Perl |
| ArchivesSpace | Access Metadata Processing | ArchivesSpace is the next-generation web-based archives information management system, designed by archivists and supported by diverse archival repositories. |
| Archivists' Toolkit | Metadata Processing | An open source archival data management system to provide broad, integrated support for the management of archives |
| Archon | Access Metadata Processing | Archon automatically publishes archival descriptive information and digital archival objects in a user-friendly website. |
| Ark service | Persistent Identification | ARK identifiers generator in python |
| AsTiffTagViewer | Quality Assurance | AsTiffTagViewer is a TIFF Tag Viewer application. |
| AudiAnnotate | Academic Social Networking Access Annotation Discovery Managing Active Research Data Metadata Processing Persistent Identification Personal Archiving Preservation System Rendering Version Control Workflow Service | To make audio and its interpretations more discoverable and usable by extending the use of the newest IIIF (International Image Interoperability Framework) standard for audio with the development of the AudiAnnotate web application, documented workflows and workshops that will facilitate the use of existing best-of-breed, open source tools for audio annotation (Sonic Visualiser), for public code and document repositories (GitHub), and audio presentation (Universal Viewer) to produce, publish, and sustain shareable W3C Web Annotations for individual and collaborative audio projects. |
| BIL (BagIt Library) | Fixity File Copy | BagIt Library is a Java software library that supports the creation, manipulation and validation of bags. |
| BWF MetaEdit | Validation Metadata Extraction Metadata Processing | BWF MetaEdit permits embedding, validating, and exporting of metadata in Broadcast WAVE Format (BWF) files. |
| Bad Peggy | Validation Quality Assurance | Scans for damaged images and photos. |
| BagIt Transfer Utilities | Fixity File Copy | BagIt transfer Utilities are a collection of tools developed for the purpose of validation and transfer of bags. |
| Bagger | Fixity Transfer | GUI application to facilitate the creation and verification of BagIt bags. |
| BitCurator | Fixity Validation File Management Metadata Extraction Metadata Processing Quality Assurance Workflow | The BitCurator Environment is an Ubuntu Linux distribution geared to the needs of archivists and librarians. It includes a suite of open source digital forensics and data analysis tools to help collecting institutions process born-digital materials. |
| BnL Mets Exporter | Metadata Processing | Command Line Interface (CLI) to export METS/ALTO documents to other formats. |
| BorgFormat | Validation File Format Identification | A web application and service that combines multiple tools for format identification and validation. |
| Brunnhilde | Content Profiling Metadata Extraction Appraisal | Siegfried-based characterization of directories and disk images |
| C3PO | Content Profiling Metadata Extraction | C3PO is a content profiling tool for visualization and preservation analysis |
| CSV Validator | Validation Metadata Processing | Validation of CSV files against user-defined schema |
| Checkit tiff | Validation Quality Assurance | a tool to validate TIFF files against given configuration profile |
| Checksum (by Corz) | Fixity | Fast hashing tool using a GUI interface |
| Cksum Unix command | Fixity | cksum computes a cyclic redundancy check (CRC) checksum for each given file, or standard input if none are given |
| Cloc | File Format Identification | Cloc (Count Lines of Code) serves not only to count the lines of Code,but also guesses the programming language, thus can be used to identify files. It is a command line tool which is easy to use. |
| CloudCompare | File Format Migration Metadata Extraction Multi Format Rendering | CloudCompare is a tool for editing and processing 3D point clouds and triangular meshes. |
| Collectus | Metadata Processing | The UVa Library's Collectus digital object collector tool allows users to to collect image or text objects from a repository. |
| ContextMiner | Metadata Processing Web Capture | ContextMiner is a framework to collect, analyze, and present the contextual information along with the data. |
| Crazy-fast-image-scan | Content Profiling File Format Identification Forensic | A script to scan media very quickly to find out what kind of content it contains |
| Curator's Workbench | Metadata Processing | Curator's Workbench is a tool that automates and streamlines the process of preparing collections of digital materials for submission to a repository |
| CyberChef | Binary & Hexidecimal Editing Decryption Discovery Encryption Detection File Management Metadata Extraction Personal Archiving | A forensic tool with workflow capabilities to analyse files and containers |
| DART (Digital Archivist's Resource Tool) | Fixity File Management Storage Transfer | Provides both a GUI and a command-line interface for packaging files and uploading them to remote repositories. |
| DBPTK Developer | Validation File Format Migration | DBPTK Developer - library and command-line tool for exection of database preservation actions |
| DIMAG | Access File Format Migration Metadata Extraction Preservation System Storage Workflow | A software suite supporting archives with preservation of digital information for eternity |
| DIMAG IngestList | Metadata Extraction Transfer | Accompanies ingest process from donor to archive, logs process steps. |
| DNS | Validation File Format Migration Metadata Processing Preservation System | DNS - DA NRW Software Suite |
| DPF Manager | Validation | A TIFF validity checker |
| DROID (Digital Record Object Identification) | File Format Identification Metadata Extraction | DROID (Digital Record Object Identification) is a software tool developed to perform automated batch identification of file formats. |
| DROID Siegfried Sqlite Analysis Engine | Content Profiling De-Duplication Metadata Extraction | Format Identification Analysis and Reporting |
| DUMPBIN Utility | File Format Identification Metadata Extraction | The DUMPBIN utility, which is provided with the 32-bit version of Microsoft Visual C++, combines the abilities of the LINK, LIB, and EXEHDR utilities. |
| DV Analyzer | Metadata Processing Quality Assurance | DV Analyzer is a technical quality control and reporting tool that examines DV streams in order to report errors in the tape-to-file transfer process. |
| DVRescue | Metadata Processing Quality Assurance | DVRescue is archivist-made software that supports data migration from DV tapes into digital files suitable for long-term preservation. |
| DataCite | Citation and Impact Tracking Managing Active Research Data Persistent Identification | DataCite works with data centres to assign persistent identifiers to datasets using the Digital Object Identifier (DOI) infrastructure. |
| DbDIPview | Access Quality Assurance Redaction | Framework for packaging the database Representation Information and pre-configured user-friendly access. Different combinations of Content Data Objects are supported by an automated deployment mechanism. Enables access to the archived databases in the archive reading room for non-technical users. |
| Demystify | Metadata Extraction Content Profiling De-Duplication | Format Identification, Analysis and Reporting |
| Dependency Discovery Tool | Dependency Analysis | The Dependency Discovery Tool searches through binary office files (.doc, .xls and .ppt) and tries to find any documents or files that are linked to the document. |
| Developer Tools in QA: Novice's Toolkit | Quality Assurance | A collaborative document which non-developers can adapt to record QA methods using built-in browser developer tools. |
| DiPS (Digital Preservation Solution) | Secure Deletion Access Active Data Storage Validation File Format Identification File Format Migration File Management Metadata Extraction Preservation System Storage Transfer Workflow Service | DiPS (OAIS compliant Digital Preservation Solution) |
| Directory List & Print | File Management Metadata Extraction | A universal metadata extractor |
| Directory Report | Metadata Processing | Show disk usage, directory printer, find duplicate files, rename files, show file CRC and maintain your files - all in 1 tool |
| DiskFormatID | Disk Imaging File Format Identification | Identify floppy disk formats from kryoflux stream files |
| Disktype | Disk Imaging Metadata Extraction | Tool for detecting the content format of a disk or disk image. It knows about common file systems, partition tables, and boot codes. |
| Docuteam packer | Fixity Data capture and Deposit File Management Metadata Processing Appraisal | Creates and edits SIPs |
| Docworks | OCR Quality Assurance Workflow | Document digitization workflow software |
| Double Commander | Fixity De-Duplication File Copy File Management Batch Rename | Open source file manager with two panels side by side |
| Duke Data Accessioner | Validation File Copy File Format Identification Metadata Extraction Transfer | Data Accessioner provides a graphical user interface to aid in migrating data from physical media to a dedicated file server, documenting the process and using MD5 checksums to identify any errors introduced in transfer. |
| EMET (Embedded Metadata Extraction Tool) | Metadata Extraction | EMET is a stand-alone tool designed to extract metadata embedded in JPEG and TIFF files. |
| EPADD | Access Content Profiling Metadata Extraction Metadata Processing Appraisal | ePADD is a software package developed by Stanford University's Special Collections & University Archives that supports archival processes around the appraisal, ingest, processing, discovery, and delivery of email archives. |
| EXE Explorer | Metadata Extraction | EXE Explorer reads and displays executable file properties and structure. |
| EXIF to DC XML normaliser | File Format Migration Metadata Extraction | Extract EXIF data and normalise it to DC XML. |
| EZARK | Persistent Identification | ARK identifiers management tool and sub-publishers registry |
| EZID | Persistent Identification | EZID (easy-eye-dee) makes it easy to create and manage unique, persistent identifiers. |
| Easy CD-DA Extractor | Disk Imaging File Format Migration Metadata Extraction | Easy CD-DA Extractor is CD Ripper, Music Converter, Audio Converter, Metadata Editor, and CD/DVD burning software. |
| EchoDep Hub and Spoke Framework Tool Suite | Metadata Processing | Tool suite to manage digital content in multiple repository systems. |
| EmbARC | Metadata Processing Quality Assurance | internal file metadata management including embedding and validation |
| EpubCheck | Validation Encryption Detection Metadata Extraction | Validator for EPUB files |
| Exact Audio Copy | Disk Imaging File Format Migration Metadata Extraction | Exact Audio Copy is an audio grabber for audio CDs using standard CD and DVD-ROM drives on Windows only. |
| ExactFile | Fixity | Making sure that what you hash is what you get |
| Exempi | Metadata Extraction Metadata Processing | Exempi is a library for handling XMP metadata, based on the Adobe XMP SDK |
| ExifTool | Metadata Extraction Metadata Processing Repair | Properties extraction, identification, metadata editing |
| Exiv2 | Metadata Processing | Exiv2 is a C++ library and a command line utility to manage image metadata. |
| FCIV | Fixity Transfer | Generates and compares MD5 values stored in an XML file. |
| FFAStrans | File Format Identification File Format Migration Metadata Extraction Planning Quality Assurance Workflow | Task automation engine, mostly used in audio and video visual content management. |
| FIDO (Format Identification for Digital Objects) | File Format Identification Metadata Extraction | A PRONOM based, command line, file format identification tool written in Python |
| FITS (File Information Tool Set) | Validation Encryption Detection File Format Identification Metadata Extraction | FITS allows data curators to identify, validate, and extract technical metadata for the objects in their digital repository. |
| File Analyzer and Metadata Harvester V2 | Fixity Validation File Management Metadata Extraction Metadata Processing Quality Assurance Workflow | The File Analyzer is a general purpose desktop (and command line) tool designed to automate simple, file-based operations. The File Analyzer assembles a toolkit of tasks a user can perform. The tasks that have been written into the File Analyzer code base have been optimized for use by libraries, archives, and other cultural heritage institutions. |
| File Format Identification Pronom | File Format Identification | Perl API to analyze and handle droid (PRONOM) signatures |
| FileAlyzer | Metadata Extraction | FileAlyzer allows a basic analysis of files (showing file properties and file contents in hex dump form) and is able to interpret common file contents like resources structures (like text, graphics, HTML, media and PE). |
| FileTrove | Metadata Extraction | FileTrove indexes files and creates metadata from them. The single binary application walks a directory tree and identifies all regular files by type with Siegfried. |
| FileVerifier++ | Fixity De-Duplication | Windows utility for verifying file contents |
| Filestar | File Format Identification File Format Migration Metadata Extraction | Universal file converter for 900+ file types. |
| Fine Free File Command | File Format Identification | This is the home page for the open source implementation of the file(1) command that ships with every free operating system (OpenBSD, Linux, NetBSD, FreeBSD, etc. |
| Fingerdet | Quality Assurance | QA tool for detecting fingers on digitised pages |
| Fixi | Fixity | Fixi is a command-line utility that indexes, verifies, and updates checksum information for collections of files. |
| Fixity Pro | Fixity | Fixity Pro is a desktop application for Windows and Mac that provides simple automated monitoring and reporting on the data integrity of your files that are stored on your computer, removable storage devices, and mounted network storage locations. Use Fixity Pro to schedule routine scans that will tell you if your files have been changed and if any files have been added, removed, or moved/renamed since the last scan that was performed. |
| Flint | Validation Encryption Detection | Validates a file against a policy, using common validation tools |
| Fq | Access Validation Binary & Hexidecimal Editing Discovery File Format Identification File Recovery Forensic Metadata Extraction Policy Quality Assurance Repair | Tool, language and decoders for working with binary data. |
| FreeCommander | Fixity De-Duplication File Copy File Management | Split-screen file manager with desirable extras |
| GNU Diffutils | De-Duplication Quality Assurance | GNU Diffutils is a package of several programs related to finding differences between files. |
| ... further results | ||