Difference between revisions of "Workflow:Archival Forensics workflow (storage media deposit)"

From COPTR
Jump to navigation Jump to search
(Created page with "{{Infobox COW |status=Experimental |tools=BitCurator, FTK (Forensic Toolkit) |input=Request to forensically process a digital deposit (storage media) to the University of Glas...")
 
Line 18: Line 18:
 
: A request to forensically process a digital deposit (storage media) to the University of Glasgow Archives & Special Collections, as part of of the [https://coptr.digipres.org/index.php/Workflow:Digital_archiving_workflow_(high-level) Digital Archiving workflow.] <br/>
 
: A request to forensically process a digital deposit (storage media) to the University of Glasgow Archives & Special Collections, as part of of the [https://coptr.digipres.org/index.php/Workflow:Digital_archiving_workflow_(high-level) Digital Archiving workflow.] <br/>
  
; PRE-ACQUISITION APPRAISAL
+
; PREPARATION
: Processes for evaluating whether a deposit request will be accepted by the University Archives.
+
: Obtain supporting resources and materials to forensically process digital storage media.
# Check the deposit request against the [https://www.gla.ac.uk/myglasgow/archivespecialcollections/collectionsdevelopmentpolicy/ Archives & Special Collections collection development policy] - does the request align with the core collecting areas?
+
# Consult the physical conservation and preservation report, documenting all actions on the acquired media and produced during Acquisition in the [https://coptr.digipres.org/index.php/Workflow:Digital_archiving_workflow_(high-level) Digital Archiving workflow].
# Prepare a records survey and/or pre-accession assessment of the proposed deposit.
+
# Retrieve the unique accession number generated for the media to be processed from the Collections Management System. Use the accession number as reference in all forensic processing actions.
# Evaluate the results of the records survey against the Appraisal & Retention policy, which:
+
# Update the conservation and preservation logs on the Collections Management System relating to the storage media, including:
#* addresses issues pertaining to selection and long-term retention of digital objects
+
#* photographic records of the storage media before processing, clearly showing state, serial number(s) and any other relevant information recorded on the media (e.g. labels).
#* extends the collections development policy
+
#* Documentation of media characteristics, such as technology, type, brand, model, serial number.
#* ensures that retention decisions are balanced between value and capacity to preserve for the long-term; and
+
#* Documentation of any hardware setup or configuration necessary to process the storage medium.
#* provides clarity to avoid assumptions over digital storage costs and availability.
+
# Proceed to Imaging.
# Decide whether the requested deposit aligns with policies:
 
#* If not, re-evaluate acquisition and/or reject deposit.
 
#* If yes, proceed to Acquisition.
 
  
; ACQUISITION
+
; IMAGING
: Processes for acquiring digital materials by transfer, donation, or purchase.
+
: Create an exact copy of storage media, encapuslating contents and structures in a single file (a disk image).  
# Follow the methodology in the Space data and information transfer systems — Producer-archive interface — Methodology abstract standard (PAIMAS) ISO 20652:2006 standard. The standard "identifies, defines and provides structure to the relationships and interactions between an information producer and an archive. It defines the methodology for the structure of actions that are required from the initial time of contact between the producer and the archive until the objects of information are received and validated by the archive." ([https://www.iso.org/standard/39577.html ISO]). For more information, see [https://www.dpconline.org/handbook/organisational-activities/acquisition-and-appraisal Acquisition and appraisal, Digital Preservation Handbook (DPC)].
+
# Use write-blocking tools (software or hardware) to only permit read-only access to storage media, so as to avoid compromising the integrity of the data; and protect the data chain of custody.
# Follow the Accepted file formats/media procedure, which:
+
# Use disk imaging software to generate a forensic image file, which can either be:  
#* Specifies decisions on file formats and/or storage media that the University Archives will accept.  
+
#* a physical image, which is a bit-by-bit (exact) copy of the storage medium and includes active (used) and free space. Any deleted data or file fragments will be copied into the image file.
#* Aligns with preservation planning decisions for format normalisation; and capability to access storage media (esp. legacy media, e.g. floppy or zip disks).
+
#* A logical image, which captures active data on the device but not any deleted space, deleted files or fragments.
#* For a summary table of options, see the Acquisition workflow section in [https://www.dpconline.org/handbook/organisational-activities/acquisition-and-appraisal Acquisition and appraisal, Digital Preservation Handbook (DPC)].
+
#* A selection of specific files and directories, also known as a targeted collection.
# For acquisitions deposited in physical storage media:
+
# Instruct the disk imaging software to create a complete file and directory listing; and verify the integrity of the generated image file by comparing hashes:
#* Place all incoming items in quarantine area on arrival, inspect for pest infestation and mould; and follow handling and moving procedures.
+
#* If verification fails and attempts at re-imaging are unsuccessful, create a "failed imaging" report in the COllections Management System logs.
#* Create physical conservation and preservation report, documenting all actions on the acquired media.
+
#* If verification is successful, store the image in process store.
#* Proceed to Accessioning.
+
# Is further forensic processing and analysis required?
# For acquisitions deposited digitally (e.g. file transfer):
+
#* If no, submit the verified disk image to the [https://coptr.digipres.org/index.php/Workflow:Digital_archiving_workflow_(high-level) Digital Archiving workflow]. OR
#* Proceed to Accessioning.
+
#* If yes, proceed to Processing.
  
; ACCESSIONING
+
; PROCESSING
:Process of formally registering deposit into the University Archives, which enables intellectual control over the digital materials.  
+
: Extract and manage information from the data in storage media, and make it available for analysis.
# Generate a unique accession number, based on the University Archives' archival processing guidelines.
+
# Collate sources for processing, by selecting specific folders/files to review and - where appropriate - aggregating data from multiple storage media.
# Compare the file manifests generated during Acquisition to make sure that the transfer includes everything that was deposited by the source.
+
# Perform virus and malware detection checks on the collated sources.
# List the accession number into the University's Collections Management System for cataloguing. Cataloguing processes include decisions over the system of arrangement and level of description that will be used for the deposited materials; definition of access and reproduction conditions; and documentation via descriptive metadata.
+
# Use forensic software to identify and, if possible, remove irrelevant or redundant files from processing. Examples may include operating systems, system files, or user-defined files that have been deemed as irrelevant.
# Decide whether archival forensic processing is required:
+
# Use forensic software to process the data, including hash generation for files; expanding compound files (e.g. zip archives); format identification and validation; creating search text indices; and preparing audiovisual, web amnd email data for analysis.
#* If Yes, follow the Archival Forensics workflow.
+
# Proceed to Analysis.
# Proceed to Transfer.
 
 
 
; TRANSFER
+
; ANALYSIS
: Processes for transferring digital materials to the University Archives.
+
: Use digital forensics methods to search, categorise, review, interpret and curate data in storage media, so as to aid selection and appraisal processes.
# Choose a method for transferring files:
+
# Review the agreement(s) under which the records were donated, in order to identify permissible actions (e.g. whether restoring deleted files is allowed).
#* Copy files from source media. OR
+
# Depending on the nature of the data and on archival needs, use forensic software to identify records of interest, and make them available for appraisal. Analysis methods may include:
#* Create a disk image from source storage media. OR
+
#* Data carving, for restoring data that was deleted or lost from the file system.
#* Request that digital materials are submitted as a BagIt container.
+
#* Decrypting encrypted files and recovering passwords for passwrd-protected files.
# Check digital materials for viruses. See [https://www.nationalarchives.gov.uk/archives-sector/projects-and-programmes/plugged-in-powered-up/digital-preservation-workflows/1-select-and-transfer/ Select and transfer workflow in the TNA guide] (section 1.3) for a reasonable process. Depending on the results of virus checks:
+
#* Viewing and exporting geolocation data from files that have geolocation information associated with them.
#* if virus is found, quarantine and attempt removal; and/or request clean versions from source. If all these fail, prepare a report documenting actions and re-evaluate acquisition.
+
#* Analysing document content to explore terms/words of interest; and automate the identification of personal information, such as names, phone numbers, credit card and social security numbers.
#* if virus-free, proceed with transfer.
+
#* Identifying the langauge in which documents are written.
# Generate checksums to verify data integrity during transmission and/or storage:
+
#* Generating thumbnails from images and videos; and extracting metadata from multimedia files.
#* For digital acquisitions <i>in situ</i>, it might be appropriate to first store files in a temporary location for virus and/or integrity checks, before transferring to process store. Not applicable to all scenarios.
+
#* Flagging duplicate files.
#* Transfer digital materials to Process store (e.g. network drive).
+
#* Discovering infomation (incluidng documents and email communications) relating to pre-defined lists of persons of interest.
# Use tools to identify file types and validate file formats (e.g. DROID, JHOVE), then proceed to Appraisal.
+
# Once all analyses have been completed, consolidate the resulting data into an appropriate file/folder structure.
 +
# Proceed to Exporting.
 +
 
 +
; Exporting
 +
: Export the forensically analysed contents of storage media as logical disk images, alongside relevant processing reports, filters and labels.
 +
# Export any custom filters and labels created to manage the data, which can be useful for other digital archiving processes. Filters help locate items of interest quickly; and labels allow for grouping files in customised ways (e.g. flagging content that requires arhivist attention; or records associated with a specific individual).
 +
# Export any reports generated during processing and analysis, such as file hashes, virus and malware detection reports, search index terms and geolocation data.
 +
# Export the forensically curated contents of processed storage media into a logical disk image.
 +
# Submit the logical disk image to the [https://coptr.digipres.org/index.php/Workflow:Digital_archiving_workflow_(high-level) Digital Archiving workflow].
  
  
Line 74: Line 78:
 
==Evaluation/Review==
 
==Evaluation/Review==
 
<!-- How effective was the workflow? Was it replaced with a better workflow? Did it work well with some content but not others? What is the current status of the workflow? Does it relate to another workflow already described on the wiki? Link, explain and elaborate -->
 
<!-- How effective was the workflow? Was it replaced with a better workflow? Did it work well with some content but not others? What is the current status of the workflow? Does it relate to another workflow already described on the wiki? Link, explain and elaborate -->
 +
 +
==Further Information==
 +
<!-- Provide any further information or links to additional documentation here -->
 +
 +
<!-- Add four tildes below ("~~~~") to create an automatic signature, including your wiki username. Ensure your user page (click on your username to create it) includes an up to date contact email address so that people can contact you if they want to discuss your workflow -->
 +
[[User:Lkon115|Lkon115]] ([[User talk:Lkon115|talk]]) 16:36, 7 November 2022 (UTC)
 +
 +
<!-- Note that your workflow will be marked with a CC3.0 licence -->
  
 
==Further Information==
 
==Further Information==

Revision as of 16:36, 7 November 2022

Archival Forensics workflow (storage media deposit)
Status:Experimental
Tools:
Input:Request to forensically process a digital deposit (storage media) to the University of Glasgow Archives & Special Collections, as part of of the Digital Archiving workflow (see Further Information).
Output:A verified, authentic copy of storage media content exported as a logical or physical image file, with or without forensic processing.
Organisation:Archives and Special Collections (ASC), University of Glasgow

Workflow Description

Archival forensics workflow produced by Archives and Special Collections at the University of Glasgow


START
A request to forensically process a digital deposit (storage media) to the University of Glasgow Archives & Special Collections, as part of of the Digital Archiving workflow.
PREPARATION
Obtain supporting resources and materials to forensically process digital storage media.
  1. Consult the physical conservation and preservation report, documenting all actions on the acquired media and produced during Acquisition in the Digital Archiving workflow.
  2. Retrieve the unique accession number generated for the media to be processed from the Collections Management System. Use the accession number as reference in all forensic processing actions.
  3. Update the conservation and preservation logs on the Collections Management System relating to the storage media, including:
    • photographic records of the storage media before processing, clearly showing state, serial number(s) and any other relevant information recorded on the media (e.g. labels).
    • Documentation of media characteristics, such as technology, type, brand, model, serial number.
    • Documentation of any hardware setup or configuration necessary to process the storage medium.
  4. Proceed to Imaging.
IMAGING
Create an exact copy of storage media, encapuslating contents and structures in a single file (a disk image).
  1. Use write-blocking tools (software or hardware) to only permit read-only access to storage media, so as to avoid compromising the integrity of the data; and protect the data chain of custody.
  2. Use disk imaging software to generate a forensic image file, which can either be:
    • a physical image, which is a bit-by-bit (exact) copy of the storage medium and includes active (used) and free space. Any deleted data or file fragments will be copied into the image file.
    • A logical image, which captures active data on the device but not any deleted space, deleted files or fragments.
    • A selection of specific files and directories, also known as a targeted collection.
  3. Instruct the disk imaging software to create a complete file and directory listing; and verify the integrity of the generated image file by comparing hashes:
    • If verification fails and attempts at re-imaging are unsuccessful, create a "failed imaging" report in the COllections Management System logs.
    • If verification is successful, store the image in process store.
  4. Is further forensic processing and analysis required?
PROCESSING
Extract and manage information from the data in storage media, and make it available for analysis.
  1. Collate sources for processing, by selecting specific folders/files to review and - where appropriate - aggregating data from multiple storage media.
  2. Perform virus and malware detection checks on the collated sources.
  3. Use forensic software to identify and, if possible, remove irrelevant or redundant files from processing. Examples may include operating systems, system files, or user-defined files that have been deemed as irrelevant.
  4. Use forensic software to process the data, including hash generation for files; expanding compound files (e.g. zip archives); format identification and validation; creating search text indices; and preparing audiovisual, web amnd email data for analysis.
  5. Proceed to Analysis.
ANALYSIS
Use digital forensics methods to search, categorise, review, interpret and curate data in storage media, so as to aid selection and appraisal processes.
  1. Review the agreement(s) under which the records were donated, in order to identify permissible actions (e.g. whether restoring deleted files is allowed).
  2. Depending on the nature of the data and on archival needs, use forensic software to identify records of interest, and make them available for appraisal. Analysis methods may include:
    • Data carving, for restoring data that was deleted or lost from the file system.
    • Decrypting encrypted files and recovering passwords for passwrd-protected files.
    • Viewing and exporting geolocation data from files that have geolocation information associated with them.
    • Analysing document content to explore terms/words of interest; and automate the identification of personal information, such as names, phone numbers, credit card and social security numbers.
    • Identifying the langauge in which documents are written.
    • Generating thumbnails from images and videos; and extracting metadata from multimedia files.
    • Flagging duplicate files.
    • Discovering infomation (incluidng documents and email communications) relating to pre-defined lists of persons of interest.
  3. Once all analyses have been completed, consolidate the resulting data into an appropriate file/folder structure.
  4. Proceed to Exporting.
Exporting
Export the forensically analysed contents of storage media as logical disk images, alongside relevant processing reports, filters and labels.
  1. Export any custom filters and labels created to manage the data, which can be useful for other digital archiving processes. Filters help locate items of interest quickly; and labels allow for grouping files in customised ways (e.g. flagging content that requires arhivist attention; or records associated with a specific individual).
  2. Export any reports generated during processing and analysis, such as file hashes, virus and malware detection reports, search index terms and geolocation data.
  3. Export the forensically curated contents of processed storage media into a logical disk image.
  4. Submit the logical disk image to the Digital Archiving workflow.


Purpose, Context and Content

Evaluation/Review

Further Information

Lkon115 (talk) 16:36, 7 November 2022 (UTC)


Further Information