Difference between revisions of "Demystify"

From COPTR
Jump to navigation Jump to search
(Corrected URLS)
(Added links to Function and/or Content Type)
Line 1: Line 1:
<!-- Use the structure provided in this template, do not change it! -->
+
{{Infobox tool
 
 
{{Infobox_tool
 
 
|purpose=Analysis and automatic generation of summary information from DROID output
 
|purpose=Analysis and automatic generation of summary information from DROID output
|image=
 
 
|homepage=https://github.com/exponential-decay/droid-siegfried-sqlite-analysis-engine
 
|homepage=https://github.com/exponential-decay/droid-siegfried-sqlite-analysis-engine
 
|license=Open source (see URL above)
 
|license=Open source (see URL above)
 
|platforms=sqlite + Python + text/html
 
|platforms=sqlite + Python + text/html
 +
|function=Metadata Extraction, Content Profiling, De-Duplication
 
}}
 
}}
 
+
{{Infobox tool details}}
<!-- Add one ore more categories to describe the function of the tool. Choose carefully, and view the list of existing categories first (see the Navigation sidebar on the left). The following are common category examples, remove those that don't apply -->
 
[[Category:Metadata Extraction]]
 
[[Category:Content Profiling]]
 
[[Category:De-Duplication]]
 
 
 
<!-- Add relevant categories to describe the content type that the tool addresses. Choose carefully, and view the list of existing categories first (see the Navigation sidebar on the left). If the tool works on any content type, do not add a category. The following are common category examples, remove those that don't apply -->
 
 
 
 
== Description ==
 
== Description ==
 
<!-- Describe the what the tool does, focusing on it's digital preservation value. Keep it factual. -->
 
<!-- Describe the what the tool does, focusing on it's digital preservation value. Keep it factual. -->
Line 45: Line 36:
 
   
 
   
 
<!-- Add the Ohloh.com ID for the tool, if known. -->
 
<!-- Add the Ohloh.com ID for the tool, if known. -->
{{Infobox_tool_details
 
|ohloh_id=
 
}}
 

Revision as of 19:54, 20 April 2021



Analysis and automatic generation of summary information from DROID output
Homepage:https://github.com/exponential-decay/droid-siegfried-sqlite-analysis-engine
License:Open source (see URL above)
Platforms:sqlite + Python + text/html
Function:Metadata Extraction,Content Profiling,De-Duplication




Description

Engine for analysis of DROID CSV export files, Siegfried YAML export files, and Siegfried 'DROID compatible' output. The tool has three purposes, break the exports into their components and store them within a table in a SQLite database; create additional columns to augment the output where useful; and query the SQLite database, outputting results in a readable form useful for analysis by researchers and archivists within digital preservation departments in memory institutions.

The tool provides archivist definitions for each of the sections output; these definitions are customisable. The tool also supports output of statistics about files that may require further triage or may not be appropriate for long-term preservation based on institutional rules, in the form of a blacklist. The tool also analyses file names and directory names for non-ascii characters, and also characteristics that may present problems cross-file-system based on known Microsoft rules: http://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx

The engine can be used to generate a list of file paths for files that may present digital preservation risks (Rogues) or files which on the surface i.e. via identification alone, look okay (Heroes) and these listings can be used in conjunction with rsync to isolate these sets from one-another to be more flexible to work with.

User Experiences

Development Activity

All development activity is visible on GitHub: http://github.com/ross-spencer/droid-sqlite-analysis/commits

Release Feed

Below the last 3 release feeds:

2024-05-05 15:18:04
[tag:github.com,2008:Repository/15066530/v2.0.0 v2.0.0]
by ross-spencer
2024-04-14 13:53:10
[tag:github.com,2008:Repository/15066530/v2.0.0rc7 v2.0.0rc7]
by ross-spencer
2024-03-25 20:52:06
[tag:github.com,2008:Repository/15066530/v2.0.0rc6 v2.0.0rc6]
by ross-spencer

Activity Feed

Below the last 5 commits: Failed to load RSS feed from https://github.com/exponential-decay/droid-siegfried-sqlite-analysis-engine/commits/master.atom: There was a problem during the HTTP request: 404 Not Found