DROID Siegfried Sqlite Analysis Engine

Jump to: navigation, search

Analysis and automatic generation of summary information from DROID output
License:Open source (see URL above)
Platforms:sqlite + Python + text/html


[edit] Description

Engine for analysis of DROID CSV export files, Siegfried YAML export files, and Siegfried 'DROID compatible' output. The tool has three purposes, break the exports into their components and store them within a table in a SQLite database; create additional columns to augment the output where useful; and query the SQLite database, outputting results in a readable form useful for analysis by researchers and archivists within digital preservation departments in memory institutions.

The tool provides archivist definitions for each of the sections output; these definitions are customisable. The tool also supports output of statistics about files that may require further triage or may not be appropriate for long-term preservation based on institutional rules, in the form of a blacklist. The tool also analyses file names and directory names for non-ascii characters, and also characteristics that may present problems cross-file-system based on known Microsoft rules: http://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx

The engine can be used to generate a list of file paths for files that may present digital preservation risks (Rogues) or files which on the surface i.e. via identification alone, look okay (Heroes) and these listings can be used in conjunction with rsync to isolate these sets from one-another to be more flexible to work with.

[edit] User Experiences

[edit] Development Activity

All development activity is visible on GitHub: http://github.com/ross-spencer/droid-sqlite-analysis/commits

[edit] Release Feed

Below the last 3 release feeds:

2016-09-30 02:52:29
[tag:github.com,2008:Repository/15066530/0.6.4-BETA A little more on internationalization...]
by ross-spencer
2016-08-31 09:45:18
[tag:github.com,2008:Repository/15066530/0.6.2-BETA Rogue directories handled better...]
by ross-spencer
2016-08-09 07:40:26
[tag:github.com,2008:Repository/15066530/0.6.1-BETA No regress...they don't work]
by ross-spencer

[edit] Activity Feed

Below the last 5 commits:

2017-10-19 05:54:19
[tag:github.com,2008:Grit::Commit/ef29f1eb4dbb39aa147a263de79dfe81a6e73928 [feature] Updated blacklist search.]
by ross-spencer https://github.com/ross-spencer
2017-05-29 05:30:37
[tag:github.com,2008:Grit::Commit/f7c7d17c7aa678381460f11a1f6631af7c0bfbcd [feature] Don't output files inside archives if count is zero.]
by ross-spencer https://github.com/ross-spencer
2017-05-29 05:25:04
[tag:github.com,2008:Grit::Commit/4843c1c8160594566662ab1fcf3ca7adf8cdd958 [feature] Updated sqlitefid.py]
by ross-spencer https://github.com/ross-spencer
2017-05-29 05:17:47
[tag:github.com,2008:Grit::Commit/58c80d226a504b086755ba0cd4b01a8e15a3fa8c [feature] Updated blacklist.]
by ross-spencer https://github.com/ross-spencer
2017-05-26 10:47:42
[tag:github.com,2008:Grit::Commit/6a2711e3ea4eec70ef47ab18aac0b4989d1ecfe3 [feature] .gitignore]
by ross-spencer https://github.com/ross-spencer


Chlara (16.1%), Ross-spencer (41.1%), Prwheatley (42.8%)