Demystify - Revision history

Ross-spencer: Modifies infobox

2025-11-28T15:50:37Z

Modifies infobox

Ross-spencer: Update with information about Demystify's Siegfried integration

2025-11-28T15:47:32Z

Update with information about Demystify's Siegfried integration

Ross-spencer: Add link to Fractal in Detail

2025-11-26T13:21:26Z

Add link to Fractal in Detail

Ross-spencer: Teminology

2025-11-26T11:59:14Z

Teminology

Carl: 16 revisions imported

2025-03-28T12:12:32Z

16 revisions imported

Ross-spencer: Fixup date

2024-03-27T07:09:09Z

Fixup date

Ross-spencer: Record updates including previous name; demystify-lite additions; more user-experiences

2024-03-27T07:08:29Z

Record updates including previous name; demystify-lite additions; more user-experiences

Ross-spencer: Ross-spencer moved page DROID Siegfried Sqlite Analysis Engine to Demystify: The name of the application is no longer 'DROID Siegfried Sqlite Analysis Engine'

2024-03-27T07:03:49Z

Ross-spencer moved page DROID Siegfried Sqlite Analysis Engine to Demystify: The name of the application is no longer 'DROID Siegfried Sqlite Analysis Engine'

Ross-spencer: Update release links

2024-03-27T06:59:02Z

Update release links

Ross-spencer: Updated summary information and introduction

2022-01-21T11:54:23Z

Updated summary information and introduction

@@ Line 1: / Line 1: @@
 {{Infobox tool
-|purpose=Format Identification Analysis and Reporting
+|purpose=Format Identification, Analysis and Reporting
 |homepage=https://github.com/exponential-decay/demystify
 |license=Open source (see URL above)
-|platforms=sqlite + Python + text/html
+|platforms=WebAssembly + sqlite + Python + text/html
 |function=Metadata Extraction, Content Profiling, De-Duplication
 }}

← Older revision		Revision as of 15:47, 28 November 2025
Line 18:		Line 18:

	[https://ross-spencer.github.io/demystify-lite/ Demystify Lite] provides a Pyscript/WASM implementation of Demystify's features and runs completely browser side for users with DROID or Siegfried reports that they would like to see analyzed.		[https://ross-spencer.github.io/demystify-lite/ Demystify Lite] provides a Pyscript/WASM implementation of Demystify's features and runs completely browser side for users with DROID or Siegfried reports that they would like to see analyzed.
		+
		+	=== Siegfried integration ===
		+
		+	Demystify lite integrates a WASM build of Siegfried enabling client-side identification of your digital records and analysis within one secure tool, for more information, see Ross Spencer's [https://exponentialdecay.co.uk/blog/client-side-identification-and-reporting-pipeline-with-siegfried-and-demystify-lite/ brief integration report].

	=== Fractal in Detail ===		=== Fractal in Detail ===

← Older revision		Revision as of 13:21, 26 November 2025
Line 18:		Line 18:

	[https://ross-spencer.github.io/demystify-lite/ Demystify Lite] provides a Pyscript/WASM implementation of Demystify's features and runs completely browser side for users with DROID or Siegfried reports that they would like to see analyzed.		[https://ross-spencer.github.io/demystify-lite/ Demystify Lite] provides a Pyscript/WASM implementation of Demystify's features and runs completely browser side for users with DROID or Siegfried reports that they would like to see analyzed.
		+
		+	=== Fractal in Detail ===
		+
		+	[https://journal.code4lib.org/articles/16351 Fractal in Detail] in the Code4Lib Journal examines some of the motivations behind Demystify, generalizing its context and considering its source data alongside similar tools like Brunnhilde, Freud, and FileDriller.

	== User Experiences ==		== User Experiences ==

@@ Line 11: / Line 11: @@
 Now known as "Demystify" (formerly 'DROID Siegfried Sqlite Analysis Engine') with thanks to Joshua Ng for the suggestion to rename it. Demystify is an engine for the analysis of [https://github.com/digital-preservation/droid DROID] CSV export files, [https://github.com/richardlehane/siegfried Siegfried] YAML export files, and Siegfried 'DROID compatible' output. The tool has three purposes, break the exports into their components and store them within a table in a SQLite database; create additional columns to augment the output where useful; and query the SQLite database, outputting results in a readable form useful for analysis by researchers and archivists within digital preservation departments in memory institutions.
-The tool provides archivist definitions for each of the sections output; these definitions are customizable. The tool also supports output of statistics about files that may require further triage or may not be appropriate for long-term preservation based on institutional rules, in the form of a blacklist. The tool also analyses file names and directory names for non-ascii characters, and also characteristics that may present problems cross-file-system based on known Microsoft rules: http://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx
+The tool provides archivist definitions for each of the sections output; these definitions are customizable. The tool also supports output of statistics about files that may require further triage or may not be appropriate for long-term preservation based on institutional rules, in the form of a denylist. The tool also analyses file names and directory names for non-ascii characters, and also characteristics that may present problems cross-file-system based on known Microsoft rules: http://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx
 The engine can be used to generate a list of file paths for files that may present digital preservation risks (Rogues) or files which on the surface i.e. via identification alone, look okay (Heroes) and these listings can be used in conjunction with [http://manpages.ubuntu.com/manpages/trusty/man1/rsync.1.html rsync] to isolate these sets from one-another to be more flexible to work with.

@@ Line 26: / Line 26: @@
 **'''[2016-05-23]''' [http://openpreservation.org/blog/2016/05/23/whats-in-a-namespace-the-marriage-of-droid-and-siegfried-analysis/ The integration of Siegfried output for consistent and repeatable reporting.]
 **'''[2016-05-24]''' [http://openpreservation.org/blog/2016/05/24/while-were-on-the-subject-a-few-more-points-of-interest-about-the-siegfrieddroid-analysis-tool/ Creating a multi-lingual consistent, digital preservation dialect and exploring alternative methods of format identification using Siegfried's capabilities.]
-**'''[2022-05-0]''' [https://journal.code4lib.org/articles/16351 Fractal in detail: What information is in a file format identification report?]
+**'''[2022-05-09]''' [https://journal.code4lib.org/articles/16351 Fractal in detail: What information is in a file format identification report?]
 = Development Activity =

← Older revision	Revision as of 12:12, 28 March 2025
(No difference)

← Older revision	Revision as of 07:03, 27 March 2024
(No difference)

@@ Line 25: / Line 25: @@
 = Development Activity =
 <!-- Provide *evidence* of development activity of the tool. For example, RSS feeds for code issues or commits. -->
-All development activity is visible on GitHub: http://github.com/ross-spencer/droid-sqlite-analysis/commits
+All development activity is visible on GitHub: http://github.com/ross-spencer/demystify/commits
 === Release Feed ===
 Below the last 3 release feeds:
-<rss max=3>https://github.com/exponential-decay/droid-siegfried-sqlite-analysis-engine/releases.atom</rss>
+<rss max=3>https://github.com/exponential-decay/demystify/releases.atom</rss>
 === Activity Feed ===