Difference between revisions of "PRONOM Signature Development Utility"

From COPTR
Jump to navigation Jump to search
(Initial page linking back to GitHub repo)
 
(Add homepage for the utility itself)
 
(8 intermediate revisions by 4 users not shown)
Line 1: Line 1:
−
<!-- Use the structure provided in this template, do not change it! -->
+
{{Infobox tool
−
 
 
−
{{Infobox_tool
 
 
|purpose=Output DROID compatible file format signature files using PRONOM syntax
 
|purpose=Output DROID compatible file format signature files using PRONOM syntax
−
|image=
+
|homepage=https://github.com/exponential-decay/signature-development-utility, http://ffdev.info
−
|homepage=https://github.com/exponential-decay/signature-development-utility
 
 
|license=Open source (see URL above)
 
|license=Open source (see URL above)
−
|platforms=PHP + JQuery + Javascript + text/html
+
|platforms=Golang + PHP + JQuery + Javascript + text/html
 +
|function=File Format Identification
 
}}
 
}}
 +
{{Infobox tool details}}
 +
== Description ==
 +
<!-- Describe the what the tool does, focusing on it's digital preservation value. Keep it factual. -->
 +
 +
Utility to enable the creation of DROID compatible signature files using PRONOM regular expression syntax. The tool outputs in an XML format compatible with DROID 4 upwards (including DROID 5 and 6). Three sequences can be combined to create a single file format signature. Signature files can be concatenated manually if more complex collections are required for testing.
 +
 +
=== Version 2: FFDev.info ===
 +
 +
[https://github.com/exponential-decay/signature-development-utility Version 2] of the utility was released in October 2020. The utility adds support for Container Signature sequences. The first iteration is bootstrapped to the original utility which has already been well proven.
 +
 +
==== Hosting ====
 +
 +
Version 2 is hosted on:
 +
 +
* http://ffdev.info
 +
 +
=== Version 1: PRONOM signature development utility ===
 +
 +
The original code for version 1 of the utility is still available in a [https://github.com/exponential-decay/signature-development-utility/tree/signature-development-utility-1.0 branch on Github].
 +
 +
==== Hosting ====
 +
 +
Version 1 is hosted at exponentialdecay.co.uk and The National Archives, UK:
 +
 +
* [http://www.nationalarchives.gov.uk/pronom/sigdev/index.htm Signature development utility at The National Archives, UK]
 +
* [http://exponentialdecay.co.uk/sd/index.htm Signature development utility at exponentialdecay.co.uk]
 +
 +
== Using its Output ==
 +
 +
=== DROID ===
 +
 +
The signature development utility output can be directly uploaded to DROID via its 'Install Signature File' mechanism. Be sure to then select the new file from 'Preferences'. Container signatures need to be added via the users /home/<username>/.droid6 directory. On Windows the user's home folder can be found under Users in 'C:'.
 +
 +
=== Siegfried ===
 +
 +
[http://coptr.digipres.org/Siegfried Siegfried] is a useful tool for testing because it can combine a new signature file from this utility with all the other signatures in the PRONOM corpus, plus its many others.
 +
 +
==== Instructions for Linux ====
 +
 +
We use a utility called Roy to extend signature files. This is installed alongside Siegfried.
 +
 +
To extend a signature file to include a custom signature file, we need to make sure that there is a 'custom' folder, where Roy can find it. We will need our signature to be on a path that looks as follows:
 +
 +
<pre>/home/{username}/siegfried/custom/{custom-dev-sig}.xml</pre>
 +
 +
Extending the signature file can then be completed in two stages:
 +
 +
<pre>1) Roy harvest</pre>
 +
 +
This will download PRONOM signature file reports. This is the most accurate way to run Siegfried. An alternative is to let Siegfried parse the DROID signature file, but this has shown to lead to some inconsistencies where PRONOM and DROID do not reflect each other entirely.
 +
 +
<pre>2) Roy build -extend {custom-dev-sig}.xml</pre>
 +
 +
Try Siegfried on a single file of the format we have created the signature for:
 +
 +
<pre>3) sf {custom-dev-format}.{ext}</pre>
 +
 +
The result will be:
 +
 +
<pre>---
 +
siegfried  : 1.7.6
 +
scandate    : 2017-10-25T23:35:31+13:00
 +
signature  : default.sig
 +
created    : 2017-10-25T23:23:59+13:00
 +
identifiers :
 +
  - name    : 'pronom'
 +
    details : 'DROID_SignatureFile_V91.xml; container-signature-20170330.xml; extensions: {custom-dev-sig}.xml'
 +
---
 +
filename : '{custom-dev-format}.{ext}'
 +
filesize : 1492992
 +
modified : 2017-10-22T16:02:13+13:00
 +
errors  :
 +
matches  :
 +
  - ns      : 'pronom'
 +
    id      : 'dev/1'
 +
    format  : '{custom-dev-format}'
 +
    version : '1.0'
 +
    mime    : 'application/{custom-dev-format}'
 +
    basis  : 'extension match img; byte match at 1024, 32'
 +
    warning : </pre>
 +
 +
=== RDF ===
 +
 +
The tool's output suggests a potential RDF representation of a DROID signature ('''NB.''' this is a version 1 option only).
 +
 +
RDF from the utility looks as follows:
 +
 +
<pre>
 +
<?xml version="1.0" encoding="UTF-8"?>
 +
<rdf:RDF xmlns:xsd="http://www.w3.org/2001/XMLSchema#" xmlns:sigdev="http://nationalarchives.gov.uk/preservation/sigdev/signature/" xmlns:bytes="http://nationalarchives.gov.uk/preservation/sigdev/signature/byteSequence/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
 +
  <sigdev:DevelopmentSignature rdf:about="http://nationalarchives.gov.uk/preservation/sigdev/signature/ddc3ab7d-db41-49c5-b3fa-262ad83dd703">
 +
    <rdfs:label>Development Signature</rdfs:label>
 +
    <sigdev:version>1.0</sigdev:version>
 +
    <sigdev:extension>ext</sigdev:extension>
 +
    <sigdev:internetMediaType>text/x-test-signature</sigdev:internetMediaType>
 +
    <sigdev:puid>dev/1</sigdev:puid>
 +
    <sigdev:byteSequence>
 +
      <rdf:Description rdf:about="http://nationalarchives.gov.uk/preservation/sigdev/signature/byteSequence/228c0626-3e0d-40bc-98d3-d897a21b20e1/1">
 +
        <bytes:string rdf:datatype="http://nationalarchives.gov.uk/preservation/sigdev/signature/droidRegularExpression">255044462D312E34</bytes:string>
 +
        <bytes:anchor>BOFoffset</bytes:anchor>
 +
        <bytes:offset rdf:datatype="http://www.w3.org/2001/XMLSchema#integer">0</bytes:offset>
 +
        <bytes:maxOffset rdf:datatype="http://www.w3.org/2001/XMLSchema#integer">0</bytes:maxOffset>
 +
      </rdf:Description>
 +
    </sigdev:byteSequence>
 +
  </sigdev:DevelopmentSignature>
 +
</rdf:RDF>
 +
</pre>
 +
 +
This can be used to generate a graph visualization of a signature, and provides another serialization of a DROID signature that could potentially be consumed by tools in the future.
  
−
<!-- Add one ore more categories to describe the function of the tool. Choose carefully, and view the list of existing categories first (see the Navigation sidebar on the left). The following are common category examples, remove those that don't apply -->
+
== User experiences ==
−
[[Category:File Format Identification]]
+
<!-- Add hotlinks to user experiences with the tool (eg. blog posts). These should illustrate the effectiveness (or otherwise) of the tool. Use a bullet list. -->
  
−
<!-- Add relevant categories to describe the content type that the tool addresses. Choose carefully, and view the list of existing categories first (see the Navigation sidebar on the left). If the tool works on any content type, do not add a category. The following are common category examples, remove those that don't apply -->
+
* [http://digital-archiving.blogspot.co.nz/2016/08/my-first-file-format-signature.html Jenny Mitcham creates her first signature using the Signature Development Utility]
 +
* [http://openpreservation.org/blog/2016/09/08/making-the-switch-from-user-to-user-and-contributor-my-first-file-format-signature/ Andrea K. Byrne creates her first signature using the Signature Development Utility]
  
−
== Description ==
+
== Additional reading ==
−
<!-- Describe the what the tool does, focusing on it's digital preservation value. Keep it factual. -->
+
 
−
Utility to enable the creation of DROID compatible signature files using PRONOM regular expression syntax. The tool outputs in an XML format compatible with DROID 4 upwards (including DROID 5 and 6). Three sequences can be combined to create a single file format signature. Signature files can be concatenated manually if more complex collections are required for testing.  
+
* [https://openpreservation.org/blogs/pronom-research-week-signature-development-utility-2-0-ffdev-info/ information about ffdev.info on openpreservation.org]
  
−
= Development Activity =
+
== Development activity ==
 
<!-- Provide *evidence* of development activity of the tool. For example, RSS feeds for code issues or commits. -->
 
<!-- Provide *evidence* of development activity of the tool. For example, RSS feeds for code issues or commits. -->
 
All development activity is visible on GitHub: https://github.com/exponential-decay/signature-development-utility/commits
 
All development activity is visible on GitHub: https://github.com/exponential-decay/signature-development-utility/commits
−
+
 
−
=== Activity Feed ===
+
==== Activity feed ====
 
Below the last 5 commits:
 
Below the last 5 commits:
−
<rss max=5>https://github.com/exponential-decay/signature-development-utility/commits/master.atom</rss>
+
<rss max=5>https://github.com/exponential-decay/signature-development-utility/commits/bootstrap-utility-1.0.atom</rss>
 
   
 
   
 
<!-- Add the Ohloh.com ID for the tool, if known. -->
 
<!-- Add the Ohloh.com ID for the tool, if known. -->
−
{{Infobox_tool_details
 
−
|ohloh_id=
 
−
}}
 

Latest revision as of 14:35, 31 January 2022



Output DROID compatible file format signature files using PRONOM syntax
Homepage:https://github.com/exponential-decay/signature-development-utility, http://ffdev.info
License:Open source (see URL above)
Platforms:Golang + PHP + JQuery + Javascript + text/html
Function:File Format Identification




Description

Utility to enable the creation of DROID compatible signature files using PRONOM regular expression syntax. The tool outputs in an XML format compatible with DROID 4 upwards (including DROID 5 and 6). Three sequences can be combined to create a single file format signature. Signature files can be concatenated manually if more complex collections are required for testing.

Version 2: FFDev.info

Version 2 of the utility was released in October 2020. The utility adds support for Container Signature sequences. The first iteration is bootstrapped to the original utility which has already been well proven.

Hosting

Version 2 is hosted on:

Version 1: PRONOM signature development utility

The original code for version 1 of the utility is still available in a branch on Github.

Hosting

Version 1 is hosted at exponentialdecay.co.uk and The National Archives, UK:

Using its Output

DROID

The signature development utility output can be directly uploaded to DROID via its 'Install Signature File' mechanism. Be sure to then select the new file from 'Preferences'. Container signatures need to be added via the users /home/<username>/.droid6 directory. On Windows the user's home folder can be found under Users in 'C:'.

Siegfried

Siegfried is a useful tool for testing because it can combine a new signature file from this utility with all the other signatures in the PRONOM corpus, plus its many others.

Instructions for Linux

We use a utility called Roy to extend signature files. This is installed alongside Siegfried.

To extend a signature file to include a custom signature file, we need to make sure that there is a 'custom' folder, where Roy can find it. We will need our signature to be on a path that looks as follows: 

/home/{username}/siegfried/custom/{custom-dev-sig}.xml

Extending the signature file can then be completed in two stages: 

1) Roy harvest

This will download PRONOM signature file reports. This is the most accurate way to run Siegfried. An alternative is to let Siegfried parse the DROID signature file, but this has shown to lead to some inconsistencies where PRONOM and DROID do not reflect each other entirely. 

2) Roy build -extend {custom-dev-sig}.xml

Try Siegfried on a single file of the format we have created the signature for: 

3) sf {custom-dev-format}.{ext}

The result will be: 

---
siegfried   : 1.7.6
scandate    : 2017-10-25T23:35:31+13:00
signature   : default.sig
created     : 2017-10-25T23:23:59+13:00
identifiers : 
  - name    : 'pronom'
    details : 'DROID_SignatureFile_V91.xml; container-signature-20170330.xml; extensions: {custom-dev-sig}.xml'
---
filename : '{custom-dev-format}.{ext}'
filesize : 1492992
modified : 2017-10-22T16:02:13+13:00
errors   : 
matches  :
  - ns      : 'pronom'
    id      : 'dev/1'
    format  : '{custom-dev-format}'
    version : '1.0'
    mime    : 'application/{custom-dev-format}'
    basis   : 'extension match img; byte match at 1024, 32'
    warning :

RDF

The tool's output suggests a potential RDF representation of a DROID signature (NB. this is a version 1 option only).

RDF from the utility looks as follows: 

<?xml version="1.0" encoding="UTF-8"?>
<rdf:RDF xmlns:xsd="http://www.w3.org/2001/XMLSchema#" xmlns:sigdev="http://nationalarchives.gov.uk/preservation/sigdev/signature/" xmlns:bytes="http://nationalarchives.gov.uk/preservation/sigdev/signature/byteSequence/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
  <sigdev:DevelopmentSignature rdf:about="http://nationalarchives.gov.uk/preservation/sigdev/signature/ddc3ab7d-db41-49c5-b3fa-262ad83dd703">
    <rdfs:label>Development Signature</rdfs:label>
    <sigdev:version>1.0</sigdev:version>
    <sigdev:extension>ext</sigdev:extension>
    <sigdev:internetMediaType>text/x-test-signature</sigdev:internetMediaType>
    <sigdev:puid>dev/1</sigdev:puid>
    <sigdev:byteSequence>
      <rdf:Description rdf:about="http://nationalarchives.gov.uk/preservation/sigdev/signature/byteSequence/228c0626-3e0d-40bc-98d3-d897a21b20e1/1">
        <bytes:string rdf:datatype="http://nationalarchives.gov.uk/preservation/sigdev/signature/droidRegularExpression">255044462D312E34</bytes:string>
        <bytes:anchor>BOFoffset</bytes:anchor>
        <bytes:offset rdf:datatype="http://www.w3.org/2001/XMLSchema#integer">0</bytes:offset>
        <bytes:maxOffset rdf:datatype="http://www.w3.org/2001/XMLSchema#integer">0</bytes:maxOffset>
      </rdf:Description>
    </sigdev:byteSequence>
  </sigdev:DevelopmentSignature>
</rdf:RDF>

This can be used to generate a graph visualization of a signature, and provides another serialization of a DROID signature that could potentially be consumed by tools in the future.

User experiences

Additional reading

Development activity

All development activity is visible on GitHub: https://github.com/exponential-decay/signature-development-utility/commits

Activity feed

Below the last 5 commits:

2025-10-03 06:58:23
[tag:github.com,2008:Grit::Commit/66b69f8bcded84388df65ce150137bed830fa2e2 Improve HTML spacing]
by ross-spencer https://github.com/ross-spencer
2025-10-03 06:54:26
[tag:github.com,2008:Grit::Commit/15fe7e48cf6fcd5975367f2963d19fe93485e58d Add links to HTML and improve formatting]
by ross-spencer https://github.com/ross-spencer
2025-10-02 21:10:01
[tag:github.com,2008:Grit::Commit/ffd44812bbf86cc7c24a0bc6ceb7386bb408b94a Correct tabs full-scroll issue]
by ross-spencer https://github.com/ross-spencer
2025-10-02 20:38:55
[tag:github.com,2008:Grit::Commit/adc76c562dba0fc48b597c2733909c8c66a05f1e Add server commands to justfile]
by ross-spencer https://github.com/ross-spencer
2025-10-02 20:36:45
[tag:github.com,2008:Grit::Commit/befb03900a6c89acb4b371955a792891bcf56d8c Format HTML]
by ross-spencer https://github.com/ross-spencer


Retrieved from ‘https://coptr.digipres.org/index.php?title=PRONOM_Signature_Development_Utility&oldid=5569’