Nanite

Nanite
A friendly swarm of format-identifying robots
Homepage:	http://openplanets.github.io/nanite/
License:	Apache 2.0

Description

Nanite is simply a heavily cut-down and slightly modified version of DROID. It was built be extracting out the core of the DROID6 identification engine, stripping away all other complexities and external dependencies (GUI, databases, web services, etc.). Even the signature files are packaged within Nanite rather than being downloaded automatically, so that they can be released under separate versions of the tool and used directly without requiring internet access.

The main purpose of doing so was to ensure that DROID could be run at scale as a Hadoop Map-Reduce job, which is far easier when software is made available in self-contained packages. The Nanite codebase also include example Map-Reduce code that can be used to test and compare identification tools. Web Archive Discovery uses Nanite to help identify web archive content.