DROID is a software tool developed by The National Archives to perform automated batch identification of file formats. Developed by its Digital Preservation Department as part of its broader digital preservation activities, DROID is designed to meet the fundamental requirement of any digital repository to be able to identify the precise format of all stored digital objects, and to link that identification to a central registry of technical information about that format and its dependencies.
DROID uses internal signatures to identify and report the specific file format versions of digital files. These signatures are stored in an XML signature file, generated from information recorded in the PRONOM technical registry. New and updated signatures are regularly added to PRONOM, and DROID can be configured to automatically download updated signature files.
The DROID binary zip file is available to download from the dedicated page on The National Archives website.
DROID is a platform-independent Java application. It can be invoked from two interfaces:
- Java Swing GUI
- Command line interface
DROID allows files and folders to be selected from a file system for identification. This file list can be saved at any point. After the identification process had been run, the results can be output in various report formats, including CSV.
User requirements
DROID requires a minimum of Java 8 Standard Edition (SE). DROID was built and tested on Java 1.8. DROID version 6.7.x will likely operate with Java 7, but this is not guaranteed.
Signature Files and their release notes can be found at the following links:
Signature files
http://www.nationalarchives.gov.uk/aboutapps/pronom/droid-signature-files.htm
Signature file release notes
http://www.nationalarchives.gov.uk/aboutapps/pronom/release-notes.xml
Code structure
The DROID code base is split up into separate projects, to facilitate only using parts of the DROID code that are required, and to enhance and enforce modularity in its construction:
droid-api Internal API module to include in third party application for easy file identification
droid-binary Additional files required in a release build
droid-build-tools A maven project to package the release
droid-command-line Command-line control over DROID
droid-container Container signature processing
droid-core Binary signature processing
droid-core-interfaces Resource processing & format recognition
droid-export Export droid profiles to CSV
droid-export-interfaces Interfaces for export of DROID profiles
droid-help JavaHelp for DROID
droid-parent An empty parent project for the other projects
droid-report Reporting for DROID profiles
droid-report-interfaces Interfaces for DROID reporting
droid-results Profile handling, database and filtering
droid-tools Tools associated with DROID (e.g. signature utility, signature file parsing)
droid-ui Swing GUI control over DROID
Third-party dependencies
beansbinding-1.2.1 Binds data to GUI controls
org.openide.util-1 Netbeans platform support
swing-outline-1.4.1 Netbeans Swing GUI tree table control, part of the Netbeans platform
The required jars can be found in the binary distribution of DROID.
Building DROID
DROID was built using Apache Maven 3. First make sure that this is installed and if behind a proxy, edit the Maven settings.xml file to reflect the proxy server details.
There are an additional three jar files required, as listed in the dependencies section above.
These can be installed to your Maven repository from their respective locations using the following commands:
mvn install:install-file -Dfile=beansbinding-1.2.1.jar -DgroupId=beansbinding -DartifactId=beansbinding -Dversion=1.2.1 -Dpackaging=jar
mvn install:install-file -Dfile=org-openide-util-1.jar -DgroupId=org.openide -DartifactId=org-openide-util -Dversion=1 -Dpackaging=jar
mvn install:install-file -Dfile=swing-outline-1.4.1.jar -DgroupId=org.openide -DartifactId=swing-outline -Dversion=1.4.1 -Dpackaging=jar
Once this is complete you can use the master build file under the main DROID project folder to build the project. This is the pom.xml that sits under the main DROID folder containing all the sub projects.
Simply run "mvn install" from the command line when in this directory and DROID will build. Follow the Maven output to see where Maven has placed the binaries.
Release Notes
DROID 6.7.0
This release has following fixes and improvements
Improvements and Fixes
- #898 Improvement: Sigtool uses the version of java bundled with Droid on the Windows OS
- #938 Improvement: Internal API result includes whether there is an extension mismatch in identification
- #861 Improvement: CSV export has column headers when there is more than one PUID per file
- #952 Improvement: SHA512 is available as an option for checksums
- #977 Improvement: Added an About Droid dialog to the GUI
- Improvement: Moved to using a different zip library for opening the zip archives
- #953 Bugfix: On CLI, -At and -Wt options without any arguments do not analyse archives or web archives
DROID 6.6.0
This release has improvements and bug fixes as listed below
Enhancements and bug fixes
- Updated to use newer version of Java 11 as bundled JRE
- Improvement to ?? signature pattern
- Ability to specify columns to export with the -co option, e.g. -co NAME PUID
- Configure CSV to only quote if there is a comma in a field, use the -qc option
- The GUI export dialog also includes column and quoting configuration options.
- Write a profile straight to a CSV file, not writing to a database, using the -o output option, e.g. -o "/home/user/results.csv"
- Write a profile straight to the console, by not specifying an output file or a profile to write to.
- Ability to filter profiles written to CSV file or console just as for an export.
- Ability to filter a more than one extensions, e.g. -F "file_ext any doc docx xls xlsx".
- Filter files before they are identified on file metadata (name, extension, last modified, size), using the -ff and -FF filter options.
- Set both binary and container signature files to use to override profile defaults.
- Update profile defaults for both binary and container signature files
- Fix for inverted syntax when the bytes or ranges are inverted
- Introduced semantic versioning
- Improved consistency across profile and no profile mode
- Introduced internal API for making droid easier to use as a dependency
DROID 6.5.0
This release has many improvements and bug fixes over the previous release
Enhancements
- Only log hikari errors.
- Manage opening of WARC 1.1 and update identification of WARC 1.0
- Ability to cherry pick types of archival container to be scanned in CLI and GUI
- Only parse container sigs once
- Adds drag drop support for files into a profile GUI window
- Match signatures in OLE2 sub folders
- Binary bundle without jre now works for windows too
- Sort the PUID values in the filter dialog.
- Full implementation of standard signature syntax in container signatures
- Bundle jre with windows
- Let archive files be embedded in any other archive file.
- Add support for Java 11
- FatFile support
- Full implementation of standard signature syntax in container signatures
- Fix: put back LF (unix) line separators in csv exports
- Fix filter button on windows on java8
- Fix: iso prefix in CLI when expanding ISO archives
- Fix graphical glitches on Windows 10
- Filter by extension should work when recursing folders
- Fix support for siard 2.1
- Fix: .bat script in jre bundle to enable running from network drive
- Manage special folders (forbidden, shortcuts)
- Fix .bat file in bundle without jre
- Fix: processing of the quiet parameter on CLI
- Fix: CLI can browse again folders recursively
- Fix: checkbox filter icon display on main screen
- Added lastmodified for files inside ziparchive
- Profiles produced now have Windows ends (CR+Newline) rather than just Unix (Newline)
- Filter on extension in CLI does not work if -R is set
- Filtering for extension and choosing a single file does not restrict to that file
- Fix: fmt-1196 SIARD 2.1 is wrongly identified as ZIP
- cannot open forbidden folders
- cannot open folder shortcuts on Windows
- CSV export incorrect
- UI fails to expand / collapse node in warc after a leaf node is clicked
DROID 6.4
The largest change to DROID in this release is the ability to scan more 'archival containers' - 7Zip, RAR, ISO, and BZip2.
- DROID can now scan contents of RAR, ISO, 7Zip and BZip2 files
- Will highlight empty folders in GUI display and CSV export
- CSV export will sort on the URI column for persistent ordering
- Added custom URI encoder to correctly encode reserved characters
- Additional option for export CSV with UTF8 BOM
- Bug fix for errors when using filters
- ZIP library has been upgraded to allow profiling of zip64 files
- Increase in memory for DROID when scanning larger files
- Improved error messages for when DROID is trying to download signatures from the wrong URL
- Bug fix for issues saving reports as text in Mac OS.