View on GitHub

DROID

DROID (Digital Record and Object Identification)

Download this project as a .zip file Download this project as a tar.gz file

DROID is a software tool developed by The National Archives to perform automated batch identification of file formats. Developed by its Digital Preservation Department as part of its broader digital preservation activities, DROID is designed to meet the fundamental requirement of any digital repository to be able to identify the precise format of all stored digital objects, and to link that identification to a central registry of technical information about that format and its dependencies.

DROID uses internal signatures to identify and report the specific file format versions of digital files. These signatures are stored in an XML signature file, generated from information recorded in the PRONOM technical registry. New and updated signatures are regularly added to PRONOM, and DROID can be configured to automatically download updated signature files.

The DROID binary zip file is available to download from the dedicated page on The National Archives website.

DROID is a platform-independent Java application. It can be invoked from two interfaces:

DROID allows files and folders to be selected from a file system for identification. This file list can be saved at any point. After the identification process had been run, the results can be output in various report formats, including CSV.

User requirements

DROID requires a minimum of Java 8 Standard Edition (SE). DROID was built and tested on Java 1.8. DROID version 6.7.x will likely operate with Java 7, but this is not guaranteed.

Signature Files and their release notes can be found at the following links:

Signature files

http://www.nationalarchives.gov.uk/aboutapps/pronom/droid-signature-files.htm

Signature file release notes

http://www.nationalarchives.gov.uk/aboutapps/pronom/release-notes.xml

Code structure

The DROID code base is split up into separate projects, to facilitate only using parts of the DROID code that are required, and to enhance and enforce modularity in its construction:


droid-api                             Internal API module to include in third party application for easy file identification
droid-binary                          Additional files required in a release build
droid-build-tools                     A maven project to package the release
droid-command-line                    Command-line control over DROID
droid-container                       Container signature processing
droid-core                            Binary signature processing
droid-core-interfaces                 Resource processing & format recognition
droid-export                          Export droid profiles to CSV
droid-export-interfaces               Interfaces for export of DROID profiles
droid-help                            JavaHelp for DROID
droid-parent                          An empty parent project for the other projects
droid-report                          Reporting for DROID profiles
droid-report-interfaces               Interfaces for DROID reporting
droid-results                         Profile handling, database and filtering
droid-tools                           Tools associated with DROID (e.g. signature utility, signature file parsing)
droid-ui                              Swing GUI control over DROID

Third-party dependencies

beansbinding-1.2.1                    Binds data to GUI controls
org.openide.util-1                    Netbeans platform support
swing-outline-1.4.1                   Netbeans Swing GUI tree table control, part of the Netbeans platform

The required jars can be found in the binary distribution of DROID.

Building DROID

DROID was built using Apache Maven 3. First make sure that this is installed and if behind a proxy, edit the Maven settings.xml file to reflect the proxy server details.

There are an additional three jar files required, as listed in the dependencies section above.

These can be installed to your Maven repository from their respective locations using the following commands:

mvn install:install-file -Dfile=beansbinding-1.2.1.jar -DgroupId=beansbinding -DartifactId=beansbinding -Dversion=1.2.1 -Dpackaging=jar 
mvn install:install-file -Dfile=org-openide-util-1.jar -DgroupId=org.openide -DartifactId=org-openide-util -Dversion=1 -Dpackaging=jar 
mvn install:install-file -Dfile=swing-outline-1.4.1.jar -DgroupId=org.openide -DartifactId=swing-outline -Dversion=1.4.1 -Dpackaging=jar 

Once this is complete you can use the master build file under the main DROID project folder to build the project. This is the pom.xml that sits under the main DROID folder containing all the sub projects.

Simply run "mvn install" from the command line when in this directory and DROID will build. Follow the Maven output to see where Maven has placed the binaries.

Release Notes

DROID 6.8.0

This release has following fixes and improvements

Improvements and Fixes
- Feature: Export Templates - Ability to customise the Droid CSV export by defining templates
- #932 Improvement: Prompt for a new profile creation on closing the preferences dialog
- #938 Improvement: The Windows distributable now includes a 64 bit Java 17 JRE
- #1063 Improvement: Updated commons-httpclient dependecy used for signature download
- Updated documentation for CSV columns in export
- Increased the default maximum memory to 1024MB in startup scripts
- #561 Bugfix: On Windows, when a profile is closed, the associated profile folder is not deleted
- #1072 Bugfix: Empty columns are not populated with a blank when a file has more than one format identification
- #1011 Bugfix: Mismatch in the format name between GUI and API when identification method is 'Container'
- #1100 Bugfix: Cancelling the preferences dialog still makes changes to the preferences
- #1072 Fewer elements in data row when exporting profiles per file when some of them have more than one identification

DROID 6.7.0

This release has following fixes and improvements

Improvements and Fixes
-   #898 Improvement: Sigtool uses the version of java bundled with Droid on the Windows OS
-   #938 Improvement: Internal API result includes whether there is an extension mismatch in identification
-   #861 Improvement: CSV export has column headers when there is more than one PUID per file
-   #952 Improvement: SHA512 is available as an option for checksums
-   #977 Improvement: Added an About Droid dialog to the GUI
-   Improvement: Moved to using a different zip library for opening the zip archives
-   #953 Bugfix: On CLI, -At and -Wt options without any arguments do not analyse archives or web archives

DROID 6.6.0

This release has improvements and bug fixes as listed below

Enhancements and bug fixes
-   Updated to use newer version of Java 11 as bundled JRE
-   Improvement to ?? signature pattern
-   Ability to specify columns to export with the -co option, e.g. -co NAME PUID
-   Configure CSV to only quote if there is a comma in a field, use the -qc option
-   The GUI export dialog also includes column and quoting configuration options.
-   Write a profile straight to a CSV file, not writing to a database, using the -o output option, e.g. -o "/home/user/results.csv"
-   Write a profile straight to the console, by not specifying an output file or a profile to write to.
-   Ability to filter profiles written to CSV file or console just as for an export.
-   Ability to filter a more than one extensions, e.g. -F "file_ext any doc docx xls xlsx".
-   Filter files before they are identified on file metadata (name, extension, last modified, size), using the -ff and -FF filter options.
-   Set both binary and container signature files to use to override profile defaults.
-   Update profile defaults for both binary and container signature files
-   Fix for inverted syntax when the bytes or ranges are inverted
-   Introduced semantic versioning
-   Improved consistency across profile and no profile mode
-   Introduced internal API for making droid easier to use as a dependency

DROID 6.5.0

This release has many improvements and bug fixes over the previous release

Enhancements
-   Only log hikari errors.
-   Manage opening of WARC 1.1 and update identification of WARC 1.0
-   Ability to cherry pick types of archival container to be scanned in CLI and GUI
-   Only parse container sigs once
-   Adds drag drop support for files into a profile GUI window
-   Match signatures in OLE2 sub folders
-   Binary bundle without jre now works for windows too
-   Sort the PUID values in the filter dialog.
-   Full implementation of standard signature syntax in container signatures
-   Bundle jre with windows
-   Let archive files be embedded in any other archive file.
-   Add support for Java 11
-   FatFile support
-   Full implementation of standard signature syntax in container signatures
-   Fix: put back LF (unix) line separators in csv exports
-   Fix filter button on windows on java8
-   Fix: iso prefix in CLI when expanding ISO archives
-   Fix graphical glitches on Windows 10
-   Filter by extension should work when recursing folders
-   Fix support for siard 2.1
-   Fix: .bat script in jre bundle to enable running from network drive
-   Manage special folders (forbidden, shortcuts)
-   Fix .bat file in bundle without jre
-   Fix: processing of the quiet parameter on CLI
-   Fix: CLI can browse again folders recursively
-   Fix: checkbox filter icon display on main screen
-   Added lastmodified for files inside ziparchive
-   Profiles produced now have Windows ends (CR+Newline) rather than just Unix (Newline)
-   Filter on extension in CLI does not work if -R is set
-   Filtering for extension and choosing a single file does not restrict to that file
-   Fix: fmt-1196 SIARD 2.1 is wrongly identified as ZIP
-   cannot open forbidden folders
-   cannot open folder shortcuts on Windows
-   CSV export incorrect
-   UI fails to expand / collapse node in warc after a leaf node is clicked

DROID 6.4

The largest change to DROID in this release is the ability to scan more 'archival containers' - 7Zip, RAR, ISO, and BZip2.

-	DROID can now scan contents of RAR, ISO, 7Zip and BZip2 files
-	Will highlight empty folders in GUI display and CSV export
-	CSV export will sort on the URI column for persistent ordering
-	Added custom URI encoder to correctly encode reserved characters
-	Additional option for export CSV with UTF8 BOM
-	Bug fix for errors when using filters
-	ZIP library has been upgraded to allow profiling of zip64 files
-	Increase in memory for DROID when scanning larger files
-	Improved error messages for when DROID is trying to download signatures from the wrong URL
-	Bug fix for issues saving reports as text in Mac OS.