The bqb File Format

In modern science (both experimental and computational), the size of the raw data can become a significant issue — often referred to as “Big Data”. The bqb format, which we have developed in 2018, enables loss­less compression of simulation trajectories – both of atom positions and volumetric data (such as electron density CUBE files).

The use of mod­ern algorithms (polynomial least-squares extrapolation in space and time, Hilbert curve re-ordering, Burrows–Wheeler transformation, multi-table canonical Huffman encoding) enables a very high lossless compression ratio of up to 35 : 1 for volumetric data trajectories, and up to 20 : 1 for standard atom position trajecto­ries. This is by far the smallest lossless storage format for those data among all competitors (see article).

The code is available as free software, and also implemented directly in TRAVIS and CP2k.

M. Brehm, M. Thomas:
"An Efficient Lossless Compression Algorithm for Trajectories
of Atom Positions and Volumetric Data",
J. Chem. Inf. Model. 2018, 58 (10), 2092–2107, DOI 10.1021/acs.jcim.8b00501

During my guest visit at the University of Zurich, I have implemented support for the bqb format directly into the CP2k program package. Starting from CP2k release 8.1 (December 2020), it is possible to directly write the total electron density to bqb files during a CP2k simulation without having to resort to intermediary (huge) Gaussian Cube files. An updated version of the tutorial will follow soon.

See the &DFT/&PRINT/&E_DENSITY_BQB manual section.

Note: If you compile CP2k 8.1 or newer via the Toolchain script, these features will automatically be included by default, and the library will automatically be downloaded and compiled in the process. If you prefer not to use the Toolchain, you need to download and compile the libvori source code from here.