``OME-Zarr'' attempt to standardize the file format of microscope data

Over the past 20 years, the number of image formats for biological microscopes has increased to around 160. Jason Swedlow and his team are working on standardizing image formats to reduce and consolidate this as much as possible.

How open-source software could finally get the world's microscopes speaking the same language

Swedlow and his colleagues have been working on standardizing image formats for 20 years. 'We feel like this is something we can manage,' said Josh Moore, senior research management officer at German BioImaging, Germany's network for microscopists and biological image analysts, who worked on the project with Swedroe. ”

Mr. Swedlow and his colleagues are working on a new format called ``OME-Zarr,'' which combines two projects.

One is ``OME (Open Microsopy Environment)'', which Mr. Swedlow himself established in 2002 to develop open source specifications for biological microscopy data. The other is Zarr, a technique for optimizing how large data arrays are stored in and downloaded from the cloud.

The first specification for OME-Zarr was submitted in 2021 as a next-generation file format (NGFF) for bioimaging, and it was launched in earnest in 2023 as an option for biologists to store their data.

Microscope image data is labeled with metadata such as 3D position, illumination level, scale, sample type, and sample preparation method. The increase in data capacity due to the addition of various metadata is not seen as a problem due to the increase in storage capacity, but in order to read the metadata of image files from each manufacturer, it is necessary to In this case, special software is required.

An example of the difficulties caused by this is the case experienced by Katrin Möller, who became a postdoctoral researcher at the University of Iceland, during her graduate school days at the University of Zurich. At the time, Mellor was imaging cells called microglia in the zebrafish brain, and the image data generated in one session was 1 TB. At this time, Mr. Mellor tested three microscopes, but they all had different data formats, and none of them were compatible with the software used to process and analyze the data. It seems that he manually converted the microscope data he chose into a TIFF file. This conversion process could take up to a day.

Also, in the case of a biologist acquaintance of Mr. Moore, it was necessary to measure a specific angle in every frame of a 72-hour experiment, and it seems that metadata was manually recorded in Microsoft Excel.

Mr. Mellor ultimately built a macro to automate the processing work to speed up the conversion, but it is not something that can be used generally because it only needs to work in the local environment, and it works with different software versions. There is a risk that it will not work. 'Version control is a big issue,' Moore said.

The OME project aims to support everything that biologists may be using and have used, especially as manufacturers may not support older versions. It is said that it is.

The Office of Research Integrity, which examines research for fraud, also welcomes the standardization of the microscope file format, saying it will simplify its work. However, Greta Sharp, a research specialist at the academic journal Nature, points out that image creators sometimes provide low-quality images with no metadata as raw data, and that missing metadata is a problem in AI-generated images. He points out that while there is some danger, the difficulty is also the result of an attempt to save time and effort.

However, despite the known benefits, widespread adoption of the OME-Zarr format is expected to be difficult. ``Biologists have a lot more work to do before they learn how to work with new formats,'' said Martin Jones of the Francis Crick Institute. Also, if it is a conventional image file, we know how to convert it so that it can be handled in a spreadsheet, and data such as pixel size and intensity can be displayed numerically after conversion, but in the case of Zarr files, nested folders It is said that it is not clear what it is just because it is displayed.

On the other hand, microscope vendors argue that it is inevitable that formats will separate as a result of competition. According to Matthias Genenger of Evident, who took over Olympus's microscope business, the company initially built compatibility with NGFFs like OME-Zarr, but as manufacturers improved their microscopes, open file formats or general-purpose file formats may not be able to maximize the performance of the product.

It has also been pointed out that there is no benefit to abandoning the file format that vendors have created so far, so if you want them to switch, you need to show them the benefits.