400 million · 70 TB storage object storage object with storage experience "STF" released as free open source

It is actually used for the management and storage of a large amount of image data by services such as livedoor Blog, "It enables to store and refer to a large amount of data safely and fast without using expensive dedicated storage, Because it is made with existing open source technology, you do not need to learn new expertise in operation compared with dedicated storage, so the cost of operation will be low "STF"Has been released. The license is "Artistic License 2.0"is.

Livedoor Lab EDGE Development Diary: Announcement of Distributed Object Storage System "STF" Release - livedoor Blog (blog)
http://blog.livedoor.jp/edge_labs/archives/1609981.html


The background to development is as follows.

In the past, livedoor blogs and other web services that users submit content use a storage system costing tens of thousands of yen per year. The answer given by Livedoor engineers to "Is it self-development of the storage system" to the question as to how to face the ever-increasing data future?

First of all, we created a type of storage system that is distributed for each user. It was named DAVStorage, and it started running for the real environment for about 5 years ago. Although DAVStorage had a merit in terms of cost, as the size of what was initially problematic became larger, the shortfall in terms of functions became apparent, and there was a need to develop a next generation type of system.

What made it is distributed object storage system 'STF' which is open source now. In STF, it becomes possible to distribute file units, it is possible to store and refer to a large amount of data more safely and faster. The convenience of maintenance and operation has also improved than before, and now it is developing and operating a system that uses inexpensive HDDs in parallel control, it has become possible to cope with cost improvement and a large number of users.

So please try STF by all means.

It is possible to actually download from the following page.

STF Distributed Object Storage System - livedoor labs EDGE
http://labs.edge.jp/stf/


Features are as follows.

· Implemented with open source software such as Apache, MySQL, Q4M, memcached, Perl

· Prepare multiple copies at the time of saving an object and also cope with data corruption by any chance

· Automatic restoration at data corruption

· Add / delete storage with zero downtime

· Highly versatile HTTP communication with clients

A typical configuration chart looks something like this


"MogileFSThe difference between the following page is explained.

STF Distributed Object Storage - Perl Advent Calendar Japan 2011 Hacker Track
http://perl-users.jp/articles/advent-calendar/2011/hacker/22

STF is distributed object storage. MogileFS is famous as a similar system of Perl main, but STF uses PSGI compatibility by taking advantage of the late benefits, adopts the protocol which is used basically as HTTP, open and withering technology, maintenance · I think that the convenience of operation is rising.

STF has been released as an open source project. Anyone can use it.

Maintenance and troubleshooting are quite easy as we are using only dead technology and open protocols. The performance is perfect. Deployment of zero downtime Once you can set up around the circumstances, system updates can be done without stress at all. it's recommended!