What is the efficient system migration method of Shopify, which is also called 'Amazon counter horse'?

' Shopify ', which can easily open an online shop, has a diverse ecosystem that is pointed out as a 'true platform' that can beat Amazon. Reported that Shopify has succeeded in increasing the response speed of the 'storefront', which is the top page of each online shop, by 4 to 6 times by migrating to the new system. It is published by Maxime Vaillan court .

How Shopify Reduced Storefront Response Times with a Rewrite – Shopify Engineering

For Shopify, reducing the response time of the storefront in milliseconds is an important issue. If the response is slow, it may adversely affect SEO, etc. and may lead to a decrease in sales. The storefront implementation, developed more than 15 years ago, lived with the Ruby on Rails system where services other than the storefront run, but Shopify's storefront has performance requirements compared to other services. Vaillancourt said he realized that it was much more rigorous and that using the same system as other services made a huge difference in nature.

So at Shopify, the traditional system is devoted to checkout, management, and handling API traffic, and storefront traffic is handled by a new implementation designed from scratch. The new implementation provides a read-only replica server to reduce the load on the primary server for writes. In addition, we are working on improving the system recovery capacity and capacity under heavy load to withstand debug tools available for developers, simple on-board settings, maintenance of documents, etc. and sudden increase in access at the time of sale. That.

Before introducing a new implementation, it is necessary to confirm that the new implementation behaves like the old one. So he said he built a mechanism to compare the output of both implementations. The verification is performed using the traffic of the production environment, the request from the client is relayed by the NUAX custom routing module by Lua , and the request is sent to the new implementation and the conventional implementation respectively.

The response from the old environment to the request is relayed by the custom routing module and sent to the new environment. The verification mechanism in the new environment compares the received response of the old environment with the response of the new environment, and responds with a positive or negative value depending on the verification result.

Once the new environment has been fully validated, Vaillancourt said it has tested it to render the storefront using the new implementation. The request from the client will be sent to both implementations as it did during the previous validation, but in the rendering test, the routing module will try to validate the request based on the previous validation results for a request similar to the one received. It is said that the destination will be decided.

This means that requests of the type that have a good validation algorithm result will be sent to the new implementation...

Otherwise the request will be sent to the legacy implementation. By continuing to improve the new implementation based on the verification results, it is possible to gradually move from the conventional implementation to the new implementation while performing verification.

The new implementation has improved performance by optimizing SQL queries and memory allocations, and the average server response time for requests served by the new implementation is four to six times faster than the previous implementation. thing. It is also safe because the new implementation will fall back to the old implementation if it fails to render.

Shopify has learned a lot in the process of replacing its storefront implementation. “We continue to look for ways to make systems faster and faster while providing the best possible developer experience for the future,” commented Vaillancourt.

in Software,   Web Service, Posted by darkhorse_log