Why is Microsoft Azure suddenly charging a large amount of money to the password leakage check site 'Have I Been Pwned?'?
Troy Hunt: How I Got Pwned by My Cloud Costs
https://www.troyhunt.com/how-i-got-pwned-by-my-cloud-costs/
The invoice for December 2021 sent from Microsoft Azure to Mr. Hunt is as follows. The usage fee for Azure Blob Storage is 4557.13 Australian dollars (about 370,000 yen) and 5012.85 Australian dollars (about 400,000 yen) including 10% tax. It seems that this invoice arrived under Mr. Hunt on January 10, 2022, but at the wrong time, the whole family except Mr. Hunt was infected with the new coronavirus, so the invoice was billed. It was January 20th that the book was actually confirmed. Surprised that the royalties were higher than expected, Hunt immediately conducted a cost analysis of Have I Been Pwned ?.
The breakdown of the usage fee is like this. The most expensive is data communication costs, which account for AUD 4457.78 (about 360,000 yen), which accounts for 90% of the total cost. According to Hunt, the communication cost is AUD 0.014 (about 1.14 yen) per 1GB of
If it is a web server, traffic may increase and communication costs may rise, but since Have I Been Pwned? Uses cloud storage, Mr. Hunt investigated the cause of the increase in data traffic. The result is the graph below, and you can see that the outbound has increased sharply around December 20th.
'The timing of the sharp increase in outbound coincides with the time when the National Crime Agency (NCA) in the United Kingdom shared data with Have I Been Pwned ?,' Hunt said.
More than 585 million passwords shared by the National Crime Agency of the United Kingdom with the personal information leakage confirmation site 'Have I Been Pwned?' --GIGAZINE
Below is a graph of the peak data traffic from 11:00 to 15:00 on December 20th. You can see spikes on the outbound every 15 minutes, jumping up to 17.3GB.
Looking at the data request log, I found that Cloudflare requested a ZIP file 'pwned-passwords-sha1-ordered-by-count-v8.7z' in Azure Blob Storage. So, here's what I checked from the Cloudflare dashboard.
'The direct cause of the symptom was clear. Cloudflare wasn't caching what it was supposed to cache,' Hunt said. To find out why that happened, Hunt reviewed the settings for the files in Azure Blob Storage. After contacting an acquaintance who works for Cloudflare, Hunt realized that there might be a problem with 'setting the maximum cacheable file size.'
Then, an acquaintance said, 'When I actually checked it, Cloudflare did not cache the ZIP file. The plan sets the maximum cacheable file size to 15GB, but your ZIP file is close to 18GB. It seems, 'he said.
In fact, looking at the two files I added to Azure Blob Storage in December 2021, they both turned out to be over 15GB. The file size of 'pwned-passwords-sha1-ordered-by-count-v8.7z' was 17.3GB, which was exactly the same amount of data that the outbound spike showed.
So Hunt adjusted the Cloudflare settings. Furthermore, raising the file limit of 15GB on the Cloudflare side solved the problem.
Mr. Hunt solved the problem on January 22, 2022. It's been over a month since Have I Been Pwned? Uploaded the password information ZIP file shared with NCA to Azure Blob Storage, and communication costs have increased during that time. According to Mr. Hunt, the total communication cost is AUD 11,448.3 (about 930,000 yen).
In Azure, there was a setting to send an alert email when the daily outbound exceeds a certain amount or when the usage fee of Azure exceeds a certain amount, but Mr. Hunt said that he did not set these. That. 'I knew there was a risk, but I didn't take the necessary steps until it was actually damaged, but it could have been worse,' Hunt said. May have been 10 times higher, and I might have noticed it sooner. '
Related Posts:
in Web Service, Posted by log1i_yk