I once set an S3 lifecycle setting that accidentally affected 3 years worth of logs to Glacier. The next morning I woke up to a billing alert and an AWS bill with an extra $250k in charges (our normal run rate was $30k/month at the time). Basically I spent my entire add annual cloud budget for the year overnight.
Thankfully after an email to our account rep and a bunch of back and forth I was able to get the charges reduced to $4,300.
I made this mistake but honestly? AWS is the most confusing clusterfuck of all time. I can’t stand it and refuse to use it for personal projects.
For me the problem came down to four conflicting sources on AWS regarding tiers and then another problem with the SDK. The SDK didn’t match the tiers at all so “archive” meant Glacier for some reason. 👎
Deleting from. We move logstores and I added an ageout policy for anything over 1 day, to "easily" empty a bucket overnight. I forgot that I had been cycling stuff to glacier after 6 months, and there were 3 years of logs in there.
I once set an S3 lifecycle setting that accidentally affected 3 years worth of logs to Glacier. The next morning I woke up to a billing alert and an AWS bill with an extra $250k in charges (our normal run rate was $30k/month at the time). Basically I spent my entire add annual cloud budget for the year overnight.
Thankfully after an email to our account rep and a bunch of back and forth I was able to get the charges reduced to $4,300.
I made this mistake but honestly? AWS is the most confusing clusterfuck of all time. I can’t stand it and refuse to use it for personal projects.
For me the problem came down to four conflicting sources on AWS regarding tiers and then another problem with the SDK. The SDK didn’t match the tiers at all so “archive” meant Glacier for some reason. 👎
Yeah luckily Amazon is good about mess-ups that are one-time like this. Was the cost because you were pushing to, or retrieving from Glacier?
Deleting from. We move logstores and I added an ageout policy for anything over 1 day, to "easily" empty a bucket overnight. I forgot that I had been cycling stuff to glacier after 6 months, and there were 3 years of logs in there.