33 minutes from impact to status page for a complete outage is a joke.

At 16:04 “Investigation commenced”. Then at 16:15 “We began the investigation”. Which is it?

“Our protection mechanisms, to validate and block any erroneous deployments, failed due to a software defect which allowed the deployment to bypass safety validations.”

Very circular way of saying “the validator didn’t do its job”. This is AFAICT a pretty fundamental root cause of the issue.

It’s never good enough to have a validator check the content and hope that finds all the issues. Validators are great and can speed a lot of things up. But because they are independent code paths they will always miss something. For critical services you have to assume the validator will be wrong, and be prepared to contain the damage WHEN it is wrong.

>> We began the investigation and started to examine configuration changes within AFD.

Troubleshooting has completed

Troubleshooting was unable to automatically fix all of the issues found. You can find more details below.

>> We initiated the deployment of our ‘last known good’ configuration.

System Restore can help fix problems that might be making your computer run slowly or stop responding.

System Restore does not affect any of your documents, pictures, or other personal data. Recently installed programs and drivers might be uninstalled.

Confirm your restore point

Your computer will be restored to the state it was in before the event in the Description field below.

What puzzles me too is the time it took to recognize an outage.

Looks like there was no monitoring and no alerts.

Which is kinda weird.

Azure Network Availability Issues

Starting at approximately 16:00 UTC, we began experiencing Azure Front Door issues resulting in a loss of availability of some services. We suspect that an inadvertent configuration change as the trigger event for this issue. We are taking two concurrent actions where we are blocking all changes to the AFD services and at the same time rolling back to our last known good state.

We have failed the portal away from Azure Front Door (AFD) to mitigate the portal access issues. Customers should be able to access the Azure management portal directly.

We do not have an ETA for when the rollback will be completed, but we will update this communication within 30 minutes or when we have an update.

This message was last updated at 17:17 UTC on 29 October 2025

AFD is down quite often regionally in Europe for our services. In 50%+ the cases they just don‘t report it anywhere, even if its for 2h+.

I'll be interested in the incident writeup since DNS is mentioned. It will be interesting in a way if it is similar to what happened at AWS.

Whilst the status message acknowledge's the issue with Front Door (AFD), it seems as though the rest of the actions are about how to get Portal/internal services working without relying on AFD. For those of us using Front Door does that mean we're in for a long haul?

They briefly had a statement about using Traffic Manager to work with your AFD to work around this issue, with a link to learn.microsoft.com/...traffic-manager, and the link didn't work. Due to the same issue affecting everyone right now.

They quickly updated the message to REMOVE the link. Comical at this point.

yea its not just the portal. microsoft.com is down too

Yet another reason to move away from Front Door.

We already had to do it for large files served from Blob Storage since they would cap out at 2MB/s when not in cache of the nearest PoP. If you’ve ever experienced slow Windows Store or Xbox downloads it’s probably the same problem.

I had a support ticket open for months about this and in the end the agent said “this is to be expected and we don’t plan on doing anything about it”.

We’ve moved to Cloudflare and not only is the performance great, but it costs less.

Only thing I need to move off Front Door is a static website for our docs served from Blob Storage, this incident will make us do it sooner rather than later.

We saw issues before 16:00 UTC - approx 15:38

DNS. Ofc.

Sounds like they need to move their portal to a region with more capacity for the desired instance type. /s

Good thing HN is hosted on a couple servers in a basement. Much more reliable than cloud, it seems!

The sysadmin subreddit tends to beat hn on outage reports by an hour+ in my experience.

Bunch of on-call peeps over there that definitely know the instant something major goes down

Wow I just left a Starbucks drivethru line because it was just not moving. I guess it was because of this.

Starbucks mobile was down during the AWS outage too...

My inner Nelson-from-the-Simpsons wishes I was on your team today, able to flaunt my flask of tea and homemade packed sandwiches. I would tease you by saying 'ha ha!' as your efforts to order coffee with IP packets failed.

I always go everywhere adequately prepared for beverages and food. Thanks to your comment, I have a new reason to do so. Take out coffees are actually far from guaranteed. Payment systems could go down, my bank account could be hacked or maybe the coffee shop could be randomly closed. Heck, I might even have an accident crossing the road. Anything could happen. Hence, my humble flask might not have the top beverage in it but at least it works.

We all design systems with redundancy, backups and whatnot, but few of us apply this thinking to our food and drink. Maybe get a kettle for the office and a backup kettle, in case the first one fails?

Ha, maybe rethink the I AM NOTHING BUT A HUGE CLOUD CONSUMER thing on some fundamental levels? Like food?

I noticed it when my Netatmo rigamajig stopped notifying me of bad indoor air quality. Lovely. Why does it need to go through the cloud if the data is right there in the home network…

Service culture is so hollow

You know you can talk to your barista and ask for a bagel, right? If you're lucky they still take cash... if you still _have_ cash. :)

Is voting there a one day only event? If not, I feel the solution to that particular problem is quite clear. There’s a million things that could go wrong causing you to miss something when you try to do it in a narrow time range (today after work before polls close)

If it’s a multi day event, it’s probably that way for a reason. Partially the same as the solution to above.

Organizations who had their own datacenters were chided for being resistant to modernizing, and now they modernized to use someone else's shared computers and they stopped working.

I really do feel the only viable future for clouds is hybrid or agnostic clouds.

Finance is increasingly reliant on it too, my bank moved their entire system to AWS. The amount of power being handed over to these cloud companies in exchange for “convenience” is astonishing.

Voting days should be a national holiday.

i'm not sure this is an easily solvable problem. i remember reading an article arguing that your cloud provider is part of your tech stack and it's close to impossible/a huge PITA to make a non-trivial service provider-agnostic. they'd have to run their own openstack in different datacenters, which would be costly and have their own points of failure.

can't believe it's 2025 and some still need to go to some place to vote. I can vote since I can remember(at least 20 years) by mail for anything, we also vote multiple times a year(4-6 times), we just get 1 Month before the things to vote by mail and then mail in back votes. Hope we can soon vote online to get rid of the paper overhead.

Could you provide a reference about the train disruption? I tried but couldn't find anything in English.

Wow thats crazy! National transport infrastructure being so fragile. What a great age we live in.

The Flemish bus company (de Lijn) uses Azure and I couldn't activate my ticket when I came home after training a couple of hours ago. I should probably start using physical tickets again, because at least those work properly. It's just stupid that there's so much stuff being moved to digital only (often even only being accessible through an Android or iOS app, despite the parent companies of those two being utterly atrocious) when the physical alternatives are more reliable.

Yet... deploy on two clouds and you'll get tax payers scream at you for "wasting money" preparing for a black swan event. Can't have both, either reliability or lower cost.

Wasn't cloud sold based on a premise to prevent the very thing that is happening? Sounds like a massive fail of the whole concept.

> wouldn't put China or Russia above this

dang even zealand didn't survive! new zealand got some soul searching with this outage which took down government person ID service, it's called RealME and it can be used to file your taxes apply for passport etc

You are not getting more 9’s rolling your own

Hetzner seems sound, but I doubt they play in the same reliability league as google

One of recent (4 months ago) Cloudflare outages (I think it was even workers) was caused by Google Cloud being down and Cloudflare hosting an essential service there

Are you after nines? Maybe do multi provider?

Many years back was the first time I used Azure, evaluating it for a client.

I remember I at one point had expanded enough menus that it covered the entirety of the screen.

Never before have I felt so lost in a cloud product.

> At this point I'd rather use GCP over Azure and I have zero seconds of experience with it.

TBH, GCP is very good! More people should use it.

The problem is that in some industries, Microsoft is the only option. Many of these regulated industries are just now transitioning from the data center to the cloud, and they've barely managed to get approval for that with all of the Microsoft history in their organization. AWS or GCloud are complete non-starters.

I've used AWS, Azure, and recently GCP. You do NOT want to use GCP.

Microsoft is better at regulatory capture, so Azure has many customers in the public sector. So an Azure outage probably affects the public sector more (see example above about trains).

Microsoft has the regulatory capture. All the European privacy and regulatory laws are good for Azure. That's why your average European government or baking app runs most likely on Azure. (or Oracle, but more likely Azure)

What Amazon, Azure, and Google are showing with their platform crashes amid layoffs, while they supports governments that are Oppressing's Citizens and Ignoring the Law, is that they do not care about anything other than the bottom line.

They think they have the market captured, but I think what their dwindling quality and ethics are really going to drive is adoption of self hosting, distributed computing frameworks. Nerds are the ones who drove adoption of these platforms, and we can eventually end if we put in the work.

Seriously with container technology, and a bit more work / adoption on distributed compute systems and file storage (IPFS,FileCoin) there is a future where we dont have to use big brothers compute platform. Fuck these guys.

The only reason you'd notice MS was down was if Github was down....

I like azure. I think they are more intuitive tua aws and good, and they have good prices for startups ( essentially free for a whole year )

What did you do when AWS was down last week?

I read "Microsoft shop" as "Microsoft slop". Fitting. But at least they open source wash themselves so much they're practically a charity right?

Azure outages don’t faze anyone because nobody notices when it happens.

IIRC, the grocery chain I worked for used to have an offline mode to move customers out the door. But it meant that when the system came back online, if the customers card was denied, the customer got free groceries.

There's a Family Dollar by my house that is down at least 2 full days per month because of bad inet connectivity. I live close enough that with a small tower on my roof i can get line of sight to theirs. I've thought about offering them a backup link off my home inet if they give me 50% of sales whenever its in use. It would be a pretty good deal for them, better some sales when their inet is down vs none.

I remember last mechanical cash registers in my country in 90s and when these got replaced by early electronic ones with blue vacuum fluorescent tubes. Then everything got smaller and smaller. Now I'm pestered to "add the item to the cart" by software.

Last week I couldn't pay for flowers for grandma's grave because smartphone-sized card terminal refused to work - it stuck on charging-booting loop so I had to get cash. Tho my partner thinks she actually wanted to get cash without a receipt for herself excluding taxes

Just to add - this particular supermarket wasn’t fully down, it took ages for them to press “sub total” and then pick the payment method. I suspect it was slow waiting for a request to timeout perhaps

In Germany many stores still accept cash and some even only accept cash and we are ridiculed for this... Seems like one of the rare instances where this is useful :D

Mind-boggling that any retailer would not have the capability to at least run the checkout stations offline.

Currently standing in a half closed supermarket because the tills are down and they cant take payments

There's a fairly large supermarket near me that has both kinds of outages.

Occasionally it can't take cards because the (fiber? cable?) internet is down, so it's cash only.

Occasionally it can't take cash because the safe has its own cellular connection, and the cell tower is down.

I was at Frank's Pizza in downtown Houston a few weeks ago and they were giving slices of pizza away because the POS terminal died, and nobody knew enough math to take cash. I tried to give them a $10 and told them to keep the change, but "keep the change" is an unknown phrase these days. They simply couldn't wrap their brains around it. But hey, free pizza!

why tf would a supermarket depend on Azure? Payment processing isn't their thing

It is interesting to see the differential across different tenants in different geographies:

- on a US tenant I am unable to access login.microsoftonline.com and the login flow stalls on any SSO authentication attempt.

- on a European tenant, probably germany-west, I am able to login and access the Azure portal.

SSO and 365 are working fine for us, but admin portals for Azure/365 are down. Our workloads in Azure don't seem to be impacted.

Guess you have NASSO now (Not A Single Sign On)

I am still stunned people choose to do this, considering major Office 365 outages are basically a weekly thing now.

All the clouds hav had major outages this year.

At this point I dont believe that any one of them is any better or reliable than the others.

> I feel pretty justified in my previous decisions to move away from Azure

I felt this way about AWS last week

Never let a good disaster go to waste ;)

Outages are one thing, but having your content polluted seems like a more serious problem? Unless you subscribed to microsoft banners somehow.

We battled https://learn.microsoft.com/en-us/answers/questions/1331370/... for over a year, and finally decided to move off since there was no any resolution. Unfortunately our API servers were still behind AFD so they were affected by today's stuff...

I have had intermittent issues with winget today. I use UniGetUI for a front-end, and anything tied to Microsoft has failed for me. Judging by the logs, it's mostly retrieving the listing of versions (I assume similar to what 'apt-get update' does, I'm fairly new to using winget for Windows package management).

Also cant do anything right now with the repo's we have in Azure Devops, how lovely...

get vscodium then

"BREAKING: Alaska Airlines' website, app impacted amid Microsoft Azure outage"

https://www.youtube.com/watch?v=YJVkLP57yvM

Pretty much every single Microsoft domain I've tried to access loads for a looooong time before giving me some bare html. I wonder if someone can explain why that's happening.

I am unable to load this article...presumably for related reasons

That specific subdomain has issues with propagation: https://dnschecker.org/#A/answers.microsoft.com (only four resolvers return records)

The root zone and www. do not: https://dnschecker.org/#A/microsoft.com (all resolvers return records)

And querying https://www.microsoft.com/ results in HTTP 200 on the root document, but the page elements return errors (a 504 on the .css/.js documents, a 404 on some fonts, Name Not Resolved on scripts.clarity.ms, Connection Timed Out on wcpstatic.microsoft.com and mem.gfx.ms). That many different kinds of errors is actually kind of impressive.

I'm gonna say this was a networking/routing issue. The CDN stayed up, but everything else non-CDN became unroutable, and different requests traveled through different paths/services, but each eventually hit the bad network path, and that's what created all the different responses. Could also have been a bad deploy or a service stopped running and there's different things trying to access that service in different ways, leading to the weird responses... but that wouldn't explain the failed DNS propagation.

wow, right after AWS suffered a similar thing.

I wonder if this is microsoft "learning" to "prevent" such an issue and instead triggered it...

"One often meets his destiny on the path he takes to avoid it" -- Master Oogway

2027: the year of migrating from your own metal to a managed provider

2028: the year of migrating from a managed provider to the cloud

2029: the year of migrating from the cloud to your own metal in a rack

People keep thinking the solution to their problems is to do something new (that they don't fully understand).

TIL it's called Nirvana Fallacy

I'd predict the year of linux desktop instead.

I've been doing it since 1998 in my bedroom with a dual T1 (and on to real DCs later). While I've had some outages for sure it makes me feel better I am not that divergent in uptime in the long run vs big clouds.

... and DNS caching, and browser file cache, and sessions...

Moving a website quickly is never fun.

That status page is never red. Absolutely useless.

> There are currently no active events. Use Azure Service Health to view other issues that may be impacting your services.

Links to a page on Azure Portal which is down...

They added a message at the same time as your comment:

"We are investigating an issue with the Azure Portal where customers may be experiencing issues accessing the portal. More information will be provided shortly."

All of our Azure workloads are up, but we don't use Azure Front Door. That seems to be the only impacted product, apart from the management portal.

FWIW, all of our databases, VMs, AKS clusters, services, jobs etc - are all working fine. Which services are down for you, maybe we can build a list?

Does their status page depend on something that is down already, so the page just fails static now hence no new updates?

Same playbook for AWS. When they admitted that Dynamo was inaccessible, they failed to provide context that their internal services are heavily dependent on Dynamo

It's only after the fact they are transparent about the impact

It still is very decentralized. We are discussing this via the internet right now.

A lot of money and years of marketing the cloud as the responsible business decision led us here. Now that the cloud providers have vendor lock-in, few will leave, and customers will continue to wildly overpay for cloud services.

From today [0].

> Big Tech lobbying is riding the EU’s deregulation wave by spending more, hiring more, and pushing more, according to a new report by NGO’s Corporate Europe Observatory and LobbyControl on Wednesday (29 October).

> Based on data from the EU’s transparency register, the NGOs found that tech companies spend the most on lobbying of any sector, spending €151m a year on lobbying — a 33 percent increase from €113m in 2023.

Gee whizz, I really do wonder how they end up having all the power!

[0] https://news.ycombinator.com/item?id=45744973

Thats the whole point, big players like AWS and MS can go down, but here we are still talking on the internet.

Decentralisation is winning it seems.

> How did we get here?

I think the response lies in the surrounding ecosystem.

If you have a company it's easier to scale your team if you use AWS (or any other established ecosystem). It's way easier to hire 10 engineers that are competent with AWS tools than it is to hire 10 engineers that are competent with the IBM tools.

And from the individuals perspective it also make sense to bet on larger platforms. If you want to increase your odds of getting a new job, learning the AWS tools gives you a better ROI than learning the IBM tools.

A natural monopoly is a monopoly in an industry in which high infrastructure costs and other barriers to entry relative to the size of the market give the largest supplier in an industry, often the first supplier in a market, an overwhelming advantage over potential competitors. Specifically, an industry is a natural monopoly if a single firm can supply the entire market at a lower long-run average cost than if multiple firms were to operate within it. In that case, it is very probable that a company (monopoly) or a minimal number of companies (oligopoly) will form, providing all or most of the relevant products and/or services.

https://en.wikipedia.org/wiki/Natural_monopoly

Consolidation is the inevitable outcome of free unregulated markets.

In our highly interconnected world, decentralization paradoxically requires a central authority to enforce decentralization by restricting M&A, cartels, etc.

Efficiency (aka cost) <---> Resiliency/redundancy

Pick your point on the scale

Meredith Whittaker (of Signal) addressed your question the other day: https://mastodon.world/@Mer__edith/115445701583902092

[dead]

> How did we get here?

Stonks

> as having no issues on their status page

Unsubstantiated idea: So the support contract likely says there is a window between each reporting step and the status page is the last one and the one in the legal documents giving them several more hours before the clauses trigger.

People feel it, but usually not general consumers like they do when AWS goes down.

If Azure goes down, it's mostly affecting internal stuff at big old enterprises. Jane in accounting might notice, but the customers don't. Contrast with AWS which runs most of the world's SaaS products.

People not being able to do their jobs internally for a day tends not to make headlines like "100 popular internet services down for everyone" does.

You're looking at it. I couldn't find any discussion elsewhere yet...

official status pages are useless most of the time.

now there is an information about "Azure Portal Access Issues". No word about front door being down.

[dead]

> Even the national digital id service is down.

Can't help but smirk as my country is ramming through "Digital ID" right now

I think the biggest features of the big cloud vendors is that when they are down, not only you but your customers and your competitors usually have issues at the same time so everybody just shrug and have a lazy/off day at the same time. Even on call teams reall just have to wait and stay on standby because there is very little they can do. Doing a failover can be slower than waiting for the recovery, not help at all if outage is spanned accross several region, or bring aditional risks.

And more importantly nobody lose any reputation except AWS/Azure/Google.

Ostensible reason.

The real reason is that outages are not your fault. Its the new version of "nobody ever got fired for buying IBM" - later it became MS, and now its any big cloud provider.

For one it’s statistics - Hetzner simply runs far fewer major services than hyperscalers. And the services they run are also more affluent, with larger customer bases, so downtimes are systemically critical. Therefore it’s louder.

On the merits though, I agree, haven’t had any serious issues with Hetzner.

Same with DigitalOcean. I run one box and it hasnt gone down for like 2 years

It's just the admin portal.

It's always DNS, unless it's not DNS.

I hate these failures because you end up with things that keep working fine because the login credentials are cached, etc; but if you restart or otherwise refresh, you're doomed.

Or, the NSA needed to upgrade their access at both.

I was gonna say that obv AWS hacked em to even things up.

This is funny but also possibly true because: business/MBA types see these outages as a way to prove how critical some services are, leading to investors deciding to load up on the vendor's stock.

> We all need to move away from these big cloud providers.

But then who do we blame when things are down? If we manage our own infrastructure we have to stay late to fix it when it breaks instead of saying “sorry, Microsoft, nothing we can do” and magically our clients accepting that…

Ah yes, let's put our multibillion dollar ecommerce site on... checks notes Hetzner.

Lol

AMZN went up almost 4 percent between the day of the outage and the day after. Crazy market.

Look how important we are, is what these failures show

That's a good thing. Stock prices shouldn't go down because of rare incidents which don't accurately represent how successful a company is likely to be in the future.

I looked into this before and the stocks of these large corps simply does not move when outages happens. Maybe intra-day, I don't have that data, but in general no effect.

well, at this point, 90% of the market cap of FAANGS plus Microsoft is... OMG AI LLM hype

Likely behind Azure Front Door.

Much of Xbox is behind that too.

Always fun when you can't trust the main status page but have to go to some opinionated social medial website to see the actual problem.

Azure goes down all the time. On Friday we had an entire regional service down all day. Two weeks ago same thing different region. You only hear about it when it's something everyone uses like the portal, because in general nobody uses Azure unless they're held hostage.

Could be DNS, I'm seeing SERVFAIL trying to resolve what look to be MS servers when I'm hitting (just one example) mygoodtogo.com (trying to pay a road toll bill, and failing).

Yeah they're suggesting to use CLI but then my Frontdoor deployment failed. Welp.

I was a little puzzled as we got notified our apps were down, and then I tried to login in the Azure portal with no success. But the Azure status page reported no incident, so I posted here and quickly confirmed that others were impacted! They did a pretty bad job with their status page as the front door service was shown green all along

I wonder if many Germans are eager to sign up for AFD.

But seriously I thought it would be the console, not a CDN.

We should've never let marketing in the door honestly, all of the product names for the big three are awful.

Microsoft CDN

There, that's it. You're selling it to (hopefully) technical people

It so strongly implies a counterpart.

Same exact time for us as well.

Status page (first link) is down for me. Second one works

Looks like the status page is overloaded...

“ Starting at approximately 16:00 UTC, we began experiencing DNS issues resulting in availability degradation of some services. Customers may experience issues accessing the Azure Portal. We have taken action that is expected to address the portal access issues here shortly. We are actively investigating the underlying issue and additional mitigation actions. More information will be provided within 60 minutes or sooner.

This message was last updated at 16:35 UTC on 29 October 2025”

That was my bet too, then I looked at ISC and noticed there were PoCs released for critical BIND9 vulns yesterday ... might be related?

Quick find someone who can actually read documentation and code!

You just paste the outage error codes back to the LLM and pray it's still working and can fix whatever went wrong!

https://www.microsoft.com works for me (with the www subdomain).

They don't run on Azure!

> What went wrong and why?

> An inadvertent tenant configuration change within Azure Front Door (AFD) triggered a widespread service disruption affecting both Microsoft services and customer applications dependent on AFD for global content delivery. The change introduced an invalid or inconsistent configuration state that caused a significant number of AFD nodes to fail to load properly, leading to increased latencies, timeouts, and connection errors for downstream services.

> As unhealthy nodes dropped out of the global pool, traffic distribution across healthy nodes became imbalanced, amplifying the impact and causing intermittent availability even for regions that were partially healthy. We immediately blocked all further configuration changes to prevent additional propagation of the faulty state and began deploying a ‘last known good’ configuration across the global fleet. Recovery required reloading configurations across a large number of nodes and rebalancing traffic gradually to avoid overload conditions as nodes returned to service. This deliberate, phased recovery was necessary to stabilize the system while restoring scale and ensuring no recurrence of the issue.

> The trigger was traced to a faulty tenant configuration deployment process. Our protection mechanisms, to validate and block any erroneous deployments, failed due to a software defect which allowed the deployment to bypass safety validations. Safeguards have since been reviewed and additional validation and rollback controls have been immediately implemented to prevent similar issues in the future.

So, so far they're saying it's a combination of bad config + their config-validator had a bug. Would love more details.

>How can one of the richest companies in the world not offer a better service?

Better service costs money.

SSO is down, Azure Portal Down and more, seems like a major outage. Already a lot of services seem to be affected: banks, airlines, consumer apps, etc.

I tried to look some things up on their support pages before 1600Z, and it timed-out. The Dutch railways are also affected (they're an MS shop, IIRC).

We're multi-cloud and it really saved a few workloads last week with the AWS issue.

It's not easy though.

It's always freakin DNS...

Trusting AI without sufficient review and oversight of changes to production.

cost cutting attempts

Might be a failsafe, if you cant get a license status, and you're aware that MS is down, just default to the highest tier.

Free upgrade

This is because Azure just copies everything AWS does. Google is a bit more innovative, they will have something else unexpected happen.

Maybe they are and no one realized yet.. :P

That said, I don't hear about GCP outages all that often. I do think AWS might be leading in outages, but that's a gut feeling, I didn't look up numbers.

And if they don't, we'll know who the culprit is.

here's hoping its Oracle's cloud instead....

Why are Azure App Services still working?

If true then what "utility" is there?

Aww man you got me curious for a sec there.

They probably hired the ones AWS laid off, causing the AWS outage.

Institutional knowledge matters. Just has to be the right institution is all.

Sounds like Shrodinger's Deploy

I didn't realize AWS got a pass?

Which itself is^H^H was down. Wow.

503 Service Unavailable

wrong thread

Our Azure hosted dotnet App Service is working fine, but our docs site served via Front Door went down. Can’t access anything through the Portal.

Maybe they need a downtime tracker. ;)

That saying is just as alive today as it ever was.

https://isitdns.com/

Yeah, I have non prod environments that don't use FD that are functioning. Routing through FD does not work. And a different app, nonprod doesn't use FD (and is working) but loads assets from the CDN (which is not working).

FD and CDN are global resources and are experiencing issues. Probably some other global resources as well.

Hate to say it, but DNS is looking like it's still the undisputed champ.

Luckily?

Majority of actual Xbox services are working fine, xbox.com itself is busted.

It doesn't matter whether you actually are safe or not. What matters is that you are in compliance.

nope, dont see any cf issues.

Can confirm

Not officially - status page says all healthy

They already announced that.

Yeah the graph for that one looks exactly the same shape. I wonder if they were depending on some azure component somehow, or maybe there were things hosted on both and the azure failure made enough things failover to AWS that AWS couldn't cope? If that was the case I'd expect to see something similar with GCP too though.

Edit: nope looks like there's actually a spike on GCP as well

Multi cloud is really hard to get right at scale, and honestly not worth the effort for the majority of companies and use-case.

Like AWS or GCP? https://downdetector.com/status/aws-amazon-web-services/ - https://downdetector.com/status/google-cloud/

I have never noticed that page being updated in a timely manner.

It shows that some people have issues accessing the portal.

still down

[dead]

I especially like how Nadella speaks of layoffs as some kind of uncontrollable natural disaster, like a hurricane, caused by no-one in particular. A kind of "God works in mysterious ways".

    > “Microsoft is being recognized and rewarded at levels never seen before,” Nadella wrote. “And yet, at the same time, we’ve undergone layoffs. This is the enigma of success in an industry that has no franchise value.”
     
    > Nadella explained the disconnect between thriving financials and layoffs by stating that “progress isn’t linear” and that it is “sometimes dissonant, and always demanding.”

    > These decisions are among the most difficult we have to make. They affect people we’ve worked alongside, learned from, and shared countless moments with—our colleagues, teammates, and friends.

Layoffs will continue until uptime improves!

> [Satya Nadella] said that the company’s future opportunity was to bring AI to all eight billion people on the planet.

But what if I don't want AI brought to me?

[dead]

Reports of Azure and AWS down on the same day? Infrastructure terrorism?

> We have confirmed that an inadvertent configuration change as the trigger event for this issue.

Save the speculation for Reddit. HN is better than that.

> Infrastructure terrorism?

Unless that's a euphemism for "vibe coding", no.

Don't visit this address.