One of the first things I do after getting an inquiry from a recruiter or friend referral is lookup the MX record for the company’s email domain. It is an anonymous one-command check to see if they’re a Microsoft shop.
If they are, it’s enormous personal red flag. MSFT is very popular so I’m only speaking about my own experience, but I have learned over the course of 20 years that an MSFT IT stack is highly correlated with me hating the engineering culture of an organization.
I know I am excluding a lot of companies with great engineering culture where I would thrive and who just happen to use Outlook/Sharepoint/Teams, etc. but it has had such better predictive power of rotten tech culture than any line of questioning I have come up with during interviews that I still use it.
I don’t mean any disrespect to MSFT-centric engineers out there - it’s not you it’s me.
I'm gonna be honest, you sound like a problem employee.
The companies not using Microsoft, are using Google. Which in my experience is equally or measurably worse.
Just personal data points, but every avowed Microsoft hater I've ever worked with has been... difficult. Like a-drag-on-the-team-because-he-refuses-to-use-company-tools difficult.
Edit: How does an aged post on this site go from +4 to -1 in the span of a few minutes?
Doing research on a potential employer and filtering out opportunities based on preferred toolchains is a green flag not a red flag.
Dev tools, sure. Self-selecting yourself out of the office/email toolset used by 90% of companies seems like a weird flex.
Companies that use Microsoft for one thing invariably use it for another, and then another, and then another, because they're "already paying for it". Their business model has always been like this.
Microsoft Office usage is highly predictive of lots and lots of other choices.
Teams is just so much more horrible than Slack and Zoom, and dev teams use Slack and/or Zoom.
Just because someone uses Outlook doesn’t mean they use Teams too. I’ve seen Zoom or Slack with Outlook/Office suite for the remainder at companies.
Most customers of both use O365.
The zoom fascination is pretty weird. It’s literally Webex 3.0 without Cisco bullshit.
Slack is pretty awesome. It wouldn’t factor in selecting an employer, but that’s just me.
I definitely wouldn't call Slack "awesome". Self-hosted tools like Zulip are doing a better job. Slack is however, the smaller evil amongst MS Teams, Zoom, MS Outlook and similarly bad software. Like, if someone told me all communication, including text chat shall happen via MS Teams, I would seriously consider looking for another job. It is a recipe for absolute disaster and completely broken communication. If the same happened with Slack, I would dislike it, but I guess it is at least usable. Still garbage, but not as much garbage, as MS Teams.
What do you do to make Zulip better than Slack? A vanilla installation is not better, and scales worse with more users, more devices per user more mobile users and more integration sources. But, I’ve never been in a situation where I was forced to make Zulip an attractive communication tool to an organization; there must be a lot that is possible. Getting away from a Salesforce product is a good goal.
I think the point is that GP red flagging all MS shops, which is more or less just sorting companies by headcount and flagging all from top, implies incompetency at GP's side than at the company side.
Like, if a fighter jet pilot came and told all American jets are equally weak and overcomplicated and ineffective, it probably tells more about that pilot than about the jets.
I don't know if that's the case, but that would be the idea.
SharePoint really is that bad though (and I say this as someone who used to develop for it as a platform).
The fact that it's so widespread in our corporate culture is more indicative of how enshittified it is. Now, realistically, we might not be able to avoid it because of that, but let's not pretend that it's not shit.
It fills a niche. What’s else does?
Yes, it’s not great, but so what?
What else? LaTeX Beamer, for one; Libre Office Impress for another.
You are confusing SharePoint with PowerPoint.
In this economy? This sounds like a fantasy.
Google is leaps and bounds preferable in my experience than Microsoft. I agree with the above. A Microsoft shop isn’t a guarantee the company culture is bad, but it’s correlated enough to be a flag.
As someone who has been accepting of MS houses and worked at a few, the heuristic holds up in my admittedly anecdotal experience. The Mac houses are fine and Linux houses have been best.
The chairman of my last big company said I was “ungovernable” at one of our last board dinners, so I’m reluctantly inclined to agree with you.
How can OP be a problematic employee when he's specifically decided never to become an employee of a company which uses such tools?
> How does an aged post on this site go from +4 to -1 in the span of a few minutes?
I just down-voted you, so I contributed to that.
OP bent over backwards to make it clear that he didn't mean any offense, and you opened with "you sound like a problem employee."
Windows is a parasitic drag-on-the-team.
Now, if Microsoft creates a Microsoft Linux desktop OS, that would be something.
That's basically WSL.
My work laptop is Windows, and the only native applications I run on it are a web browser, Zoom, and the company's VPN software. Everything else runs inside WSL.
I greatly prefer Debian to Homebrew, so if I can't run actual Linux, this is (to me) superior to trying to develop on a Mac.
I agree that Debian beats Homebrew. But wouldn’t a persistent Debian container on Mac be better? WSL is nothing more than a container on the system, no?
The Mac hardware is vastly superior to most Windows laptops, especially enterprise Windows laptops.
With Windows 11, WSL has X and Wayland support, so you can run graphical applications as if they're native (e.g. share the same cut-and-paste buffer, switch between windows using alt+tab, and so on). It's also much easier to attach USB devices like Yubikeys to an already-running container than the last time I tried to do the same with Parallels. (That was quite a few years ago, so maybe it's gotten better.) You can also launch Windows applications from Linux, which is makes it trivial to control my (Windows-native) browser from within WSL.
I strongly disagree about Mac hardware vs. Thinkpads or Framework, but to each their own.
> The Mac hardware is vastly superior to most Windows laptops, especially enterprise Windows laptops.
Man alive, what you mean is normie "Apple-style" Windows laptops with a bit of an "enterprise" makeover. Mobile enterprise workhorses (e. g. Panasonic, Getac)? Apple has no hardware in this segment. Detachables with extended five-year warranties plus certified dual-OS support? Nothing. Some of you fruit afficionados need to get out more.
You can do that at least for CLI apps with OrbStack. Not sure if it has X or Wayland support.
> Windows is a parasitic drag-on-the-team.
Not in my industry. And workstations, mobile or otherwise, on the clock? You work with what's certified and available. But to be fair, "Apple people", praise the Great Maker, are utterly irrelevant here. Hardware- and software-wise.
Well, in my experience every Microsoft shop I've ever interacted with has been a problem employer. Why do you feel your angle has greater moral defensibility?
I don't know man, you're gonna have a very tough crowd if you're gonna try and convince anyone that Teams is as good as Google Meet.
They are all equally crap. I'm convinced the people designing collaboration tools don't have to use them on a daily basis.
IME the call quality varies quite widely between video calling software. And being able to reliably hear and be heard with reasonable latency is pretty important!
The plague that is currently infesting our software industry is "Promo-Driven Culture". Employees are incentivized to get a promotion, not to make life better for anyone, except for their manager's promotion.
I’m sure the people who designed Teams and Meet use their own products on a daily basis. And if those are crap, what’s a better alternative?
It is funny, that even a Slack Huddle, something that's not even the core of Slack's function, is better than anything one gets with MS Teams. MS Teams is so laughably bad, I think I have never used a worse chat/voice chat/video chat program. Probably not even Skype in its single core days was worse, even though it ate one third of my single core CPU, just to have a call back then.
I currently work in a Microsoft shop that has Slack. Everyone uses Slack and all the Microsoft tools, including email, are crickets. This was never the case in the Google shops; we still used email.
Outlook is objectively a terrible experience.
Microsoft's softwares do not follow standards thus they hard to work with.
"using the biggest software suite tailored for offices/IT environments is a red flag"
honestly the things i read here sometimes hahaha
If this is "tailored", then I don't even want to know what how bad other MS products are. Oh wait, we can see that in Windows in general. But then again MS Teams is worse. It's almost as if the more MS has its fingers on something, the worse it gets.
If a company provides a Mac laptop, that to me is a green flag, if it provides a Windows laptop, that is a red flag.
The best company I ever worked at, provided every software engineer both a Mac laptop and a Linux desktop as standard equipment.
My employer provides a Mac laptop with the Office suite. Red flag, green flag, or yellow?
Word, Excel, and arguably PowerPoint are still the best tools im their respective classes, so if you mean those then very much a green flag.
If they're also making you use Outlook or especially Teams then they're going to start losing "points".
What if they provide both?
My calculations tell me that would be a yellow flag.
My knowledge of colors tells me red and green make brown.
What does a brown flag tell us?
I’ve definitely noticed a correlation with low regard for labor (h1b abuse). But maybe that’s just a location thing, I’m in California where regard for labor, especially local talent, is non-existent. You know, move fast and break things like nascent tech worker unions and the state itself.
WTF is this even supposed to mean?
H1Bs use Microsoft products more than others? Or they do it because they have to…or what??
Please explain yourself.
Companies more likely to want to save money on labor costs (employing many h1bs) are also likely to want to save money on Tooling costs, by using safe options like MSFT stuff, rather than finding better tools.
Also yes, due to availability and various other reasons, H1bs, particularly from India, seem more likely to use a MSFT stack.
MSFT tools aren't even cheap - they're very expensive. Many FOSS tools are just better and cheaper. End of the day, even RHEL is cheaper.
it's generally pretty remarkably bad. i think i agree. it sets a sort of psychological baseline culture that computers and their software should be shit, which is a pretty bad influence for people making software to be engaging with day in and day out.
Too bad Microsoft shops run the world. All the factories and shops, nearly every commercial backoffice runs windows, office/exchange and what not.
the software is so bad it's literally a national security risk.
How can you see from the MX record if it is Microsoft?
The "dig" command can get them for you
$ dig ycombinator.com mx
;; ANSWER SECTION:
ycombinator.com. 300 IN MX 20 alt1.aspmx.l.google.com.
ycombinator.com. 300 IN MX 10 aspmx.l.google.com.
ycombinator.com. 300 IN MX 20 alt2.aspmx.l.google.com.
ycombinator.com. 300 IN MX 30 aspmx4.googlemail.com.
I love this tool so much. It makes so many difficult things easy, and it does it cheaply or free in almost every instance.
My company uses a MSFT for domains, email, office work etc. but hands all the employees (not just engineers, HR as well) Macs. I don't know what kind of places you're working for but I'm not really interested in spending more time debugging your mattermost instance or email server instead of working on the core product I was hired to work on. I agree microsoft software is a plague but good luck convincing the people with the money to use something else lol
Companies that don't use Outlook? All five of them?
I've seen companies with varying levels of MS product integration but Outlook is pretty foundational.
Now, if a company says they use SharePoint or Teams to store their documentation, run to the hills. Wikis or bust.
God, Teams is absolutely miserable. Video calling on Teams makes you appreciate just how well Zoom works.
Teams macOS client? Crashes on startup, even after clearing all of my user data.
Teams iOS client? You can join a call by a link, but you can't see the call UI because it's behind the login window.
Teams on Firefox? No video support for years, and most recently just glitches out and shows an empty page when trying to join.
Teams on Chrome? Tried joining a meeting, and was told by the organizers that they couldn't admit me because the button wasn't doing anything.
I've had all four of these things happen within the last month, and it's made me want to tear my hair out. I get that none of these are "Microsoft Edge/native Windows client", but they could at least pretend to care about other platforms...
The Teams mac client is so awful I completely gave up on it
Over the years I have used teams on Windows, Mac, iOS, Android and various Linux distros (where I was limited to Chrome and Firefox due to lack of an official client). While it is certainly not the greatest tool in the world, I have never encountered issues like these.
You’re probably doing something cute with your network filtering or EDR.
This varies widely by niche. My experience is that a solid majority of West Coast tech companies / startups use Gmail or other non-MS hosted solutions. Outlook or MS365 are a good indicator that the codebase may be older than some of the people writing it.
Silicon Valley in particular uses Google Workspace at a much higher rate than the rest of the world. If you count every one- or two-person startup as a company, Google probably does have a solid majority. If you count mailboxes, Microsoft still easily wins.
Note that MX records are misleading here. They have no false positives, but are full of false negatives --- daisy-chaining MTAs is common, and since Microsoft owns the mailbox, it's invariably last in the chain. So the MX record will show something like Proofpoint (pphosted) or Mimecast or an internal company host, when really it's Microsoft in the end.
Wild to see the different experiences here. I haven't worked for a company that uses Outlook in 20+ years.
Recently it's all been gmail/google workspaces.
Similar experience; I haven’t had to use Outlook since the late 90s, and even then only for about a year.
Every company I worked for before or since just used IMAP.
What did you have as the IMAP client?
I’ve worked for six companies and only one of them uses Outlook. I think there is some availability bias by industry or job type. I know there are lots of companies that use Outlook, but you may be overestimating how many do, particularly among the companies more likely to be represented here (tech and/or startups).
Large enterprises (1000+ employees): probably 70-80%+
Mid-sized businesses (100-1000 employees): around 60-70%
Small businesses: more variable, maybe 40-60%
this reply was written by “AI” :)
I tend to work at banks, multinationals and power.
My direct employer uses GSuite (and Google docs as a source of record is as bad as a 2000s file share)
I've been at quite a few places that wouldn't touch the MS ecosystem with a twenty-foot pole, and history has proven that to be a wise decision on their part. It certainly has not cost them any business.
> Now, if a company says they use SharePoint or Teams to store their documentation, run to the hills. Wikis or bust.
It's never just Teams or SharePoint or a wiki. It's almost always some abomination created by putting various bits of knowledge on all three. Also, corporate wikis suck because how your team classifies data is almost invariably different from how someone else wants to see it.
SharePoint, for all of its flaws, typically gets used by the major announcement-and-policy makers at a company, because they just want to use MS stuff (primarily out of ignorance of alternatives), so at least it's somewhat coherent for everyone in the company.
As usual with all these types of posts, people go "HA HA, MICRO$OFT SUCKS" without understanding business practices that keep them afloat.
Don't use Exchange? Cool, what should we use instead? Does it support 15 people all the way up to 150000 people? I used to run Exchange cluster for 70k people, is there other mail software out there complete with non-shared disk redundancy? Where the users connect to single endpoint and software figures it out from there?
Sharepoint with another 2 RCEs. Not shocked, the software is terrible. However, it's only software that will stand up under load and let us shard it easily. All open-source software is one of those, runs fine in Homelab, likely falls down under load. Few Open Source Developers want to work on this stuff which I get because it's tedious work interfacing with computer illiterate end users. I'd rather chug sewage then do this work for free.
Finally, it's somewhat backwards compatible. Most businesses are filled with ancient software that no one has worked on in 20 years. That Excel document with Macros from 1997. With some registry changes degrading security posture, still works. I doubt you will find Office software with level of backwards compatibility unless they are using Microsoft Office level of compatibility.
Microsoft has real gordian knot here and few solutions besides "Backwards compatibility is OVER. Upgrade to modern or GTFO". Meanwhile, I get hit up by $ThreeJobsAgo over some Exchange Web Services solution I slapped together for them in Python they wanted me to upgrade to GraphAPI since Microsoft turned off Exchange Web Services in Office365.
I see you build a case for traditional MS product in Exchange, yet this issue is about Sharepoint.
Just like with Windows, Microsoft has built a moat with Exchange, but the question is why do all the companies buy into their full ecosystem, especially for anything relating to web technologies (you even bring up Exchange Web Services), because this they do really badly, and Sharepoint seems to be the worst.
However, I am certain there are big Postfix/Dovecot installations scaling easily to 150k people, but we probably wouldn't know about them. Eg. here a couple of accounts of people doing that: https://www.reddit.com/r/linuxadmin/comments/32fq67/how_woul...
I was running millions of accounts using Postfix/Dovecot on shared-nothing storage with a single MUA-facing endpoint and complex policy options, and that was over a decade ago.
Fastmail today would be much bigger again, and they’re on CMU Cyrus.
150k is rookie numbers. Perhaps that was meant ironically to satirise mediocre enterprise thinking?
>Perhaps that was meant ironically to satirise mediocre enterprise thinking?
It's a serious post, unfortunately.
Yep, my point was “What is the alternative besides other enterprise cloud like GSuite and others?”
Cool. I did that with qmail in 1998 on a couple of Ultra 5s.
Try managing a calendar or booking resources.
Integrated CalDAV is also available. Not in qmail, however. The patch for that would be large.
> but the question is why do all the companies buy into their full ecosystem,
Old manager I had one told me: "I wish Microsoft made all the software in the world because it works so well together!" He was the guy who bought our company a one-way ticket to O365. He was also woefully tech ignorant and could barley drive software outside of office programs.
Yup, proves the old adage that you never let the tech fluent make tooling decisions for normal people. Nothing would kill a large orgs momentum faster than half their employees stuck reading man pages for trivial tasks.
Microsoft is a good black and white, you can do this or you can't. Which works better organizationally than the "I bet I could hack this together in a few weeks" and have everyone wait around so one "10x dev" can feel like a special snowflake
Not sure the total number, but a university near me serves 50K active students and hundreds of thousands of alums with Postfix/Dovecot.
Craigslist has also uses Haraka to scale their email.
https://haraka.github.io
There are plenty of open source email alternatives now days.
I used Exchange because it was what I most familiar with. SharePoint operates in similar matter with all sharding (though backend is still MSSQL with it's sharding last I checked)
Sure, PostFix/DoveCot will scale if you are doing just email. Once you add GroupWare requirements, PostFix/Dovecot are no longer in same boat.
How oh how did these nuclear weapons facilities manage to function in the days before Exchange and Sharepoint?
Just like everyone else before invention of Email and Document sharing? However, like every other business, no one is willing to slow down velocity for security reasons so now we are here. Unless you have a fix for "Line must go up", market pressures will always cause this.
> market pressures will always cause this.
Market pressures dominate nuclear weapons development?
Sure, all the “Let’s run government like a business” types. Cut IT budget and outsource to contractors who want maximum profit.
Um, email was invented, like in the last millenium, well before Microsoft was a thing (only slightly sarky)
Microsoft was a thing before email.
Microsoft was founded in 1975. The standard for SMTP wasn't published in 1981. Most early predecessors were the late 70s.
https://en.wikipedia.org/wiki/History_of_email
In 1971 Ray Tomlinson sent the first mail message between two computers on the ARPANET, introducing the now-familiar address syntax with the '@' symbol designating the user's system address.[2][3][4][5] Over a series of RFCs, conventions were refined for sending mail messages over the File Transfer Protocol. Several other email networks developed in the 1970s and expanded subsequently.
Proprietary electronic mail systems began to emerge in the 1970s and early 1980s. IBM developed a primitive in-house solution for office automation over the period 1970–1972, and replaced it with OFS (Office System), providing mail transfer between individuals, in 1974.
How many organizations on the planet require their Exchange server to support 150k users? I doubt most manufacturing plants fall into this category.
They don't but whole point is massive Enterprises use the software, people get accustomed to it and want it in their smaller business. So, Microsoft Small Business Server is developed until O365 came along.
You can use hosted versions of Google Workplace or Office365 if you can’t figure out how to secure software (places like this typically can’t clearly). Additionally it enforces a separation of concerns where a compromise of your email server doesn’t lead to a compromise of the plant itself (again - clearly IT didn’t know how to partition the network into different parts).
Sure, this business should have converted to either of those and let someone else take over administration since they were clearly negligent. This is stuff that FedRAMP or it's replacement was supposed to fix but didn't.
FedRAMP is only for hosted software for the federal government afaik, not on-prem and not private companies (nuclear reactors afaik are operated by grids/private operators and the federal gov is responsible for auditing and regulating)
> Sharepoint with another 2 RCEs. Not shocked, the software is terrible. However, it's only software that will stand up under load and let us shard it easily. All open-source software is one of those, runs fine in Homelab, likely falls down under load. Few Open Source Developers want to work on this stuff which I get because it's tedious work interfacing with computer illiterate end users. I'd rather chug sewage then do this work for free.
All just empty claims without showing any evidence. Did you ever set up a multi-client syncthing setup to test your theories about it falling over? Or do you have any references, pointing us to analysis, that shows, that any such tool doesn't hold water? What about some bit torrent setups? There are many options in this space, and one doesn't even have to lump synchronization and viewing in a web UI into one service. If one doesn't, then there are many tools that can accomplish the job better than Sharepoint.
And btw. paid MS Office doesn't even hold water for some 80 people, delivering me my e-mails some half an hour later, at a snail's pace, one or two a minute, while my 1 EUR per month free software using e-mail provider (posteo) manages to give me all my new e-mail almost instantly, the moment I open Thunderbird.
Your replacement for Sharepoint is BitTorrent or Syncthing?
Yes, there is other tools, none of them is as integrated as Microsoft suite except other cloud only options like Google Workspace and other cloudy software.
Exchange has valid arguments for it, but I don't think SharePoint has anything going for it other than "we already got a license for that as part of out package deal". As software in its own right, it's uniquely bad even for Microsoft.
I mean this is nuclear wepons were talking about, who cares about features vs security? They could run the department on snail mail if they tried
Sharepoint is enterprisey and all but how about "less software/surface area is more" when it comes to nuclear silos?
Why is this comment glowing? \s
Hahaha, how stupid must anyone be to deploy SharePoint anywhere near anything of national security relevance! How can it still be a thing, that anyone entrusted with such sensitive matter dates to even touch MS products of the kind of SharePoint? That includes the complete MS Office 365 disaster suite, MS Teams and Edge.
Sounds like they need to seriously redesign their security policies.
I have some reaallllly bad news for you on that front.
What would you recommend instead?
For security-critical or sensitive situations, auditability should be a requirement. That implies access to source code and capabilty to build it.
Decisions like these need to be done from first principles. SharePoint shouldn't even have been a contender here if looked at seriously. Do your own homework.
Think you answered just about everything except the question asked
Doesn't Microsoft have government programs that grant source code access for products like Windows and (probably) SharePoint?
Wait until you hear about the guy storing Top Secret Nuclear documents in the public toilet of his resort....
In general you'll get downvoted if you're talking about any politician or political party. You are allowed to shit on (or advocate for) the government doing stuff tho.
Or the one that invites journalist to Signal group during combat mission.
But, look at everything we get for free! /s
So I once brought down an alerting system using Excel
(btw, this story is more about unintended consequences instead of MSFT)
- I own an alerting system
- For log based alerts, it looks for a keyword e.g. "alert_log"
- I make a spreadsheet to track data about alerts and call one of the sheets "alert_log"
- Alert system starts going crazy: using tons of CPU, number of alerts processed goes through the roof but not a lot of alerts generated
- Turns out that I was using the cloud version of Excel so any text entered transited the firewall
- Firewall logs store the text "alert_log"
- Alert system thinks it's an alert BUT it's not a real alert so triggers an alert processing alert
- That second alert contains the text from the firewall log and so cycle begins
In other words, systems can operate in weird ways and then cause things to happen you didn't anticipate. It's why things like audits, red teaming and defense in depth all matter.
As a firewall engineer I have to tell people to make sure to disable traffic logs for syslogs from the firewall for this reason.
Reminds me of the time I set up tcpdump to log network traffic on a troublesome server. To save disk space I sent it over SSH to my laptop. Oops!
If it is that bad why don’t we see it being exploited at scale? I work with many Fortune 500 companies and I would say 9/10 use SharePoint. Also some deployments are much better than others, so I would rather say many implementations of SharePoint are shit but if done right it’s actually pretty solid. There’s really no better alternative unless you want to maintain 5-10 separate tools owned by multiple vendors. I also don’t get the hate for Teams. I use Zoom, Slack even Discord for work and don’t have strong feelings for Teams. I can take calls, join meetings from my calendar, record them and summarize them with Copilot. I don’t need anything else and Teams does that just fine. I do like Discord ability to share multiple screens and jump into a channel to collaborate, particularly useful when debugging or pair programming.
Sharepoint is one of the worst, most bug-ridden softwares I've worked with.
It has a bug with Solidworks (3D design suite) that sporadically makes files completely un-openable unless you go in and change some metadata. They are aware of this, doesn't seem to be any limitation preventing them from fixing it, and it has sat unfixed for years.
Microsoft's cloud storage as a whole is an insane tangle where you never know where you'll find something you're looking for or whether it will work. Some things work only in browser, some only in the app, zero enumeration of these things anywhere.
Completely unsurprised and I'm sure there are many more vulnerabilities ripe for the picking.
Every time I need to touch anything made my Microsoft lately I am met with multiple levels of glitchyness, straight up bugs, most frustratingly it’s so excruciatingly slow.
Recently I tried to configure a new subdomain to handle mail on 365 and even finding their DKIM configuration section was a mission. Once finding it, I learned that their DNS check fails to properly handle subdomains for email, so you have to put their DKIM keys against your root domain. Genius!
But wait! 35% of Microsoft's code is now written by AI so surely it will get better
Yep, especially after laying off several thousand veteran engineers (who, in many cases, were the only ones with a solid understanding of how a given product works as a whole, and why it is the way it is).
I'm working on a gov contract right now and they're forcing everyone to migrate off of Slack and into Teams. I somehow have managed to avoid MS corporate products for the better part of two decades. People's tolerance to UX pain seems to be boundless in corporate/fed worlds.
We sync content to MS hosted Sharepoint using rsync. When the file arrives, they change the internal metadata inside the file, which changes the checksum, which causes rsync to think the content is different and needs syncing again.
Edit to say: this is for MS files like Excel docs
Is that a supported method?
Supported by who? Microsoft?
If a file server breaks basic Unix tools it should be unplugged and put in the garbage.
Microsoft Word online deletes text in Firefox Linux (maybe others too) for at least two years now [1]. The one thing you want a text editor to do is be able to write text into a document, and somehow this bug goes unfixed. You would think it would be priority #1 for paying customers of Business Office 365 - and yet nothing.
It ended up being easier just to switch to paid Overleaf and teach our non-tech members how to write LaTeX and/or use the built-in editor. The documents are beautiful, Overleaf doesn't miss a beat and we are very happy with their solution.
Microsoft should be ashamed - I don't know how anybody would ever consider using them for any serious production work.
[1] https://learn.microsoft.com/en-us/answers/questions/5216132/...
Not defending Microsoft in any way but my guess of what's happening:
* Too few people use Firefox to access Office online, they don't care
* Your organization is too small for them to care
Firefox is the only browser other than Chrome (and derivatives) on their OS. The web is supposed to be multi-platform. I guess it isn’t that surprising that modern MS is happy to just live in Google’s ecosystem though.
if they will lose data when you're on a rarely used browser, can you really trust them not to lose data in general?
"yes, your car exploded, but you were driving on a dirt drive way. it works just fine on the highway"
I am a social worker and SharePoint is unfortunately widely used by nonprofit agencies for storing client records. It's a real shame, but they can't afford anything better.
That bug has been around for years. I always wondered if that was deliberate. I guess that Microsoft support answer settles the question...
>Sorry for that we may have no enough resources about the Linux environment.
It's such a critical backbone to so many of their services but they treat it like a forgotten stepchild for the most part
They've managed to mess up sharepoint even worse lately.
I went there to try to find where company meetings got recorded to.
I went to my sharepoint bookmark, which weirdly is www.office.com after some previous nightmare rebrand.
Except what used to be the way into your sharepoint files, is now just a full page copilot screen with no hint of where the fuck your files are.
Even though you've been visiting this bookmark for years, to get to your sharepoint files.
Ok, so you search bing sign into sharepoint.
Top result is office.com . You ignore it.
Next result is:
https://support.microsoft.com/en-gb/office/sign-in-to-sharep...
This links you to https://m365.cloud.microsoft/
Ok great. Nope! Redirects you back to copilot.
I do NOT want to ask copilot to dig out my files every time you want a file. I want to get back to the directory listing so I can find the directory listing to find the company meeting recording.
How does MS not understand that replacing all UX with copilot is not an improvement, and is not helping sell copilot.
Kilobytes or single digit megabytes. It happens because Sharepoint sporadically alters created/edited metadata for any (?) file it stores. Most programs don't care about that but Solidworks does.
Developed and maintained in China by Chinese nationals, with untechnical escorts overseeing their work.
As a company that supports OT systems we hate seeing level 5 in the Purdue model with direct write access to level 1 and 0.
Thanks CJ, I live with that chart, but forget maybe most don't. And to add 4 to level 2-0 can also be an attack vector, but seeing straight 5 to 1-0 happens more then people want to admit even with the "firewalls"
MSSQL is one of the few Microsoft products I would consider to be genuinely decent. Like, there's a lot of idiosyncratic stuff there (but then that's also true for Oracle), yet the feature set and stability are good.
Whoever puts a nuclear fission facility on the internet should be put behind bars.
It is not a nuclear fission facility, it is "a plant that produces the vast majority of critical non-nuclear components for US nuclear weapons".
The also targeted the IT side, not the operational side, which, according to the article is likely to be airgapped. Even sensitive production facilities need some internet access, people work there and like everyone else, they need food, office supplies, toilet paper, etc... they can't be cut off the rest of the world completely.
Something tells me they also use it to order operational side materials, including nuclear gear and materials, from the IT side. To expose this on the internet screams of idiocy.
The timeline here is interesting. Microsoft releases info and instructions for mitigation on July 19, and a more complete report on July 22nd, here's a copy of that:
https://archive.ph/plNZU
Then according to this report, 'sometime in August' the exploit is used against the Honeywell-managed nuclear facility, since it wasn't patched, if I read correctly? So it really could have been anyone, and it's hardly just Russia and China who have a record of conducting nuclear espionage in the USA using their nation-state cybercapabilities (Israel?). As the article notes:
> "The transition from zero-day to N-day status, they say, opened a window for secondary actors to exploit systems that had not yet applied the patches."
Also this sounds like basically everything that goes into modern nuclear weapons, including the design blueprints. Incredible levels of incompetence here.
> "Located in Missouri, the KCNSC manufactures non-nuclear mechanical, electronic, and engineered material components used in US nuclear defense systems."
There needs to be a law that all nuclear and nuclear-adjacent facilities have no connection to the Internet. The fact it's allowed is unbelievable.
It's believable when the industry has pivoted to pushing SaaS garbage in every place imaginable to the point that on-prem solutions don't exist anymore. Do you expect them to not use email either?
Remember, the industry told us we're in a 'zero trust' world now. The network perimeter is an anachronism.
OTOH you know damn well they keep the important stuff airgapped, in which case the title (and your predictable reaction) is just fanning the flames. It could very well be they 'breached' the receptionist's PC she uses to browse Facebook to pass the time.
I have some sad news for you, about the realities of "airgapped security" IRL.
It starts with military officers using the hallway photocopiers for secure documents, and ends with TS docs stored in a Florida hotel's restroom.
From the article:
> OT cybersecurity specialists interviewed by CSO say that KCNSC’s production systems are likely air-gapped or otherwise isolated from corporate IT networks, significantly reducing the risk of direct crossover. Nevertheless, they caution against assuming such isolation guarantees safety.
This was also not a nuclear facility, however. The article says it makes "non-nuclear components".
In my experience auditing critical infrastructure, most facilities are "air gapped". I put that in quotes because while you can't browse the Internet from the control network(s), there are ways to exfiltrate data. The managers, engineers, regulators, and vendors need to know what is going on in real-time. Back in the day this could've been a serial port connecting two systems for a one-way feed. Now I imagine it's something far more sophisticated and probably more susceptible to abuse.
As an example, you might have a collection of turbines manufactured by GE and GE needs to have real-time data coming from them for safety monitoring and maintenance. The turbines might have one connection for control traffic and another for monitoring. How to secure these vendor connections was always a debate.
Btw, there are strong cybersecurity regulations around critical infrastructure. CIP-005-07 covers security perimeters. You can view them here: https://www.nerc.com/pa/Stand/Reliability%20Standards%20Comp...
Ah yes, "likely air-gapped", what a high-confidence statement. Any competently designed air-gap must be precisely auditable and demonstrably, positively air-gapped.
The only world where "likely" is a reasonable word is in reference to possible physical taps or a precise enumeration of physical access points that went unaudited, but have reliably followed safe access control/configuration procedures. Anything else is plain incompetence.
KCNSC is a large organization that will have hundreds of distinct networks at different risk and control levels. Every variation of "public internet" to "single-site air-gapped network" probably exists there, including many levels in between like multi-site secure networks and networks with limited internet connectivity. Many networks air airgapped, this sometimes means that they consist of a small number of assets in a single room, and it sometimes means that they have connectivity to airgapped enclaves of AWS and hundreds of other military, government, and contractor sites. All of these controls will have been determined by a combination of risk scoring, compliance policies, legal requirements, office politics, and happenstance. Multiple contracting authorities will periodically audit many of these networks against various standards, which may or may not allow connectivity to specific other networks depending on risk levels. Connectivity between networks is sometimes controlled by NSA accredited cross-domain solutions and multi-level security systems that enforce complex policy, in other cases it's controlled by an administrative assistant with a DVD burner. There will be case-by-case risk analysis decisions made for specific systems, ultimately signed off by a government official who may or may not have read them. Inevitably some of these will appear reasonable and cautious in retrospect and others will not.
The root fault with this article, and the resulting discussion, is the extent to which it generalizes over one of the larger organizations in a very complex part of the defense industrial complex. Many parts of KCNSC's operations are absolutely not exposed by this incident. Other parts absolutely are. Determining which fall into which category, and to what extent that is acceptable, keeps quite a few people employed.
> Anything else is plain incompetence.
It's an answer from talking heads, not from people from the facility.
They have multiple networks. One of them is definitely airgapped (red for RD). The medium security one is protected by annoyingly strict network ACLs (yellow for ITAR). Then there's a low security one for stuff like sharepoint (green).
This article is full of nonsense and speculation.
The standard you linked literally talks about: "High Impact BES Cyber Systems with External Routable Connectivity" and "Remote Access Management" for "High Impact BES Cyber Systems". That explicitly indicates non-airgapped critical systems. Furthermore, the proscribed auditing specifically spells out "network diagrams or architecture documents" as good evidence. Obviously, that is a high level document, but I see nothing to indicate robustness against state-level actors which are a expected threat.
How do you go about positively demonstrating such a system is air-gapped?
Speaking from past experience with the DoE (I'm happy I don't need to deal with security like this anymore), there were constant and randomized checks to make sure fiber cables (they were all fiber to make it harder to tamper with and to avoid accidental RF) were fully visible (e.g. not hidden under a desk or something) and not tampered with. Also, lots of locks and doors, both electrical and mechanical. The guy at the front desk with a big gun probably helped too.
While we're at it "and not use Microsoft products". Literally every time a story like this surfaces...
That's more of a form of survivorship bias. Microsoft continued to maintain its lockdown on government IT and infrastructure through the decades, over the alternatives.
> While we're at it "and not use Microsoft products".
I'm not sure if Oracle would be better.
I don't think any Microsoft Surfaces were involved in this..
Wasn't the internet literally created by the military for military comms? The decentralized routing was in part to ensure that comms could survive some areas being taken out by nuclear weapons.
As the effect of yesterday's AWS event demonstrates, the major Amazon, Microsoft, and Google data centers are surely top tier targets in every adversary's war plans.
The decentralized internet is less of a reality today than it was years ago.
Don't we have more internet submarine cables and less single points of failure in our internet infrastructure today than years ago? If so, shouldn't that make it easier to route around failures?
The web though I agree isn't very decentralized.
Considering that the AWS outage took out a lot of lines of communication (email, video, chat systems) for both commercial and government entities, I'd say that US-East-1 is a pretty big single point of failure. Even if it didn't result in infrastructure impact directly, if there was some kind of infrastructure issue and you had delayed or unavailable communications, how would you know? How quickly could a response be mounted? There's some parts of the infrastructure that could damage themselves irreparably in the time it would take to to fix the outage or get comms routed through a backup channel - like parts of the electrical grid or water treatment plants.
An attacker (read: nation-state actor) wouldn't even need to take down US-East-1, it could just take advantage of the outage.
I assume (hope?) there's some kind of backup comms plan or infra in place for critical events, but I don't actually know.
Maybe yes in that regard. But in the past, most organizations ran their own mail and web servers. Software supporting the business ran on-prem. Now they use Google or Azure or AWS. So business and civilian usage, at least, seem more vulnerable now.
We sacrificed resillience for effeciency. Now things are much more fragile and liable to exploitation.
That's fine, when all the nodes run autonomously and the internet is only used for real information sharing. What we now have is that the nodes are display control servers and all the computation and storage happens externally. That is not how it was designed by the military.
The very very earliest form of some of the protocols involved it were, yes. But not really now at all. That "internet" would not be worth using.
Wasn't it literally designed for that specific task? As a robust C&C system during nuclear war? The fact that we're doing it wrong doesn't mean we need to pull the plug on everything. How else do you survive WWIII?
https://ieeexplore.ieee.org/document/5432117
That only works, if the nodes still operate just fine, without the Internet.
You don't. Internet or not.
> needs to be a law that all nuclear and nuclear-adjacent facilities have no connection to the Internet
You want to make everything about a nuclear facility bespoke and subject to air-gapped drift? What about the guard booth that verifies peoples access, the receptionist who schedules meetings, and the janitor who wants to watch YouTube on his break? It seems unrealistic to lump everything that goes on at a nuclear facility under this umbrella.
Opening up the internet to a nuclear facility so that the janitor can watch Youtube seems preposterous. People can afford to do things slower for the sake of security. Having things typed out, verifying security via phone calls, etc like it's the 1970s seems reasonable to me. Does it really matter if things aren't fully optimized for speed and convenience in nuclear facilities?
IRL the way we do it is separating the business network (Youtube, finance people, HR, etc.) from the operational network (relays and sensors). You use data diodes to send business-critical data from the operational network to the business network.
Also, the Kansas City Plant is like a watchmaker's factory, not a power plant. They make widgets and gewgaws, not literally split atoms.
> really matter if things aren't fully optimized for speed and convenience in nuclear facilities
For hiring and retaining people, yes. It's understood that the "guts" of what's happening at these facilities needs to be locked down to the max. But, for supporting roles you need to be able to bring people in off the street without 1) a bunch of specialized training on your bespoke way of doing things, and 2) making your employees less attractive on the job market.
Just my opinion, though. Maybe I'm completely off base but it doesn't seem like a good idea to me long-term.
Being airgapped didn't help Iran avoid Stuxnet.
That also had a HUMINT element.
There is likely a small number of people who could collectively list out the events it _did_ help Iran avoid.
Defense in depth is still valuable.
No, but it made the attacker's job 10000X more difficult.
It is funny to read this kind of comment knowing at the same time this kind of stuff was happening while the launch codes were 0000000 or some such non-secure code. At same time, the computers in the nuclear launch facilities were still using 5.25" floppies. I did wonder how often they were loading updates from those, if ever.
I heard that once you put up a website on the public internet, it would immediately gets attacked by all kinds of scanners or other worse things. Not sure if it's true as I'm not a web guy.
All IPv4 addresses, domains (maybe more so for recently-registered ones), and subdomains from Certificate Transparency Logs (for HTTPS certs) are all constantly checked and poked.
Back in the day, I made the mistake of hooking up a fresh Windows XP (at least I think it was; pre-SP2) install directly to the internet. There was no firewall or NAT to protect me. The machine got pwned almost immediately.
Every public IPv4 address is port scanned multiple times a day.
Watching my website's firewall and ssh logs show all the various hacking attempts is calming in the same way that watching waves crash on to the shore is.
More like looking a thin net preventing mosquitoes from biting your skin, as there is some intention behind it, not just physics.
Per day? per minute or second.
Which really isn't a problem, unless you're being scanned so much your bandwidth is being overwhelmed. Certainly not the case for me, despite having port 80 and 443 open
I have a server that has a slow (5s) response to unknown pages, returns it as 200, and makes the next failing request even slower (for unauthenticated users). That seems to keep the number of requests limited. Perhaps I should just drop the connection after a certain number of requests.
BTW, quite a few of these port scanners are companies that offer to scan your ports for vulnerabilities. Temu pen testing, so to speak.
Do you configure this in your firewall? How can I replicate this?
what firewall do you use?
IIRC Carnegie Mellon did a study years ago which showed that you could not unbox a new Windows machine, connect it "directly" to the Internet, and get it fully patched before it was pwned.
You mean its a bad idea to slap a Starlink dish in the same building as the nuclear football?
Which breach was that again?
I mean there were also rules about non-sanctioned network connections in the pentagon, or using only sanctioned apps to discuss secrets, but thats not really been enforced recently.
Just wait until these places get flooded with vibe coded stuff that even those deploying it have little understanding. What could go wrong!?
Sleep well.
Microsoft could have been sold this with a special "nuclear license".
> needs to be a law that all nuclear and nuclear-adjacent facilities have no connection to the Internet
Why the special treatment for nuclear? Do you really think redlining a dam or storm-levee system would be less damaging?
Also, turning off internet connections means less-capable remote shut shut-off. Less-responsive power plants. Fewer eyes on telemetry.
We should be mindful of what is and isn't connected to the internet, and how it's firewalled and--if necessary--air gapped. That doesn't mean sprinting straight for the end zone.
> Also, turning off internet connections means less-capable remote shut shut-off.
Why does it have to be remote what's wrong with it being in-house? Besides a shut-off should never be able to be triggered remotely.
The same goes for digital emergency shut off buttons; all should be physical.
> Less-responsive power plants.
What? How is remote any more responsive than physical workers being in-house?
If power-plants operated efficiently back in the 50's without internet, they should be able to now without internet.
> Why does it have to be remote what's wrong with it being in-house?
Nothing wrong with it being in house. But having a back-up is never bad.
> How is remote any more responsive than physical workers being in-house?
If the on-site workers are incapacitated. It's a remote (hehe) risk. But so is foreign hackers doing anything with our nukes.
> If power-plants operated efficiently back in the 50's without internet, they should be able to now without internet
If you're fine paying 50s power prices again, sure, I'm sure a power company would happily run their plants retro style.
> But having a back-up is never bad.
It is always an increase in risk, in a security sense.
> $0.32 is $0.41 accoreit BLS, which is less than I'm paying today
Out of curiosity, what was the real power price where you live in the 60s?
good argument against having nukes
One can paraphrase the joke about democracy for nukes. Having nukes is the worst, other than every situation where you don’t have nukes and the other guy does.
The one exception I can think of is remote shutdown in the face of a rapid natural disaster. Like how the japanese train network is set to shut down rapidly when a high power quake is detected.
But that is very geography dependant.
Fine, keep it on the internet. But SharePoint, seriously? A 15 year old version of nginx pointed to the ~/.ssh folder is more secure.
Does this kind of thing happen to China + Russia?
I don't see news about that much - but to be fair, I am not looking for it.
They may also be less likely to admit it or allow any reporting on it
yes.
but it doesn't get covered by western media.
much like how NATO airplanes violating Russian airspace
is not reported about either.
Yes, recently some russian airline was hacked, they also used microsoft mail servers
they breached it* meaning that they had access to their "Welcome !" page in sharepoint lol.
That guy who jumped the office chair will be the end of us all
The jump was amazing though! At his age.
Say it ain’t so. Another Microsoft security problem? Inconceivable!
A flaw? In Sharepoint?
I'm shocked. Shocked, I tell you.
Side gripe:
I'm sitting here with a very performant computer running its native web browser.
It's ridiculous that I kept losing my place in that article because the page kept getting shifted to fit yet another damn ad (there were at least three in-view at all times as I was looking at it) onto the screen.
Either make the ads fast and don't load the page until they're all there, or better yet, admit that online content isn't a way to make your private equity group even more obscenely rich, and cut back on the monetization that you put on it.
When I try to access sharepoint files in my browser, the site goes through 37 redirects (thanks single sign on) shows all the files, then despite me very obviously being fully authenticated, it pops up a modal that says "sign in to see files", and I click "Cancel" and then I get to actually interact with the files.
What?
Gee, who would have guessed this isn't secure.
No, they did not breach anything through SharePoint. The flaw is that IDIOTS exposed these servers to the Internet. I am very pro holding vendors accountable but this is just stupid. "Pro-tip" btw. SharePoint installations often have the pw sharepoint, sharepoint123, sharepoint-123 and so on in various casing and delimiters.
Microsoft is a national security threat but no one cares because they automate genocide.
The down votes are a signal of how much Zionists have control on this platform, but I suspect mostly from India. Those guys have a huge problem... SMH.
