citricsquid
6 hours ago
As the person ultimately responsible for the Minecraft Wiki ending up in the hands of Fandom, it is great to see what Weird Gloop (and similar) are achieving. At the time of selling out, the Minecraft Wiki and Minecraft Forum cost tens of thousands of dollars per month to run and so it didn't feel too much like selling out, because we needed money to survive[1]. 15 years later, the internet is a different place, and with the availability of Cloudflare, running high-traffic websites is much more cost effective.
If I could do things over again, on today's internet, I like to believe Weird Gloop is the type of organisation we would have built rather than ending up inside Fandom's machine. I guess that's all to say: thank you Weird Gloop for achieving what we couldn't (and sorry to all who have suffered Fandom when reading about Minecraft over the years).
[1] That's a bit of a cop out, we did have options, the decision to sell was mostly driven by me being a dumb kid. In hindsight, we could have achieved independent sustainability, it was just far beyond what my tiny little mind could imagine.
Svip
5 hours ago
I was approached about a decade ago to combine The Infosphere with then Wikia's Futurama wiki. I asked it was possible to do a no-ads version of the wiki, and while initially they seemed like that might be possible, they eventually said no, and so we said no. So now there are two Futurama wikis online. I still host The Infosphere, haven't checked the Fandom one in years.
Fortunately for me, Futurama isn't as popular as Minecraft (for some reason!), so I've been able to pay out of my own pocket.
Svip
4 hours ago
A bit of a follow up to this; after a bit of thought, I am considering reaching out to Weird Gloop. I do not feel I am able to give The Infosphere the care that it deserves. And with Futurama back on Hulu, we are naturally seeing an uptick in activity. We have a very restrictive sign up in place, because I don't have time to moderate it anymore. It keeps the spam down, yes, but also new users away.
Note: The reason I'm writing I'm _considering_ reaching out and not just straight up reaching out is because the domain itself has a different owner than me, and I want to make sure they are also approving of this decision.
echelon
an hour ago
Their growth people emailed me again and again and tried to do the same with StrategyWiki decades ago.
Here's one of their emails:
> [Redacted] mentioned that your site was very cool - and that you're heading off to college. As you may know, Wikia is founded by Jimmy Wales (of wikipedia fame) and we are trying to become THE resource for all gamers
> I was wondering if you'd consider moving over to wikia now that you're going to might have less time with your studies. As an incentive I can point to a few things that might make the move easier
> 1. We have cool new software - gaming.wikia.com lets users do more blog-like contributions in addition to wiki editing - new social networking tools on the wiki - see our test server at http://sports.box8.tpa.wikia-inc.com/index.php?title=User:[R...
> 2. We could also hire you to help moderate the strategy wiki and other wikis if you need some beer and pizza money :-)
> 3. or we could offer to pay all the hosting costs and share some of the ad impressions/revenue with u
> If nothing else, I'd love to chat by phone and get to know you better.
> Let me know if that'd be ok :-)
babypuncher
2 hours ago
The Infosphere has always been one of the best fan wikis out there, thank you for your hard work (and for not selling out to Fandom)
ryukoposting
6 hours ago
I remember reading the Minecraft wiki back in the early 2010s, back when Fandom was still Wikia. It would have been much more appealing at the time than it is today - not just for the reasons you list, but because Wikia actually kicked ass in the early 2010s. It was sleek, modern, and easy to use. And today, it isn't.
epiccoleman
5 hours ago
Every time I wind up on some garbage Fandom page I reminisce about the good old days of Wikia. I remember many a fun night trawling through pages while playing Fallout or Skyrim or whatever - all the information you could ever need, right there at your fingertips. It's an ethos you don't see so much on the modern net.
iamacyborg
an hour ago
It’s funny that people are now looking back at wikia fondly because at the time most folks thought it was full of ads and shit. To the point where Curse/Gamepedia managed to get serious market share by not screwing with the community in the same way at the time.
Funny how they somehow managed to make it worse.
hinkley
an hour ago
How did Curse end up making money?
iamacyborg
18 minutes ago
Lots of ads across their wiki and other community websites and D&D Beyond was remarkably successful.
mossTechnician
5 hours ago
Wikia is a great example of enshittification - provide great value to users, then take it away from users and hand it to other businesses (eg advertisers), then take it away from businesses too.
Will Weird Gloop inevitably suffer the same fate? I hope not.
diggan
5 hours ago
> Will Weird Gloop inevitably suffer the same fate? I hope not.
Unless explicitly structured to prevent it, my bet is it will. If it's backed by a for-profit entity, it'll eventually need to turn a profit somehow, and users/visitors are the first to lose their experience at that point.
However, if Weird Gloop is a properly registered non-profit with shared ownership between multiple individuals, I'll be much more likely to bet it won't suffer the same fate.
I skimmed around a bit on the website to try to get an answer to if it is an non-profit, but didn't find anything obvious that says yes/no.
cookmeplox
5 hours ago
We're already turning a profit! And there are no third-party investors (or debt) – it's all controlled by wiki people[1]
diggan
5 hours ago
Aw, I take that as it is in fact a for-profit company already.
Regardless, I wish you luck for the future! May you not go down the almost inevitable enshittification hole.
robotnikman
4 hours ago
At least it is a private company though, meaning they are are required to make constant year over year gains for shareholders and investors. They have much more control over where the company goes and how it operates.
ChadNauseam
3 hours ago
publicly traded companies are not "required" to make constant year over year gains for shareholders and investors, that is just what the owners usually decide to tell the company to do. The owners of a privately traded company could decide to, and the owners of a publicly traded company could decide not to. For example, zuckerberg controls 53% of the voting stock of facebook, so whatever zuck says goes and if other shareholders don't like it they can kick rocks. This is pretty much the same situation that people imagine is the case with privately traded companies, even though facebook is obviously publicly traded.
atomicnumber3
3 hours ago
"that is just what the owners usually decide to tell the company to do"
Because the entire system encourages it. The market rewards growth FAR more than it rewards a consistent dividend payout. (See: companies growing 40% YoY command a significfantly higher earnings multiple than those growing 10% YOY). So imo this is a like saying "people could decide to just invest money and then not seek the best returns possible." Also remember these shareholder are seldom John Smith principled human retail investor. It's firms whose entire purpose themselves is to seek maximum return.
"The owners of a privately traded company could decide to"
Meanwhile this DOES actually happen sometimes. See: Valve. We all know there's ways Valve could put up really great growth numbers for about 2-3 years while completely destroying all of the things that make Steam so god damn compelling to users that they can command the same cut as Apple, on an OPEN platform (vs Apple fighting utterly tooth and nail to keep iOS 100% airtight locked down). But they don't.
"For example, zuckerberg controls 53% of the voting stock of facebook, so whatever zuck says goes"
TBC most founders/CEOs are NOT majority voters in their companies. They answer to the board. Most company founders lose voting control. The fact that Zuck is still in control is incredibly unusual and is a testament to how fast Facebook has grown that he's been able to keep hold of the reins.
Reason077
2 hours ago
Elon Musk is another CEO in total control. Although Tesla is a public company and therefore has a board, it’s stacked with Elon’s allies/appointees and answers to him, not the other way around. Despite Elon not being a majority owner of Tesla stock.
And when he took over Twitter in 2022, he immediately dissolved the board and fired the executives who were on it.
basicallybones
an hour ago
This is not totally accurate. For reference, here is the Wikipedia entry for Dodge v. Ford Motor Co. (1919) (copy and pasted at bottom). https://en.wikipedia.org/wiki/Dodge_v._Ford_Motor_Co.
In fact, the relatively new concept of a "public benefit corporation" is (at least in part) an effort to allow for-profit entities to pursue goals other than shareholder enrichment. However, some have criticized public benefit corporations as being entities that simply strengthen executive control at the expense of shareholders. https://en.wikipedia.org/wiki/Benefit_corporation
About Dodge v. Ford Motor Co.:
Dodge v. Ford Motor Co., 204 Mich 459; 170 NW 668 (1919),[1] is a case in which the Michigan Supreme Court held that Henry Ford had to operate the Ford Motor Company in the interests of its shareholders, rather than in a manner for the benefit of his employees or customers. It is often taught as affirming the principle of "shareholder primacy" in corporate America, although that teaching has received some criticism.[2][3] At the same time, the case affirmed the business judgment rule, leaving Ford an extremely wide latitude about how to run the company.[citation needed]
The general legal position today (except in Delaware, the jurisdiction where over half of all U.S. public companies are domiciled and where shareholder primacy is still upheld[4][5]) is that the business judgment that directors may exercise is expansive.[citation needed] Management decisions will not be challenged where one can point to any rational link to benefiting the corporation as a whole.
mossTechnician
3 hours ago
Shouldn't it be worrying that companies are required to make consistent gains* for shareholders and investors? At some point, a company will naturally reach a market saturation point.
* ETA: I meant "growth" here, not profit
lupire
3 hours ago
If it can't generate profit, it's worth more liquidated than operating.
Employees should buy out investors if they want to keep operating for their own personal profit.
xp84
2 hours ago
>If it can't generate profit
This wasn't exactly the question. The question was about growth. A company could be very profitable without growth (say, they own a mine which produces $40 million worth of ore each year with expenses of $10 million with no end in sight) or can have growth without profit (Open AI is a great example, or for history, the first 5 years of Facebook.)
I know most of stock investing is about capital gains and not dividends, but I think GP was saying it's inherently impossible to have growth forever.
On a financial level I get why people prefer to invest their money in a stock that goes up rather than one that pays them 8% a year consistently in dividends, but it seems unfortunate that somehow it seems like we aren't allowed to just have sustainable companies that don't depend on infinite growth to stay in business.
sph
3 hours ago
s/are/aren't/ required to make constant profit
adw
3 hours ago
It’s a company limited by guarantee, which is the structure you use in the UK for non-charity non-profits.
cinntaile
4 hours ago
If it started that way, I'd say it's less likely to end up "bad". Compared to non-profit websites that get sold to ad businesses.
Imustaskforhelp
5 hours ago
How is it making money?
cookmeplox
4 hours ago
We have services agreements with the League of Legends and RuneScape developers, and we run 1 ad (below-the-fold, not in EU/UK) on the RuneScape wikis. This covers all expenses (including 5 staff) by a pretty healthy margin
hiatus
3 hours ago
It is described in the linked article.
> The company primarily relies on three streams of revenue: user donations, serving ads on select Weird Gloop wikis, and a contract with Jagex that includes a fee to cover hosting and administration costs.
dingnuts
2 hours ago
I didn't see anything in the article about setting up incentives to keep the same thing from happening to Weird Gloop that happened to Fandom, which means the blog post is just empty marketing.
The only difference is that Weird Gloop is the little guy. Competition is good! That might be a good enough reason to choose them if you're in the market for wiki hosting!
But the moral posturing won't last if they become dominant, unless they set up incentives fundamentally differently than Fandom did, which doesn't seem to be the case.
As long as advertising is one of their revenue sources, the user experience will get crappy as soon as the network effects make it hard to leave. The cycle continues.
35skill
2 hours ago
Did you read the post? There's a whole section talking about how they are entering into binding agreements that let communities leave (and take the domain) if they have a better option
madeofpalk
5 hours ago
Can we flip it? Some companies are explicitly structured to guarantee enshittification.
Venture capital/private equity is what causes this. We've been poisoned to believe that websites should exist purely to achieve hyperscale and extract as much money as possible. When you look at the real physical world there are tons of small "mom and pop" businesses that are content with being self sustainable without some special corporate structure to legally require that.
Maybe websites could be the same?
SoftTalker
4 hours ago
There are millions of websites like that. They don't show up on the first page of search results, so nobody finds them.
zellyn
5 hours ago
The article explicitly covers this question. Looks like they're setting up explicit legal(?) agreements. One key point is the domain name: minecraft.wiki, for example, not a subdomain of something owned by Weird Gloop. So the wiki can leave if it wants to.
mossTechnician
4 hours ago
Does that mean that to the users of these wikis, the switching costs[1] of the backend would basically be zero (one day they might just end up on a different server with the same content), while on the administrators' side the switching costs are at a reasonable minimum?
[1] a variable in whether something can be enshittified, via https://en.wikipedia.org/wiki/Enshittification#History_and_d...
Nadya
4 hours ago
To my understanding wikis can take all their data, host it themselves, point the domain to their new hosting, and the move would be entirely invisible to end users if done properly and the quality of the hosting infrastructure wasn't considerably worse.
Observant users might notice the removal of any Weird Gloop branding but otherwise the only way people would know if the wiki itself announces the move or performance of the wiki becomes noticeably worse.
And Weird Gloop won't do what Fandom does and keep a zombie copy of your wiki online. So you won't be competing with Weird Gloop wiki traffic to reclaim your traffic. In fact, the obligations they agree to forbid it.
Reading the Minecraft.wiki Memorandum: https://meta.minecraft.wiki/w/Memorandum_of_Understanding_wi...
Upon termination by either party, Weird Gloop is obligated to:
- Cease operating any version of the Minecraft Wiki
- Transfer ownership of the minecraft.wiki domain to the community members
- Provide dumps of Minecraft Wiki databases and image repositories, and any of Weird Gloop's MediaWiki configuration that is specific to Minecraft Wiki
- Assist in transferring to the community members any domain-adjacent assets or accounts that cannot reasonably be acquired without Weird Gloop's cooperation
- This does not include any of Weird Gloop's core MediaWiki code, Cloudflare configuration, or accounts/relationships related to advertising or sponsorships
This sort of agreement means Weird Gloop is incentivized to not become so shit that wiki would want to leave (and take their ad revenue with them) because they've tried to make leaving Weird Gloop as easy as possible.
mossTechnician
2 hours ago
This is very reassuring. Usually, I assume agreements between different groups will inordinately benefit one party, but this particular agreement sounds like it creates a more level playing field.
And besides, it's not like non-profits are exempt from restructuring and becoming worse. There is no silver bullet.
cookmeplox
4 hours ago
Yeah - it would be on the same domain, so way users access it wouldn't change at all.
If any of the wikis we host want to leave, we'd provide them with a database dump. The admins would have to configure all of their own MediaWiki stuff of course, but I figure that's a pretty reasonable switching cost.
stonemetal12
6 hours ago
Thanks(seriously). Fandom may not be great, but you could have said I don't want to foot the bill, turned off the servers and walked away. Then the community would have lost every thing. Leaving it with Fandom gave Weird Gloop something to start with instead starting from scratch.
beAbU
4 hours ago
I can't imagine that this would have happened, like ever. The wiki was basically essential reading prior to starting to play Minecraft, especially in the early days. I think most the crafting recipes were documented by the developers themselves during those days.
If they killed the wiki, they would have killed their userbase.
Dwedit
5 hours ago
Ah Cloudflare, where you constantly get captchas for attempting to read a web page.
whstl
5 hours ago
At least they moved away from Google Captchas, which really hates disabling of 3rd party cookies and other privacy-protection measures.
I haven't had a problem with Cloudflare and their new Captcha system since their changed, but I still suffer whenever I see another website using Google Captcha :(
ChocolateGod
5 hours ago
Ironically its now easier for robots to solve Google Captchas than it is for humans, as evident by the browser extensions that solve them that exists.
matt_heimer
5 hours ago
That's up to the site owner.
For example I configured my osdev wiki (mediawiki based) so that the history and other special pages get the Cloudflare test but just viewing a page doesn't trigger it. OpenAI and other bots were generating way too much traffic to pages they don't need.
Blame the bots that are DDOS'ing sites for the captchas.
treefarmer
11 minutes ago
And god forbid you use a VPN and try to do anything on a Cloudflare site
kbolino
5 hours ago
Even better, you can get a captcha before you're allowed to see 404 Not Found.
theamk
5 hours ago
Cloudflare dropped captchas back in 2022 [0], now it's just a checkbox that you check and it lets you it (or does not).
And this mean that my ancient android tablets can no longer visit many cloudflare-enabled sites.. I have a very mixed feelings about this:
I hate that my tablets are no longer usable so I want less Cloudflare;
but also when I visit websites (on modern computers) which provide traditional captchas where you click on picture of hydrants, I hate this even more and think: move to Cloudflare already, so I can stop doing this nonsense!
eviks
4 hours ago
but there are more user-friendly captchas than the hydrants, which on average could be better that a total block on the tablets?
theamk
3 hours ago
total block on _old_ tablets - Android 4.4 specifically, and I am sure many people on HN would be horrified to see those anywhere close to internet. New tablets are fine.
As for "more user-friendly captchas" - I have seen some of those (like AliExpress' slider) but I doubt they will work as well as hydrants. And with new AI startups (1) slurping all the data on the web and (2) writing realistic-looking spam messages, I am sure anti-bot measures would be more important than ever.
fwip
4 hours ago
The checkboxes are also captchas.
preciousoo
6 hours ago
You and your team made(a good portion of) my childhood. I remember spending nights studying all the potion recipes and enchantment odds. Thanks for all you did
why_at
3 hours ago
One thing I find interesting about playing video games in modern day is that with the proliferation of Wikis, there is assumed to be some kind of third party guide for every game. Especially in smaller/newer games it seems like developers sometimes don't bother putting necessary information in the game at all because they don't have the person-hours for it.
For instance, back when I first played Minecraft in Alpha the only ways to find the crafting recipes was through a wiki, or trial and error.
It's nice that it makes development easier, but I wonder if this trend is making it harder for new people to get into video games, since it's hardly obvious if you're not used to it.
christianqchung
3 hours ago
I don't really know how exploratory most games are compared to old Minecraft. Some games like Stardew Valley have certain things that are much easier to do because of third party wikis but I don't think the same is true of a lot of games in the same way it was for Minecraft.
hinkley
an hour ago
One of the things on my todo list is to spend some solid time thinking about load-shedding, and in particular tools and methods for small or hobbyist projects to practice it. Like what do you turn off on the site when it's the 15th of the month and you're already at 80% of your SaaS budget?
Like maybe if a request for an image doesn't result in a 304, instead of sending a 200 response you redirect to lower res versions, or just 429 out. How much throttling do you do? And do you let bots still run full speed for SEO reasons or do you do something else there?
oreally
6 hours ago
> with the availability of Cloudflare, running high-traffic websites is much more cost effective.
sidetrack but how does cloudflare make things cost effective? wouldn't it be cheaper if i just hosted the wiki on a simple vps?
pjc50
6 hours ago
Cloudflare get the best deals on bandwidth. It will usually be cheaper to serve a terabyte from Cloudflare than to do it yourself: you could probably run the wiki on the free plan!
account42
5 hours ago
Perhaps, but VPS traffic prices are also already a lot better than "big cloud" traffic prices, especially if you choose your VPS provider with that in mind. And once your traffic is large enough there are also options where you pay for a fixed pipe instead of a transfer amount.
diggan
5 hours ago
> Cloudflare get the best deals on bandwidth.
If you want to pay for bandwidth then yeah, CloudFlare is a great option.
Otherwise, if you like the experience of not paying per GB/TB, go for a dedicated server with unmetered connection that has the same price every month, regardless.
KomoD
5 hours ago
You don't need to pay anything to run TBs through Cloudflare, you could use the free plan.
Rent VPS or managed hosting or host wherever you want, proxy it with Cloudflare on the free plan, Cloudflare caches it.
robertlagrant
4 hours ago
It's more like: if you have a website that (sometimes) gets a lot of traffic, do you want Cloudflare to cache it and serve it with very few hits to your cheap server, or do you want your compute costs to expand to cope with the requests?
rjmunro
3 hours ago
Cloudflare don't charge per GB/TB. You get unlimited bandwidth even on their free plan. The problem with paying per GB is that it's in the CDN's interest for you to get a DDOS attack so they can charge you for all the bandwidth. It's in Cloudflare's interest to reduce DDOS attacks and unwanted bot traffic because it costs them bandwidth, not you.
wpietri
3 hours ago
Your point on interest is spot on.
I moved a few of my personal websites to AWS's CloudFront and it cost me like a buck a month, way cheaper than maintaining a virtual server to do it. Except that somebody somewhere decided to try their DDOS tool on one of them for a few hours in the middle of the night, and I got a bill for $2541.69.
Eventually they credited it, but it was not a fun ride, and decided that I was done using a CDN with misaligned incentives: https://sfba.social/@williampietri/111687143220465824
Aachen
2 hours ago
> it's in the CDN's interest for you to get a DDOS
What kind of conspiracy is this? As if anyone charging for bandwidth hopes to get their infrastructure attacked
0cf8612b2e1e
43 minutes ago
Why not? They have the capacity they could absorb nearly any kind of attack without blinking.
citricsquid
6 hours ago
More than a decade has passed since then so I am stretching my memory. At peak we were serving in the region of 10 million page views per day which made us one of the most popular websites on the internet (Minecraft was a phenomenon and every Minecraft player needed the wiki). We were probably the highest traffic Wiki after Wikipedia. Nowadays Cloudflare could absorb most traffic because of the highly cacheable nature of it, but at the time, Cloudflare didn't exist, and every request hit our servers.
owyn
5 hours ago
Yeah, Wikia in aggregate was in the top 50, maybe a top 20 site at various points. Wikia was built on caching. From my memory, about 99% of page views hit some kind of cache. If that dropped down to 97%, servers started to suffer. It's good to remember that the Fastly CDN company is a spinoff of Wikia, it was developed internally there first. Without that (varnish cache plus lots of memcache) Wikia would not have been able to handle the traffic. Mediawiki is horribly inefficient and one reason why Wikia was attractive as a host was that we had figured out a bunch of tricks to run it efficiently. The default configuration of mediawiki/wikipedia is real bad. Bigger independent wikis just couldn't handle the scale and many of the best independent wikis moved there for that reason. Just as one example, every link/url on a page hits a hook/callback that can call into an extension literally anywhere in the code base, which was several million lines of PHP code. I remember the "Batman" page on the DC wiki used to take several minutes to render a new copy if it fell out of the cache. That was one page I used for performance optimization tests. The muppet wiki and the lyrics wiki also had huge performance issues and fixing them was some of the most fun engineering work I've done. Every useful feature had some kind of horrible performance side effect, so it was always a fun puzzle. I also hate landing on a Fandom wiki now but thanks to the actual editors, it's still got some good content.
immibis
6 minutes ago
How much of your server load was Grand Exchange Market Watch?
Ambroos
6 hours ago
If you can run your application on Cloudflare Pages / Workers with Cloudflare's storage/DB things, it really gets dirt cheap (if not free) and very fast. And even without that, Cloudflare's caching CDN is very good, very cheap and very easy.
bombcar
6 hours ago
Ten years ago bandwidth was expensive. Still is, even if not as much. A simple VPS gets overwhelmed, but a simple VPS behind cloudflare can do quite well.
thinkmassive
5 hours ago
s/cloudflare/a CDN/
pornel
5 hours ago
Cloudflare caches pages at many many datacenters, often colocated with large ISPs.
This lets Cloudflare deliver pages from their local cache over local links (which is fast and cheap), instead of fetching the data every time across the world from wherever the VPS is located.
Washuu
2 hours ago
Hey Criticsquid!~ \( ̄︶ ̄*\)) It's Azxiana[1].
I hate that MCW ultimately ended up with Fandom in the end. Keeping MCW and the other wikis running smoothly was essentially my one huge passion in my life that I lost after Fandom acquired Curse. No one wanted it to happen that way. Even internally at Curse/Gamepedia we were all devastated when we learned that the company was buying bought out by the rival we were striving to overcome all those years. I am so glad to see after the past few years that the wikis are finally healing and going to places that are better for them.
[1] I'm the tech lead/manager that worked on Gamepedia at Curse that administered Minecraft MCW for many years before Fandom bought Curse in December 2018. I'm just writing this here since I figure other readers won't have any idea. ヾ(≧▽≦*)o
jchw
6 hours ago
In all fairness, running modest to large MediaWiki instances isn't easy. There's a lot of things that are not immediately obvious:
- For anything complex/large enough you have to set `$wgMiserMode` otherwise operations will just get way too long and start timing out.
- You have to set `$wgJobRunRate` to 0 or a bunch of requests will just start stalling when they get assigned to calculate an expensive task that takes a lot of memory. Then you need to set up a separate job runner in the background, which can consume a decent amount of memory itself. There is nowadays a Redis-based job queue, but there doesn't seem to be a whole lot of documentation.
- Speaking of Redis, it seems like setting up Redis/Memcached is a pretty good idea too, for caching purposes; this especially helps for really complicated pages.
Even to this day running a Wiki with an ambient RPS is kind of hard. I actually like MediaWiki because it's very practical and extensible, but on the other hand I know in my heart that it is a messy piece of software that certainly could make better use of the machine it's running on.
The cost of running a wiki has gone down over time in my experience though, especially if you are running things as slim as possible. A modest Digital Ocean machine can handle a fair bit of traffic, and if you wanted to scale up you'd get quite a boost by going to one of the lower end dedicated boxes like one of the OVHcloud Rise SKUs.
If anyone is trying to do this I have a Digital Ocean pro-tip. Don't use the Premium Intel boxes. The Premium AMD boxes are significantly faster for the money.
One trap I also fell into was I thought it might be a good idea to throw this on a hyperscaler, you know, Google Cloud or something. While it does simplify operations, that'll definitely get you right into the "thousands of dollars per month" territory without even having that much traffic...
At one point in history I actually felt like Wikia/Fandom was a good offering, because they could handle all of this for you. It didn't start out as a bad deal...
noen
4 hours ago
This is so true.
I adopted mediawiki to run a knowledge base for my organization at Microsoft ( https://microsoft.github.io/code-with-engineering-playbook/I... ).
As I was exploring self-host options that would scale to our org size, it turned out there was already an internal team running a company wide multi-tenant mediawiki PLATFORM.
So I hit them up and a week later we had a custom instance and were off to the races.
Almost all the work that team did was making mediawiki hyper efficient with caching and cache gen, along with a lot of plumbing to have shared infra (AD auth, semitrusted code repos, etc) thst still allowed all of us “customers” to implement whatever whacky extensions and templates we needed.
I still hope that one day Microsoft will acknowledge that they use Mediawiki internally (and to great effect) and open-source the whole stack, or at least offer it as a hosted platform.
I tried setting up a production instance af my next employer - and we ended up using confluence , it was like going back to the dark ages. But I couldn’t make any reasonable financial argument against it - it would have taken a a huge lift to get a vanilla MW instance integrated into the enterprise IT environment.
bawolff
2 hours ago
Microsoft did open source a bunch of their mediawiki extensions. https://github.com/microsoft/mediawiki-extensions
Last i heard though they were moving off it.
account42
5 hours ago
A lot of things should be solved by having (micro)caching in front of your wiki. Almost all non-logged in requests shouldn't even be hitting PHP at all.
jchw
4 hours ago
In my experience this hasn't been necessary yet on anything I've ran. I know WMF wikis run Varnish or something, but personally I'm trying to keep costs and complexity minimal. To that end, more caching isn't always desirable, because RAM is especially premium on low-end boxen. When tuned well, read-only requests on MediaWiki are not a huge problem. The real issue is actually just keeping the FPM worker pool from getting starved, but when it is starved, it's not because of read-only requests, but usually because of database contention preventing requests from finishing. (And to that end, enabling application-level caching usually will help a lot here, since it can save having to hit the DB at all.) PHP itself is plenty fast enough to serve a decent number of requests per second on a low end box. I won't put a number on it since it is obviously significantly workload-dependent but it would suffice to say that my concerns with optimizing PHP software usually tilt towards memory usage and database performance rather than the actual speed of PHP. (Which, in my experience, has also improved quite a lot just by virtue of PHP itself improving. I think the JIT work has great potential to push it further, too.)
The calculus on this probably changes dramatically as the RPS scales up, though. Not doing work will always be better than doing work in the long run. It's just that it's a memory/time trade-off and I wouldn't take it for granted that it always gives you the most cost-effective end result.
bawolff
2 hours ago
Varnish caching really only helps if the majority of your traffic is logged out requests. Its the sort of thing that is really useful at a high scale but matters much less at a low scale.
Application level caching (memcached/redis/apcu) is super important even at a small scale.
Most of the time (unless complex extensions are involved or your wiki pages are very simple) mediawiki should be io-bound on converting wikitext -> html (which is why caching that process is important). Normally if db is healthy, db requests shouldn't be the bottle neck (unless you have extensions like smw or cargo installed)
jchw
an hour ago
Most of MediaWiki seems to avoid too much trouble with contention in the database, but I was seeing it prior to enabling application-level caching. It seemed to be a combination of factors primarily driven by expensive tasks in the background. Particularly complex pages can cause some of those background tasks to become rather explosive.
tempest_
4 hours ago
Have any of Intels server offerings been "premium" since epyc hit the scene?
I just assumed they were still there based on momentum.
jchw
4 hours ago
With Digital Ocean the cpuinfo is obfuscated so figuring out exactly what you're running on requires a bit more trickery. With that said I honestly assume that the comparison is somewhat older AMD against even older Intel, so it's probably not a great representation of how the battlefield has evolved.
That said, Digital Ocean is doing their customers a disservice by making the Premium Intel and Premium AMD SKUs look similar. They are not similar. The performance gap is absolutely massive.
Nux
2 hours ago
> At the time of selling out, the Minecraft Wiki and Minecraft Forum cost tens of thousands of dollars per month to run.
What kind of decisions got you in that position? Hard to phatom.
Arch-TK
5 hours ago
You say you were a kid when you sold it. I could have sworn you weren't from conversations we had on IRC at the time.
Although I most assuredly was a kid.
citricsquid
4 hours ago
I was a teenager at the time. I'm in my mid 30s now, it feels like I was a kid back then.
fwip
4 hours ago
"Kid" doesn't really have a hard cutoff. When you're 15, 12-year-olds are kids. When you're 30, 20-year-olds are kids.
jagermo
6 hours ago
holy crap that minecraft wiki is fast now. I actually stopped going to fandom because it was so slow.
sammy2255
5 hours ago
Tens of thousands to run a static webpage? LUL
misode
5 hours ago
The Mediawiki software is not a static webpage
tredre3
4 hours ago
Mediawiki is trivial to cache, though. For all intent and purposes most hits will be cache hits, and thus "static" content.
I'm also shocked at the tens of thousands per month, it can't possibly be hosting alone. It has to be that the maintainer had a generous salary or something.
citricsquid
4 hours ago
I could have the numbers wrong, archive.org is down otherwise I would check as we shared information publicly at the time. As far as I recall, we weren't taking money from the websites, we were spending on infrastructure alone with more than $10k in spend in the final month before the sites were acquired. I think it is easy to forget how much more expensive running things on the internet was back then along with the unprecedented popularity of Minecraft. Once archive.org is back online, I'll track down numbers.
bawolff
2 hours ago
Not everyone is a professional web hoster with requisite knowledge on how to setup caching properly.
Mediawiki involves edits that users expect to propagate instantly to other pages. Sometimes this can easilt result in cache stampedes if not setup carefully.
MediaWiki supports extensions. Some of the less well architectured extensions add dynamic content that totally destroies cachability.
nemothekid
4 hours ago
Seriously? How does that even make sense to you? The OP had an asset generation 10k+ a month in profit and was so squeezed for cash he had to sell it.
Doesn’t it make more sense that a media have site would have been paying through the nose for bandwidth, hence the callout for cloudflare which would have made that cost free?
Sharlin
4 hours ago
I have no idea how it works, but given that the read:write ratio is probably 100:1 or more, certainly it could just serve static, prerendered pages straight from the filesystem or something like memcached?
bawolff
2 hours ago
[Im a mediawiki dev]. Typically people use varnish for that use case. MediaWiki does support serving logged out views from a filesystem cache, but varnish is generally a better idea. There are also some caches out of memcached (mediawiki has "parser cache" in memcached which is the part of the page that stays constant between all users. Typically people use varnish on top of that for the entire page for logged out users)
Sometimes people add things to their sites that are incompatible with caching, which will make hosting costs go way up.