If an air purifier review site is publishing research on their search traffic and affiliate site rings, I feel it's safe to say that their specialty is in seo, not air purifiers.

“Do all evil” its new unofficial slogan.

I think that'd be a good approach. There was an idea in that time that everything had to be algorithmic, that hand-ranking or singling out individual sites was a bad idea. Ie: tweak a generic algorithm and then test what it does to top garbage sites, don't just penalize the site directly. I think that's not a bad principle but in practice it didn't seem to work well.

[flagged]

If so, the author deserves a Web Signal Award for succeeding where customers, regulators and execs failed despite years of trying.

You're totally right about that symbiotic relationship. We were aware of that risk in the early days when AdSense launched, we saw some very innovative and gross exploitation and created some policies to rein it in. But ultimately if Google makes a buck coming and going, they will do that.

Wasn't there a big story last year in the wake of the DOJ antitrust investigation about Google manipulating search quality to boost ad revenue? I can't put my hands on a reference now, in part because Google is so bad at search these days I can't find anything more than a few months old.

> Google wins

Define "wins". From what happening right now, it seems that google may lose much more than it earned by aligning with seo spammers

Maybe they need to start locking employee stock options for 100 years to prevent them optimizing short-term gains?

Because ChatGPT is dependent on good search when it searches the web? Or because it completes with Google when it provides a good answer without searching? Or what do you mean specifically?

> My best guess is it's because they finally have a real competitor in ChatGPT.

ChatGPT doesn't even fulfill the same function, to say nothing about the poor reliability inherent to the way it works. In no sense is it a real competitor to Google.

> My best guess is it's because they finally have a real competitor in ChatGPT.

My guess is it's because a bunch of articles about it have been posted to hn recently.

> My best guess is it's because they finally have a real competitor in ChatGPT.

Bingo. I always chuckle when people here say Google has lost it, and become incompetent. Well, they all make the mistake of assuming that they’re trying but failing, rather than that it’s deliberate simply due to boring economics.

Now look at how quickly decades-long problems, so big they have an entire cottage industry built around it, suddenly be cleaned up. Incompetence? Nah.

Of course, this does nothing to convince regulators and not even average HN user that innovation is harmed by these dominant players. Someone’s gotta think of the poor mega-corps.

You're mostly describing Kagi. They do have AI results but you have to explicitly ask for them. They have an "No AI" image search option as well.

I also like my "Before AI" lens I can click on to search the internet pre-2021. And you can downrank or fully block those garbage spam sites. They even have a "leaderboard" for most blocked/pinned sites you can use to get started.

I think it would be easy to make, with two decades of hardware improvements.

The problem is that the web of 2003 doesn't exist any longer.

Google existence changed the websites for better or worse. The Google of 2003 is no longer capable of dealing with today's web SEO dirty tricks.

Install a spam filter for search engines, like uBlackList.

Use bunch of different search engines. In Firefox, enable search entry, then visit search engines and click green + in the entry, to add search engine.

Kagi and Phind are both impressing me. They both use AI but in ways I like.

It would take a benefactor who wants to pay for running it for its own sake and not for profit. As soon as there's a profit motive, enshittification sets in since you're serving whoever pays rather than your users.

Just traffic.

Seems like post hoc fallacy.

But people were complaining about the sAme issues under Matt Cutts. Also, there has been A Ton more money and work chasing the SEO farm game. Now big private equity companies have focused on buying a stable of big brands to do the same that used to be garage startups.

I think it's nick fox now and he's old school and as competent as they come

Maybe the professors knew and were taking advantage to look the other way to make their job easier.

> If it didn’t, this scam wouldn’t work.

It would, as this scam relies on the domain authority, regardless of their "offline authority".

Search has degraded substantially in the past 5 years for reasons wholly, completely, and absolutely unrelated to SEO blogspam.

Google of 5 years ago didn't ignore words in almost every single query I made. Google of 5 years ago didn't constantly give me irrelevant garbage because they keep ignoring the words I use in my query.

This is a wholly separate issue from SEO crap. Ignoring search terms is 100% a Google issue and is 100% Google's fault!

Google search degraded in usefulness before the panda update, when spammers had filled the web with low quality content designed to exploit Google's algorithms. Google improved their search to punish the content farms, and people were happy with that search for many years.

Agreed. That’s really my thoughts.

The internet or rather the crawlable WWW itself has changed.

The WWW content itself is no longer what it was.

What I really want is me from 5-10 years ago. When can we get a pill that will do that?

I'm fine with this internet but the previous search.

A close approximation might be a search-mode which penalizes results based on how many ads they have and how much of the page they cover...

It did degrade a bit. But Kagi and Perplexity proved (in very different ways) that you can get significantly better search from the same internet that Google uses.

Exactly. In a constantly changing world, you need constantly changing policy to achieve the same outcomes. Even then you probably won't replicate the past universe perfectly.

append ' before:2023' to your google searches.

I want the Internet from 5-10 years ago before Google incentivized this much SEO garbage. It wasn't awesome then but it was a lot better.

> You don't notice that it isn't giving you the best result

That's fine. It's always been fine. I don't need Google to read my mind and fulfill my dreams.

The problem isn't that they're not divinely perfect. The problem is that they used to be good enough, and now they're not.

> There are more intelligent, more accurate, more safe, ways to solve the problems people have, that are not "a search engine". It's time we start implementing them.

What solutions are there that fulfill all the use cases of a search engine, while definitively not being a search engine? An AI chatbot that gives me synopses of the same websites that I was searching for does not count.

> You can't go back to the way things were.

> type in anything, and somehow it will read your mind

I think we can go back to the way things were, which had nothing to do with mind reading. In the past, you could type in word, and google would offer 10 million results, and you could page through each of them. That was very powerful, and google does not do that today.

>Think about the premise: type in anything, and somehow it will read your mind, intuit who you are and what you really wanted, find the exact thing amid the morass of the whole web, and then give it to you?

I never once asked for anything remotely like this. Maybe you could just show me results for the fucking thing I typed? When I go to the library, the Dewey decimal system doesn't rearrange itself based on all the metadata the library has on me and people fitting my demographic criteria, it just shows me what I fucking searched for.

Web search has never been about this, where are you getting this fable from? It was about finding pages that include certain words, why would you overmistify this.

Not sure why you’re being downvoted, this is a pointed analysis of why crawl-based search is insufficient for an Internet of our current scale. There is no corporate-curated algorithm that is up to the task, especially when the primary purpose is to profit from advertising.

In before your comment ends up in QuickBooks search results for the rest of time :-)

Google does perfectly on the latter search. It returns a relevant blog post written by an actual human, and a bunch of forum threads about that exact upgrade path.

DDG has never been very good.

Were those organic results, or the paid ads that always appear first?

I tried Kagi but just didn't see notably better results than other search engines. Maybe if I spent more time on the power user tools, or if Kagi offered more of a trial period I would have, but adding yet another monthly subscription is a high bar for me and what I saw didn't clear it.

These days my default assumption is that any SAAS product will get worse and more expensive over time, so it has to be pretty good to justify reworking my online habits around, given that I don't know how long I'll keep using it. Hopefully Kagi will be the exception to that rule, but I wouldn't bet on it.

That subscription fee is just too big of an obstacle in a time when everything has a subscription and is still often degrading in quality. Seems like an unsolvable chicken and egg scenario though, since relying on advertising to make it free would just result in the same issues as everything else.

This is how I do it

1. Make HTTP

2. Send HTTP via TCP

3. Perform text processing on the response body (I create own SERP instead of using Google's)

Personally, I use multiple programs, some I wrote myself in C, to perform these individual steps, connected by UNIX pipes and the shortest, simplest possible Bourne shell scripting

However there are countless ways to perform these steps in wide variety of programming languages; there is no need for UNIX or shell scripting, it is purely personal preference

How do you do that?

As others noted, this has been an issue for years. What prompted Google to act now, using a manual override that was supposedly not feasible in the era of algorithms? Did the viral articles by Lars provide ammunition for teams within Google?

Are locals particularly attached to that name?

Seems like a referendum for a name change would be a good idea, considering the current typical usage of that term.

I've never noticed that behavior from Google when searching for any other topic. Have you?

What do you mean by the opposite?

The search behavior is the same whether $foo is a popular generic term or something niche.

Kagi does let the user adjust the rankings of these sites if they don't want them coming up. While it would be nice to have this done proactively for the link farms, at least the user does have some control.

Forbes did make it on the blocked and lowered leaderboards.

https://kagi.com/stats?stat=leaderboard

DDG gives me:

forbes.com healthline.com medicalnewstoday.com fortune.com

But I usually use site:reddit.com in my searches anyway so it doesn't matter to my personal habits.

And Forbes the SEO giant was going to buy out the legacy media property.

What's the problem with stock buybacks?

The company is giving money back to shareholders. What exactly is wrong with that?

Is healthline spam? It's my go to whenever I have a medical question to add healthline to the search

Good point. I just asked ChatGPT for the best CBD gummies, then asked what sources it used for the list. This was the first thing it said…

> Consumer reviews from trusted websites like Healthline, CNET, and Forbes Health that provide in-depth reviews and rankings based on effectiveness, ingredients, and customer feedback.

So the LLMs are now giving us affiliate link garbage, but we can’t easily see that was the source, and the affiliate links don’t even work. Everyone loses in this scenario.

That blogger talks about his speculations which I agree with.

Google has been fighting against SEO spammers since the very beginning, in fact there was keyword stuffing even before there was Google.

Google at some point made a decision to privilege sites like Forbes because they were dealing with hundreds of subprime sites that looked even sketchier. Rather than playing whac-a-mole with new spammers every month as they change the rules they could put sites like Forbes and Wirecutter on top, ultimately discouraging other SEO spammers from even trying, and being able to manage the relationship. (e.g. if you see Forbes rank for "best CBD gummies" just once it looks a little sketchy but you get over it, you only really catch the bad smell after you've done 5,000 searches and see Forbes ranks so high on so many of them)

I've been working with a financial analyst to quantify the value of online engagement (what value does YC get from an HN comment? what value do users get from it?) and reviews are one of the clearest examples since a review adds or subtracts revenue from a vendor and gives customers a certain amount of satisfaction. It's an interesting area in my mind because the stakes are so high.

There are many criteria for "best" that are acceptable to many people, e.g. lowest price, proven high quality ingredients, efficacy, etc. "Website with high reputation that happens to be running ads for company XYZ" is usually not how people define "best".

And the notion of "the best" is inherently subjective

For the sake of this exercise I want to hear how people would solve that harder problem.

Hm, I think that Amazon shows that just user votes might not be sufficient - e.g. because users can be paid off to give 5 star reviews, which bias the results.

should have taken a screenshot.