Show HN: A tool to analyze Hacker News sentiment on any term in seconds

76 pointsposted 7 hours ago
by lorddustingale

52 Comments

wongarsu

5 hours ago

If anything, this tool tracks with my general opinion on sentiment analysis: it would be awesome if it actually worked, but most algorithms just predict everything as neutral.

For example if you search for bitwarden it ranks three comments as negative, all others as neutral. If I as a human look at actual comments about bitwarden [1] there are lots of comments about people using it and recommending it. As a human I would rate the sentiment as very positive, with some "negative" comments in between (that are really about specific situations where it's the wrong tool).

I've had some success using LLMs for sentiment analysis. An LLM can understand context and determine that in the given context "Bitwarden is the answer" is a glowing recommendation, not a neutral statement. But doing sentiment analysis that way eats a lot of resources, so I can't fault this tool for going with the more established approach that is incapable of making that leap.

1: https://hn.algolia.com/?dateRange=pastMonth&page=0&prefix=tr...

team-o

2 hours ago

I haven't looked in the specific classifications of this particular model, but what your comment shows is the importance (IMO) of having a "no sentiment" class when classifying sentiment. E.g. if someone says "John doe is an average guy", the sentiment to John is neutral. But if someone says "John doe is my uncle" there's no sentiment and it should be classified as that. Perhaps the classifier here already takes this into account, but just thought it was worth mentioning the importance of having this extra class, or a separate pre-filter classifier. In your example I also see many that could be filtered out. E.g. "I store them in Bitwarden not in dotfiles" doesn't contain negative/neutral/positive sentiment, or at least you're not able to tell from just this sentence. I appreciate it's a fine line between neutral and no sentiment though.

Mockapapella

4 hours ago

I think this is fair criticism for where it's at and mirrors my experience while building the tool. For generative AI at least, the smartest models + a good prompt will waffle stomp our tool in terms of quality.

For example, while testing it on "Founder Mode" there were a couple comments that mentioned something like "I hate founder mode but I really really like this other thing that is the opposite of founder mode..." and then just continues for a couple paragraphs. It classified the comment as positive. While _technically_ true, that wasn't quite the intention.

We think there are some ways around this that can increase the fidelity of these models that won't involve using generative AI. Like you said, doing it that way eats a ton of resources.

wongarsu

an hour ago

Just spitballing, but maybe a good tradeoff is to use NLP to find good candidate comments that are likely to contain a sentiment, and then analysing a small number of them with a more expensive model (say a quantized 3B or 7B LLM with a good prompt). The quality over quantity approach

zzleeper

4 hours ago

BTW, which algo did you use to classify sentiment? bert or something related?

bhaney

34 minutes ago

I like the concept and the interface, but after trying one of the suggested examples, I'm not convinced this actually works?

I tried "remote work" like the initial instructions recommended as an example. The graph it gave me showed large spikes of "neutral" sentiment with a few negligible bouts of negative sentiment and even smaller bouts of positive sentiment. The sample comment it gave was from a "Who Wants to be Hired" post where the poster demanded exclusively remote offers, which the tool classified as "neutral" (with 98.7% confidence!)

Very slick tool, but if the sentiment analysis itself doesn't really work well then I don't see what value this could have.

codetrotter

6 hours ago

I think this is actually one of the very first times I have seen neumorphic design in the wild.

Prior to this I’ve mostly only seen it on dribbble.

I actually like this style a lot, and I wish more apps would use it. But at this point I thought that this style was one that “came and went” before it saw any significant actual use in any apps or OSes. Maybe there is still hope after all :)

Edit: oh and I had to try asking your tool for sentiment about neumorphic design after this of course. It returned my own comment lol :p and it called it “neutral”. Is it only evaluating the first paragraph that the word appears in in the comment? (Also I guess other people more commonly refer to it as “neumorphism” than as “neumorphic design” and maybe that’s why when I asked it for neumorphic design it returned my own comment.)

wongarsu

5 hours ago

To me, neumorphic design looks like skeumorphism for those cheap interfaces made of one continuous sheet of thin printed (sometimes vacuformed) plastic with the actual buttons hidden underneath. Stuff like [1] or [2]. And while I love skeumorphism, I hate those kinds of plastic interfaces. They always use the cheapest mushy buttons, and the pressable area of the buttons is always much smaller than what the printed buttons suggest. It's probably the one kind of physical interface I hate with a passion, and imitating it digitally doesn't evoke any positive feelings in me.

Still better than making everything flat without shadows and making me guess where I can click, I guess.

1: https://www.alamy.com/stock-photo-finger-pressing-the-button...

2: https://www.alamy.com/close-up-of-clothes-washing-machine-bu...

Edit: just checked, this comment was analyzed as "Sentiment: neutral (Confidence: 79.56%)" on the topic of "neumorphic"

Mockapapella

6 hours ago

(the other part of the team that built this here)

Honestly makes me pretty happy you called out the theme. I've always enjoyed this style of design and was sad to see that it never picked up steam. I love how it seems to combine a digital Material design with a more physical and real feeling. I'm doing my part to bring it back.

The tool definitely has some kinks in it that we have plans to iron out over time; we just wanted to get it in front of people to see if anybody would even like it. Right now it's just grabbing the first 256 tokens and categorizing on that, and it grabs the first 5000 comments (split over 5 calls) over the past month.

WantonQuantum

5 hours ago

An aspect of the UI I really like is the way previous results stay available while doing new ones.

chucksmash

5 hours ago

Tokens are characters in this context?

I tried "neumorphic design" based on the comment this replies to. It is classified as neutral.

Mockapapella

4 hours ago

Tokens are approximately 3/4ths of a word. That might sound odd if you're unfamiliar, but in short it has to do with how LLMs break down words to process them. Letters are too granular for sufficient quality, full words don't allow LLMs to produce novel text like "dkjmcf0248375cyu18c7437tr18c237m yt034c8nwey4mtc0r23q9p,a;" (should it need to), so the industry has settled on the middle ground of a part of a word.

ranger_danger

4 hours ago

It's uncommon for several reasons: it's not very accessible, its predecessor skeuomorphism (like Win95 style interfaces) was overdone for decades, and it ignores all the reasons we transitioned to flat design in the first place.

https://www.nngroup.com/articles/skeuomorphism/

https://www.nngroup.com/articles/flat-design/

>Neumorphism never quite made it mainstream because it comes with its own set of problems. The low contrast does not offer sufficient visual weight, making the experience not accessible. Additionally, it is difficult to determine clickability, as neumorphism is often used inconsistently on nonclickable and clickable elements.

Don't get me wrong, I still like the design and I think it's cool, but I understand the reasons why it never got popular.

Mockapapella

4 hours ago

Yeah I've heard about the accessibility argument. I tried to mitigate that somewhat with the button designs having a black/while outline (depending on the theme), but I may need/want to add a "high contrast" mode later on.

Unearned5161

5 hours ago

yeah, looks epic. Makes me think of Superman's spaceship in Man Of Steel , mega clean

Pannoniae

5 hours ago

(tangential) I wish actual skeuomorphism would also come back....Even in things you can theme, it's just choosing between different flat looks.

solardev

3 hours ago

Hmm, it thinks HN is neutral on crypto. Hmmmm.

Is it actually doing anything?

__MatrixMan__

an hour ago

Apparently we prefer death over Oracle. But on a second search Oracle is then better than death...

No, I don't think it is.

brunocvcunha

an hour ago

Cool idea. But as a red/green color blind person, I have no way to figure out what is positive/negative in the graphs.

avodonosov

4 hours ago

Idea: single number metric, something like, %-of-positive - %-of-negative avearaged over a time period.

So that we could compare terms based on this result metric: google vs microsoft, rust vs go, rust vs microsoft, etc

(Will not work for Go as it's a common word in addition to the programming language, but anyways)

Mockapapella

2 hours ago

I like it. We'll put it on the roadmap.

IgorPartola

6 hours ago

It would be really interesting to see, in the limit of this tech, how any given comment ranks on the group-think to contrarian scale for every comment you read. There are a lot of signals on HN about comment quality, like karma, account age, reputation of certain individual names, but this could be an interesting one. Wonder what kind of emergent behavior it would drive.

caseyy

2 hours ago

NFTs — neutral with 98% confidence. Hmm…

Also, it seems like putting in the same phrase twice generates different graphs and results at least sometimes. So it’s difficult to use comparatively.

nomilk

4 hours ago

It’s great. Simple and works in 10 seconds. Only thing is I’d prefer to know sentiment over at least a year (defaults to just 1 month)

Still, great idea and execution

Mockapapella

4 hours ago

Thank you! We plan on letting users select their own timeline later -- this was more of a defense for us to make sure we didn't get rate limited and everyone could try it out

Nuzzerino

4 hours ago

Seems pointless to go through all that effort to chart this over time, and have that time limited to..... a single month.

> you can track trends with it

No, no, you can't.

stevage

5 hours ago

A month doesn't seem like a useful time frame? Or maybe I'm misunderstanding the main use case.

Also, you could drastically improve styling on mobile. Lot of wasted space.

Mockapapella

4 hours ago

Main case right now is to see if people like it (and so that we don't hit Algolia rate limits). You're right, a more useful case might be letting the user customize the time frame.

As for mobile, I am but a humble backend dev, but I agree completely. Will put it on the roadmap. Thank you for your feedback!

grogenaut

3 hours ago

it'd be great if I could turn off neutral, it's drowning out the positive and negative lines... for example Kubernetes... mostly neutral.

MeetingsBrowser

5 hours ago

On mobile when I select the text input the whole page zooms so far in that I can’t see the submit button (or anything else for that matter).

Took me a minute to figure out what had happened, but I was able to submit a phrase. The response was

> Randomly sampled comment from the pulled data: Sorry, there was an error processing your message.

op00to

5 hours ago

The sentiment of “fart” is neutral.

analog31

5 hours ago

Out of curiosity, is the sentiment index weighted for up-voted threads and comments? Because those things are a kind of sentiment index too.

bagels

4 hours ago

Are there any terms that don't say "neutral" as the sentiment?

fortran77

13 minutes ago

I tried “your mother” and it went into the positive area twice, but mostly neutral.

max-m

an hour ago

"dang" is liked by HN! :D

poniko

5 hours ago

Nice idea, not sure it works though.. got neutral on all words I tried .. even beer

ranger_danger

3 hours ago

My biggest issue when I tried to analyze sentiments was that there is no practical way to bias for sarcasm, such as "it's great if you love pulling your hair out".

eth0up

6 hours ago

Exit vim - Sentiment: neutral (Confidence: 98.32%)

Stockholm Syndrome?

WheatMillington

an hour ago

RTO and Elon Musk are both overwhelmingly neutral sentiment? Seems unlikely.

aliasxneo

6 hours ago

Appears to be struggling from the hug of death.

Mockapapella

6 hours ago

We're actively taking suggestions from people to improve it. The webserver was probably just restarting with the new suggestions :)

aliasxneo

6 hours ago

How often is it doing that? I've tried a few times over the last ten minutes. The latest attempt ended with Cloudflare returning a 504.

Mockapapella

5 hours ago

If that's the case maybe I'm seeing a caching thing on my side. I upsized the droplet some

wavemode

4 hours ago

I searched Blockchain and it said "neutral". This is the sample comment it displayed:

> Ah yes, because blockchain is the 100% true source of ultimate truth.

The model can't detect sarcasm.

gotoeleven

5 hours ago

>> white board interviews

nothing

>>hobos in san francisco

nothing

>>accommodating my neurodivergence

error

>>is everyone I don't like hitler or just some people

nothing

breck

6 hours ago

Love the fact I didn't need to signup to get value out of this!

Note: I would suggest just removing dark mode for now. Works WAY better in light mode. I almost missed the light mode, and that would have been too bad.

Here's my user test: https://news.pub/?try=https://www.youtube.com/embed/2eac5XZe...

Mockapapella

4 hours ago

Oh wow, was not expecting this level of feedback -- thank you for the video!

Addressing some points you made in the video:

1. Yes, this is definitely a V0.1. Thank you for taking that into account :D

2. Interactive graphs (custom colors, intensity, what trends to show, etc.) were something we thought might be useful but wanted to hear what others had to say before we invest the time into a feature nobody wanted

3. That super slow "Searching..." loading message should be showing updates for when it's scraping comments, when it's classifying, etc. Not sure why it's not. Some searches like "open source" are more likely to have a ton of comments to pull so it'll take longer.

4. Default mode changed to light mode because it looks better :^)

bbor

7 hours ago

Cute, love the idea! Would perhaps cut the examples to be substrings to avoid long comments that mention the issue in passing once, but that’s a personal preference. Either way, actually seeing the graph on mobile is wonky, and IMO the example is way less interesting than the graph — maybe make it bigger and more front n center?

Thanks for sharing :) TIL sentiment for “communism” is slightly less negative than for “capitalism” on here! Tho both are, surprisingly, mostly neutral.

Mockapapella

5 hours ago

1. Comments have been truncated to 100 characters

2. The graphs now go full screen when they are clicked

3. Most comments will be neutral. This is kind of a quantitative way to see "the silent majority"

There's some nuance that is lost due to it classifying the comment as a whole as opposed to specific aspects of it, but it works for the most part. Thanks for trying it out!

lorddustingale

5 hours ago

Thank you for your feedback! We appreciate it.

We’re actively working on making the tool more user-friendly and intuitive. As expected, most sentiment is neutral, but we plan to add a toggle view in future updates to enhance the experience.