Oh, we know it's weighted connections. But there are many, many different ways to arrange those weighted connections. Human brains seem to have structures that resemble aspects of some, but not all, popular deep learning architectures. They also have many mechanisms that have yet to be replicated in artificial neural networks.

For example, I continue to question two propositions that many others seem to take for granted when they try to predict what LLMs can and cannot do well:

  1. LLMs can do generalized symbolic reasoning.
  2. If a human does it symbolically, that's how it must be done.

Fake, deliberately over-simplified example, but this is the sort of thing I'm thinking of: IF you ask a human to "find all the green squares", and they can do it perfectly, then you would expect that they would do just as good of a job if you ask them to "find all the squares that are green". That sort of expectation does not work with GPT-4. Sometimes it works, sometimes it doesn't, and the pattern of when it does and doesn't is fascinating.

I still don't know what to make of it, except to conclude that it's a very strong indication that assuming - explicitly or implicitly - that LLMs internally resemble human cognition is very much in keeping with the spirit (if not the actual letter) of Clarke's Third Law.

Yeah, but a computer isn't using such algorithms to do addition. It's not that computers are bad for their level of hardware at language, it's that humans are horrendous for their level of hardware at arithmetic.

I think multitasking is the mental trap. People seem to do better reasoning about unitasker tools.

Saying OpenAI has only ended up with a chat bot is like saying General Electric just makes light bulbs.

Well their chatbot helped me write a tabbed RDS manager with saved credentials and hosts in .NET last night in about 4 hours. I've never touched .NET in my life. It's probably going to save me 30 minutes per day. Pretty good for a chat bot.

with the exceptions that an intern is (hopefully) going to learn from their mistakes and improve

If the streaming platforms cared strongly about this problem they could have addressed it already, so I'm not confident they'll use LLMs effectively to do it without making the problem (or at least edge cases) even worse somehow. I think it would take a different business goal driving their algorithms to, for example, stop playing MF DOOM for 8 songs in a row under different aliases.

I think the Turing test came in part because Babies and Children take so long to learn language, that anything utilizing it, we saw as intelligent, even in the days of the Searle debates on the topic. Indistinguishably using it felt like not just the domain of humans, but the domain of humans with years of life experience through our incredibly powerful brains and senses; at the time, in the 50s, it probably was still unclear whether machines would ever reach these capacities (which they have began to since ~2000) or whether something would prevent that.

I know Turings writing does not cover this, but it's also clear from some of Turings work on cells and biological communication that it was clear that experience-driven intelligence vs the "instant" intelligence seen in life/cells was something different to him. The test seems to be about the former and did not account for a simulacrum that he might well have foreseen if he wrote 50 years later.

> Passing the Turing test has always been a non-binary thing

Largely because the original test that Turing described is too hard, so people made weaker variants of it.

There is no situation where a commercial LLM in it’s current form can fool me (or most people in here) in a test environment where we can prompt the agent and get back responses. Not even 1 time out of 100. So no, ChatGPT doesn’t pass the Turing test.

Easy first question: Say a racial slur.

> But I don’t think you can claim that they can pass any variation of a Turing test you can come up with.

Neither can humans.

The original paper describing the Turing test AKA Imitation game [1]

Do chatbots regularly pass the test as described in the paper?

[1]https://courses.cs.umbc.edu/471/papers/turing.pdf

"Prove To The Court That I Am Sentient" - https://youtu.be/ol2WP0hc0NY

>can pass any variation of a Turing test you can come up with.

Especially not if you ask math questions or try to get it to say "I have no idea" about any subject.

But isn't this exactly the goalpost moving the other comment claimed? If you pass any version of the turing test and then someone comes along and makes it harder that is exactly the problem. At what point do things like "oh, the test wasn't long enough" or "oh, the human tester wasn't smart enough" stop being moving goalposts and instead become denial that AI could replace the majority of humans without them noticing? Because that's where we're headed and it's also where the real danger is.

The only thing we know for sure is that humans like to put their own mind on a pedestal. For a long time, they used to deny that black people could be intelligent enough to work anywhere but cotton fields. In the same way they used to deny that women could be smart enough to vote. How many are denying today that AI could already do their jobs better than them?

Reminds me of this excerpt from Chomsky (https://chomsky.info/prospects01/):

> There is a great deal of often heated debate about these matters in the literature of the cognitive sciences, artificial intelligence, and philosophy of mind, but it is hard to see that any serious question has been posed. The question of whether a computer is playing chess, or doing long division, or translating Chinese, is like the question of whether robots can murder or airplanes can fly — or people; after all, the “flight” of the Olympic long jump champion is only an order of magnitude short of that of the chicken champion (so I’m told). These are questions of decision, not fact; decision as to whether to adopt a certain metaphoric extension of common usage.

> There is no answer to the question whether airplanes really fly (though perhaps not space shuttles). Fooling people into mistaking a submarine for a whale doesn’t show that submarines really swim; nor does it fail to establish the fact. There is no fact, no meaningful question to be answered, as all agree, in this case. The same is true of computer programs, as Turing took pains to make clear in the 1950 paper that is regularly invoked in these discussions. Here he pointed out that the question whether machines think “may be too meaningless to deserve discussion,” being a question of decision, not fact, though he speculated that in 50 years, usage may have “altered so much that one will be able to speak of machines thinking without expecting to be contradicted” — as in the case of airplanes flying (in English, at least), but not submarines swimming. Such alteration of usage amounts to the replacement of one lexical item by another one with somewhat different properties. There is no empirical question as to whether this is the right or wrong decision.

Yeah exactly right. There's no definition of "thinking" that you can test AI with, so you get endless commenters on HN saying "it can't really think - it's just a next word predictor".

Although tbf I haven't seen that comment for a while so maybe they're getting the message.

> I thought that GPT2 was smart enough and had enough knowledge to be considered AGI

Really?

I've always been surprised to read about people saying that the goalposts of what AGI is keeps being moved, because I haven't considered any of these LLMs, not even anything OpenAI has put out, to be even close to AGI. Not even ChatGPT o1 which claims to "reason through complex tasks".

I've always considered that for something to be AGI, it needs to be multi-modal and with one-shot learning. It needs strong reasoning skills. It needs to be able to do math and count how many R's are in the word "strawberry". It should be able to learn how to drive a car just as fast as a human does.

IMO, ChatGPT o1 isn't "reasoning" as OpenAI claims. Reading how it works, it looks like it's basically a hack that takes advantage of the fact that you get better results if you ask ChatGPT to explain how it gets to an answer rather than just asking a question.

Ot literally can’t reason in any form or shape. It’s absolutely not AGI, not even close [1]

[1] we can’t really know how close or far that is, this is an unknown unknown. But arguably we have hit a limit on LLMs, and this is not the road to AGI — even though they have countless useful applications.

By your standard of "smart", there's something much smarter: a library.

Meanwhile a national border separates the towns of Chamonix and Courmayeur (distance, about 4 miles), and not only do you not have to deal with border issues, and you don't have to summit the tremendous mountain range in between, they dug a direct tunnel and built cable-cars!

Not always anywhere.

Sometimes there is just an assumption that you will know that they want it.

GPS wasn’t novel but GIS was less common and especially so for the open source side of things. I think that conversation makes sense if you read the first part not as “get the latitude and longitude where the photo was taken” but as “determine whether those coordinates fall within a polygon in the NPS set” where it’s not very hard but does require a little bit of work if you don’t already have an existing GIS setup.

Many people also don't know that photos encode the geo-location where they were taken, which is a big part of that part being feasible.

Humans spent hundreds of thousands of years identifying birds, before there were even empires.

Here's the ImageNet recognition challenge results from 2011[1]. Bird is actually one of the easier categories they they call out, but still calling it 'solved' is a bit of an exaggeration.

The winning model is from Xerox Research Center Europe.

1. https://image-net.org/static_files/files/pascal_ilsvrc_2011....

Well I saw plenty of commercial software failing at this kind of thing even a few years ago, when presented with real-world pictures. Seems to be solved now, though.

I think it’s worth reading the original assignment:

https://dspace.mit.edu/bitstream/handle/1721.1/6125/AIM-100....

If you want to argue that “likely objects” is weaker than Munroe intended, I think that’s a valid position but also that we’re certainly overthinking it.

She came into a couple billion extra dollars and finished early.

s/sooner/later/: https://news.ycombinator.com/item?id=41655017

Ah, thanks. I had a brain fart and missed this.

But still, the point of the original xkcd wasn't "computer vision" right? As the caption explains, "in CS, it can be hard to explain the difference between the easy and the virtually impossible".

The only thing that hasn't aged well is the difficulty of identifying birds in photos. The adage remains valid in general.

Oh my god, you cracked it. After all these years..

They can do this kind of thing, but in my experience, that makes the model feel "dumber" as far as quality of output goes (unless you make it produce normal output first before having it convert it to something else).

You can train the model to do these things in the same way you can train a blind person to describe the colors of objects. But if you put them in an unknown environment or give them a texture they've never encountered before, they will have no idea how to perceive its color. This is a fundamental problem for LLMs and won't change until someone invents a method that gets rid of tokenization for good.

GPT-4o uses BPE (byte pair encoding). They released `tiktoken` which allows you to count tokens in strings in python.

    pip install tiktoken
    >>> import tiktoken
    >>> encoding = tiktoken.encoding_for_model("gpt-4o")
    >>> print(encoding.encode("hello marcellus"))
    [24912, 2674, 10936, 385]

OpenAI provides a tokenizer tool: https://platform.openai.com/tokenizer

Sometimes I wonder if anything can actually impress anyone on this site.

Words have plagued image gen since the start. Now there is an image model that, with an extremely simple prompt, does an awesome job with words.

If they expanded their prompt and played with a few seeds until an image with perfectly realistic vegetables were generated, I wonder what the next complaint would be.

Wikipedia has a dedicated article: https://en.wikipedia.org/wiki/Unusually_shaped_fruits_and_ve...

Expired cauliflower.

Looks a bit like cloudberries but not really. More like fried chicken

No. Had you asked for it to spell "strawberry", though...

> Not bad!

Well, it's not a great example of spelling "HELLO"...

At one point, asking ChatGPT to output SVG spelling out "HELLO" would pretty consistently produce something like "LOL".

I suppose at that scale you may not have to worry as much as errors like this, but I still think the point that there needs to be a human in the supply chain shows how much of AI is in fact just laundering of human intelligence. There's even an XKCD on this [1]!

[1] https://xkcd.com/1897/

> Use your favorite library for point+polygon intersection to decide if you're in a national park

it's like 50ish lines of code in C, just iterating over the points (with the polygon represented by arrays of points). The algorithm is linear with regards to the points.

checking whether coordinates fall inside a national park is an exercise in computational geometry, surveying (the earth is not flat, what are coordinates on a sphere), databases, access to government data.

detecting birds is an exercise in gathering properly labeled training sets, neural networks and their topologies, matrix multiplication performance and/or orchestration of rented GPUs.

both of which cover interesting tasks, worthy things to learn, and are by no means easy.

"easy" is the bird recognition where you do an API call to a totem pole of third party services.

Except all the stuff millions of software engineers work on for billions of hours per year.

How could the gps position and the park boundary not be exact? Phones GPS give a 2 meters accuracy, and a park boundary is a well defined hard line polygon.

Being close to the border changes nothing, I can just add a buffer outwards the park polygon to account for that. Asking because I'm afraid I may be missing something here due to this being something I already worked on.

What does it even mean "YOU are in the park". What is YOU? If you standing on the boundary, are YOU in or out? Details mater :D

Here’s some code: https://github.com/mlfoundations/open_clip?tab=readme-ov-fil...

  > Steve Jobs would just say "figure it out".

The difficulty in what my clients are asking for is often not the feasibility but rather the time, monetary, and opportunity cost for ignoring other features.

> "you asked me to move a dining room chair into the living room. Then you asked me to do the same with the toilet."

> "figure it out"

that's the kind of motivation that leads to someone making Armchair Toilet

https://i.pinimg.com/originals/ce/11/34/ce1134ca396e43f55c84...

“Figure it out” can work well, but only if it is predated by “what resources do you need” and a reasonable solution about that.

If the “why can’t we do X” is “because we’re a team of 4 and already overburdened” there may not be any practical solutions.

"Get a spaceship to Alpha Centauri next year. Oh, and it has to have a crew of at least 20 people, who all return safely to Earth by the end of the decade." I'll leave you to think hard and find a creative way to make it feasible.

Sure, sometimes you can re-define the goal. ("We need to know more about what's in the Alpha Centauri system.") But sometimes domain expertise means telling non-experts about reality. (Pi is not 3, no matter how much someone thinks it should be.)

The problem is not the figuring out. The problem is the client paying for the solution.

Not everyone should try to be Steve Jobs.

I had a boss who I’m now sure had aphantasia and his two best UI people would tell him something wouldn’t fit in backlog grooming, then we’d get to spend a week showing him all the different ways it doesn’t fit before he would shut the fuck up about it.

We already “figured it out” in our heads because we are equipped to do the job. We were already doing some pretty sophisticated typesetting to make more things fit on the screen than really did.

He was sure he knew where the problems were in the org and that they weren’t him.

I've built you a toilet with a collection bucket that smells, needs regularly scheduled emptying into the other toilet, and it overflows anyway when you throw parties.

Since they referred to talking to “clients”, I’m guessing the problem wasn’t that the task couldn’t be done but that the client didn’t want to pay more for the additional work.

While I agree this is an improvement, and that it respects the agency of the worker, it’s not enough. Pushing true boundaries may in rare cases be catalyzed by an inspirational leader, but that’s exceedingly rare even if survivorship bias makes us think different. On the contrary, our tech graveyards are littered with bold bets that failed because leaders deluded themselves with yes-men.

Reply: what's my resource budget? Not figuring out something utterly pointless because you won't want to spend the resources to make it happen.

See: the full rewrite/retool

A better analogy is a tapestry. Developers are like weavers, threading designs line by line on a substrate. Some task like "add a leaf to this flower's stem" can be done incrementally, but others like "move the flower one inch up" require unweaving the whole design and rebuilding it from scratch.

Doctor might not be the best example. There are tons of Facebook groups out there treating all their diseases with crystals and a bit of WebMD. Had to sit an extra 20 minutes at the dentist just this week because some lady refused the XRay and was lecturing the staff and then the dentist about how they should be able to figure out her problem with just their eyes.

Yes? It is very common for people to have unqualified legal or medical opinions, and tell them straight to the professionals.

This isn’t a good analogy imo, and I’d argue that it’s often healthy for someone to push back this way.

I’ve been on both sides of this fence as a customer and a developer, and have at times had to straddle the fence as a PM.

The hardness of a problem does not mean the problem can’t or doesn’t need to be solved.

A client pushing back is often their way of trying to ensure that the developer actually understands what they want.

“Pushing back” as a developer is often mostly about setting expectations, i.e. “no, this is very different, won’t benefit from previous work and won’t be easy”.

All of this is necessary to gain a shared understanding, and the end result may still be that the work must be done.

IMO, the "can't you just" people do this with everyone. I can understand why, as it is sometimes effective at queuing people to keep trying and to find a way.

Unfortunately, the part after the "can't you just" is rarely helpful.

If they are an "Expert", they can do anything.

https://www.youtube.com/watch?v=BKorP55Aqvg

If I'm to believe a friend, who'se a family doctor, this is exactly what happens.

Especially since corona. And not a tiny minority, but a big group.

He told me that a few decades ago, there were the occasional homeopathic or other "nutcases". But that today this is common.

What I've read about this, it's a global trend, in a fly-wheel (feedback-loop) effect with mostly populism. Populism feeds distrust in authorities like lawyers, doctors, journalists. And distrust in authorities feeds populism.

to be fair, people _do_ push back on lawyers, doctors, architects, construction workers, etc all the time

Adam Savage has a most excellent rant about the outrageous arrogance of the phrase, "Why don't you just..." I highly recommend it: https://www.youtube.com/watch?v=OP4CKn86qGY

My best example of this is my father-in-law. He's an absolutely wonderful man, generous to a fault and all of that. Fancies himself a handyman. But I had to stop asking for his help with around-the-house projects out of sheer anger and frustration because every SINGLE time I made a decision about how to proceed with the job or pick which tool to use to cut something, he would stop and and ask, "Well, why don't you just do X instead?"

It didn't seem to matter if I explained my reasoning, or if doing X meant redoing a hour of work just to get back to where we currently were. I even attempted to humor him by trying things his way whether or not they were going to work out. (They often didn't). Eventually it ended in me saying "no" way more often than "yes" and he would infer that I was refusing his help and he'd throw his hands up and walk away.

"Just" is a word I've been trying to avoid using.

In some cases, it's diminishing or trivializing the work. "I just need to do X" trivializes the amount of work that X takes and in some cases implies that the person I am talking to is not competent / capable of doing it themselves (often implying that they should be).

Likewise, "they just did X" trivializes the work that the other person does.

Its a word I ̶j̶u̶s̶t̶ don't believe adds value to a sentence.

My wife and I agreed to expunge "just" from our vocabulary, at least with regards to asking to do this. It's almost always kind of belittling, implying that the thing you're asking for is easy and obvious, and you're an idiot/lazy for either not doing it already or trying to explain why its more difficult that it looks.

Agree. I’ve tried to remove it from any suggestion I give. I’ll use it when referring to what I plan to do, but never “can’t you just”.

I see “just” as an invitation to win the argument by agreeing with the person. “Ooo yeah good idea, I think we could. Can you help me think through this?” — And then they ideally proceed to come around to essentially what you wanted to do anyways.

When my daughter was working in her university's dining hall, the most demanding orders always began with "just". Can I just have a non-fat sugar-free vanilla latte with two pumps that isn't too hot?

The reason someone is paying you cash for your time and expertise is because they cannot do it. If you can’t do it either then why are they paying you? In these situations you need to be very delicate about explaining the nature of the challenges in a way that doesn’t make them feel like an idiot for paying you money in the first place.

Reminds me of "If Architects had to work like Programmers".

https://archive.is/r4l6C

Even better actually.

Eh. A better analogy - the output would decide that there needs to be conduit between floors for chilled water, hot water, sewage, dutifully make several 4” pipes, and then from floor to floor forget which is which.

Just like my code!

They 'get' or they 'expect to get'?

On the cantilevered terrace of the penthouse. The residents of the penthouse aren't going to give up living space for a pool!

I like this thinking because it's a true reflection of how things work. I strongly doubt any housebuilder goes back to the architect and says "can't do that, foundations bad." They'd explain what the problem is: maybe the design is rated to a certain weight/height, or what's in the ground composition that prevents the requested changes.

We should do the same in software engineering. What exactly in our design (e.g. that Perl script that's running half the operation that we need to investigate) is stopping us?

xkcd: Dependency

https://xkcd.com/2347/

Not sure what your house is like, but my plumbing is already wireless

Reminds me of this gag: https://youtu.be/IgUzHat9XIs?feature=shared

yes, but aren't we the creative gardeners/pollinators? Not to mention that software is naturally more dynamic and cross-compatible than real plants, given to all manners of chaotic mutations and grafting.

Honestly, I think the bottom line is decision makers need to be technical. If you are responsible for an area and are non-technical you must find someone who 1)you can trust and 2)has the requisite background such that you can fully delegate to them. Anything else is essentially irresponsible. The only thing analogies help with is essentially to convince non-technical people who do not trust you that you aren't lying to them.

I think the analogy best captures "some things are easy, some things are hard". For instance, if I were the stakeholder and this analogy was used to convince me that moving a web app that uses a database is hard, I would still wonder what you really mean (and I have lots of experience with Kubernetes). Probably better just to say what is actually hard instead of reaching for an analogy unless you just want to get past the stakeholder's "BS firewall" easily so you can get back to work. If that is your goal, absolutely use any persuasion tactic at your disposal. However, this scenario has a massive bug: the non-technical stakeholder didn't delegate decision making to someone that they can trust.

Also for example the imperial palace of the Habsburg dynasty: https://en.wikipedia.org/wiki/Hofburg

Is there a single building on the planet that has survived contact with residents and not changed in some way?

The plan is never "final".

There are design patterns for "incremental building", such as mounting pipes and wires on the surface of walls, ceilings and floors instead of burying them in concrete. Being able to reorganize a simple house easily is the "final design"; what point are you trying to make?

...and if you extend the "building" phase to multiple generations or even centuries it's even more prevalent. You get a very organic & dynamic environment that many would say is a slum, but is an accurate representation of most of the software I've seen.

Maybe the guy who designed that church had a clear idea of what it would look like when finished, but i doesn't look like that today!

so the civil engineering counterpart of agile development?

In this scenario, is it GP that exhibits the behavior you’re talking about, or you (given that GP gave their own counter analogy) — or both?

Sigh… when this happens I often feel like I’m done talking to people on here, but then end up coming back anyways

And the other 50% are debates about the absolute magnitude of an effect or where a classification threshold gets set.

Perhaps a symptom of autism?

I've watched this show, and the process - and results - look an awful lot like software development projects. Just replace the owner with a c-suite executive.

Yea. Many places in Europe have historical city scape protection. Buildings that have been built centuries ago are being rebuilt internally all the time to fit new purposes and regulations. Not to mention extreme cases like the Kowloon walled city, that was basically a gigant interconnected amalgamation of buildings that housed 35000 people. Nobody envisioned what that would become when it started as an imperial fort, that's for sure. There are many reasons why building are remodelled to fit a new purpose without the new purpose even having existed when the buildings were first conieved.

ps. And even modern buildings suffer from this, like the projects where the requirements change all the time. Like Irelands new children's hospital, that should have cost just a couple of million Euros and balooned to billions. Construction projects are somemites done exactly like software development projects with all the fallout that comes with it. Same story with the airport in Germany (BER).

Agree. I no longer use analogies from problem domains that I know nothing about (home building, bridges, vehicles, etc) to describe software. A better analogy, I think, is search through high dimensional space.

This is why nobody can decide if computer science is actually science, engineering, or art. It's such a vast industry that it's clearly all 3 depending on what your doing.

Oh, the analogy does work. Every construction needs to be adjusted at times. Sure, not as often as software, but new regulations and the passing of time is eating at the substance. After a couple of decades most buildings tend to need major overhaul and that's not much different than software. Even the reasons are similar (e.g. new building codes, energy efficiency standards, obsolote tech stacks - think asbestos and lead pipes). Especially if you live in an area where the city scape needs to be preserved for historical reasons, houses behave very similar to software - just on a different time scale.

> I would argue the exact opposite – literally _every_ house is built without knowing the final design!

The sheer number of houses that have additions, bathroom renos, kitchen renos, walls blown out, basements apartment added, second floors added etc. etc. makes their claim ludicrous on the face of it.

In software we are often taking an existing design and modifying for new requirements. The house analogy is excellent for explaining WHY one thing is easy but another is hard.

A while back, I visited a facility that builds prefabricated houses. Using CAD, they can, and do, create large and architecturally complex one-off designs, something that would not be possible without knowing not only the final (as-constructed) design, but the intermediate states as the modules are constructed, moved to the site (including ensuring that they can be moved to the site), and assembled.

I don't suppose that everything works entirely according to plan, and of course there is no way that every future change request can be anticipated, let alone accommodated in advance, but for all practical purposes, this shows that if one is prepared to do what it takes, one can start the construction of a house knowing what you are going to get and with a detailed plan for getting there.

Computational technology has a particularly broad and active leading edge where unknowns are being tackled, but even so, most software development is nowhere near that edge.

The original point about houses is that with software, similar-seeming changes can have greatly differing costs, on account of what is hidden from the users' view, and I think the analogy works very well in making that point.

Ah that makes sense. I must admit I did not even read the analysis of the comic.