giobox
5 hours ago
I just asked it to create a torque spec diagram of the suspension for my car, a subject I'm pretty familiar with. It amazingly drew everything correctly, displayed the correct torque figures and allowed me to click on individual components to zoom in further, providing more specs.
Genuinely one of the most impressive demos I've tried in a long time. I was able to use it almost like a living version of a classic illustrated Haynes workshop manual.
tomashubelbauer
5 hours ago
I asked it about designing a 12 V solar system for a garden shed and it got everything but the broadest of strokes wrong. It figured out there should be a solar panel, a solar charge controller, a battery and some loads, but the wiring was non-sensical and when I drilled in on the solar charge controller settings etc. it completely fell apart. Absolute non-starter for any information you plan on depending on, but good entertainment value and impressive execution.
debo_
an hour ago
I queried "your mom" and it created a historical social timeline of motherhood superimposed with a placenta. I approve
tiltowait
an hour ago
Interesting! To join the cavalcade of others sharing their experiences:
I first asked it "how big are geckos". It gave me a cool comparison diagram between three gecko extremes (leachianus, Jaragua dwarf gecko, and leopard gecko, if curious). Info all looked correct. Drilling into the Jaragua brought me to a less-impressive page with utter gibberish text and duplicated info boxes. So it goes. I drilled further, but they were more esoteric topics I'm less versed on (lamellar setae), I can't evaluate the accuracy without further research.
I also gave it something broader: "tokay gecko". More duplicate info boxes, and for some reason it "drew" two geckos on top of each other. Kind of cute, but tokays are extremely territorial, so happy cohabitation isn't their default (though it's not unheard of).
Still, despite the issues, I thought it was very neat.
thegrim33
4 hours ago
I decided to test it out myself.
Went to the website, typed in "Jeep Wrangler JK engine bay with components labeled" (Since I'm intimately familiar with JK engine bays). Seems like a pretty analogous test to what you did, if anything an even easier test.
Let's see what we get .. a very nice looking diagram of a wrangler engine bay with components labeled, looks good.
But wait ..
- The brake fluid reservoir is on the wrong side of the engine bay
- Where the brake fluid reservoir is, it's labeled as the coolant overflow tank, and while the actual coolant overflow tank does exist in the diagram, it has no label.
- The battery is on the wrong side of the engine bay.
- The top of the front grill is labeled as the "oil filter cap".
- The oil fill cap is in the wrong place.
- Half of the battery is labeled as the fuse box, when the fuse box is correctly shown, but unlabeled, on the other side of the engine bay.
- It shows two different windshield washer reservoirs next to each other.
I could keep going on ...
Now I tried clicking on the incorrectly labeled coolant overflow reservoir and it switches to a new page which now shows a completely different looking coolant overflow, but now it's at least located in the correct place in the engine bay.
But of course it doesn't look remotely like the actual coolant overflow container. It also shows the radiator cap as on the top of the coolant reservoir, when in reality it is very much on the top of the radiator itself.
Like .. I can find fault with every aspect of it. But of course, if you didn't actually know much about the topic it'd all look fairly believable. The story of LLMs basically.
ofjcihen
an hour ago
Does it make sense that maybe it has a model of the vehicle it can pull from its corpus wholesale but then the “guess the next letter” portion takes over for labeling and just guesses poorly?
toraway
4 hours ago
I had a tab on nuclear reactors open and so typed in "Pressurized Water Reactor" and the result while very visually appealing is completely nonsensical (connected the high/low pressure coolant loops together) and would definitely explode.
dugidugout
4 hours ago
It does poorly on creative concepts as well.
I attempted to explore the works of Kinoko Nasu/TYPE-MOON through its characters and the relationships across works and it was mostly nonsense. Sure it had some broad relations correct, but it presented a tiny set of meaningful characters and only attempted to touch Fate/Stay-Night and Tsukihime.
Even more damning was that it produced garbled text for a few of the textual representations and often even if the lettering was clean, the grammar was off.
jazzypants
3 hours ago
Do we ever simply accept that LLMs weren't made for this kind of detail-oriented work? I can't imagine something like this ever being anything other than a toy which can't be trusted.
Will Silicon Valley executives ever accept this reality? If we acquiesce and admit that LLMs are a good tool for prototyping and boilerplate-reduction, but not finished products-- is that when the bubble finally bursts?
macprothrowaway
4 hours ago
I also replied because I asked it about a Mac Pro case I had right in front of me. Mostly right words, totally wrong visuals. And while I see what you mean by 'story of LLMs', I ask LLMs about things I know often, and for the last 12 months theyve been pretty dang accurate. This ai visual example is the strongest 'its just guessing' Ive seen in years. For a demo, pretty cool still though. Not sure why OP exaggerated, or simply doesnt know his car as well as he thinks he does.
macprothrowaway
4 hours ago
I have a Mac Pro 5,1 taken apart on my desk right in front of me. I asked it for a diagram of the 5,1 internals. While it was MacProish looking, it was wrong about every visual element. The text fields were right at first glace. Every click I did was basically all wrong too. Visually it looked cool, but actually the first time Ive seen AI be wrong constantly since maybe 2023.