Openpilot – Operating system for robotics

113 pointsposted 7 hours ago
by punnerud

65 Comments

rsp1984

5 hours ago

Without taking away anything from the substance or achievement of this release, I find phrases like "openpilot is an operating system for robotics." always quite fishy.

No, it's not an OS for robotics. You can't do actual robotics stuff with it, like drive actuators to control limbs or grippers, do motion control or SLAM or perception or any of the usual robotics stack.

Their website correctly says openpilot is an open source advanced driver assistance system that works on 275+ car models of Toyota, Hyundai, Honda, and many other brands. Should've stuck to that.

Thinking about it some more, it's probably just another engagement baiting strategy to get attention and I'm their gullible puppet. Well played.

conradev

2 hours ago

I believe the idea is that openpilot replaces the usual robotics stack with an end to end neural net.

While I agree operating system is usually a marketing term, it does feel correct in this case as it is the operating system for the Comma Three, which can operate cars but also this thing: https://www.comma.ai/shop/body

metal_am

4 hours ago

I definitely thought it was a ROS clone based on that first line.

notum

4 hours ago

ROS doesn't need a clone, it needs a successor.

Took the bait as well.

spookie

35 minutes ago

ROS2? I'll see myself out...

punnerud

3 hours ago

Isn’t the software for training end-to-end NN to be used in automation? Just a first version that it’s used for cars, and they have been using it for their own robot.

So the claim still stands?

tamimio

2 hours ago

Yeah came to say the same, I thought a new big player is in the market. It looks great nonetheless.

jmacd

3 hours ago

I have a Dodge Ram. Last night I had a 400km drive to do after a very long day. I wasn't exhausted, but I certainly felt like I did not want to drive an extended period of time.

I have a Comma 3x in the truck and felt way more confident, alert and comfortable for the entire drive. OpenPilot/Sunnypilot/Frogpilot are not FSD, but they are hands off driving assistance. The 2020 Ram performs incredibly well. The latest driving models are very smooth as well, no ping-ponging and they handle passing and traffic extremely well.

A legacy car maker would be smart to acquire Comma if its for sale. They would be extremely close to a viable assisted driving capability with it.

pj_mukh

an hour ago

Mind boggling to me that a non-ping pongy lane keeping is not standard in cars. Is it standard in luxury cars? Seems like an obvious thing to add/upsell.

hasperdi

8 minutes ago

Non ping-ponging lane following assist is already available in many cars including KIA and Hyundai models. They're very conservative and disengage very easily. I think it's by design to minimise their legal accountability

bilsbie

5 hours ago

I can’t wrap my head around the fact that 275 car models include all the actuators needed for self driving driving and there’s some kind of port third party software can hook into.

kube-system

2 hours ago

Level 2 driving assistance is commonplace on many new vehicles, often as a standard feature. They are just significantly more conservative in their functionality compared to the level 2 offerings from Tesla, and marketed as safety features rather than "self driving" features.

It's important to note that nothing we're talking about here is actually "self driving" per SAE standards. Openpilot, Tesla's Autopilot/FSD, Honda Sensing, Toyota Safety Sense, Hyundai SmartSense, etc are all level 2 driving assistance features.

This turns level 2 driver assistance features into ... nicer level 2 driver assistance features.

mavhc

2 hours ago

Really need some decimal points in the SAE list. Fully self driving but a human has to watch, and lane keeping are both level 2

flessner

5 hours ago

I don't know if there's a physical port in most cars, but it uses the CAN bus which has been around since the 1980s.

Also, most cars that have distance assist and lane keeping probably have the required hardware to control speed and steering to some extent.

Nevertheless, it's still impressive that so many cars are supported... and that it can be retrofitted like this at all!

fkyoureadthedoc

4 hours ago

They also lie about models that are supported and won't assist you when you run into that. Had to return one myself. Found no evidence anywhere that my model/year was ever actually supported and anyone was using it either.

BitsAndBlobs

2 hours ago

There's a very active community on Discord that would have been happy to help you add support for your car assuming it has the right hardware.

https://comma.ai/vehicles has also been improved quite a bit in the past year.

jmacd

3 hours ago

Sounds like they refunded you?

olabyne

3 hours ago

Could you say what car ?

fkyoureadthedoc

2 hours ago

2023 Forte

https://www.reddit.com/r/Comma_ai/comments/197k04q/2023_kia_...

https://github.com/commaai/openpilot/issues/30936

You can find a few attempts of people trying to get it to work in their discord with no clear positive outcome, discord is unfortunately not search indexed.

BitsAndBlobs

2 hours ago

It looks like support was added in February

https://github.com/commaai/openpilot/pull/30761

fkyoureadthedoc

2 hours ago

Yeah it was on the supported list when I bought it in October of 2023. The website asks your make/model/year when you buy if I recall correctly, that's how it tells you which harness to get.

I found out I needed to update some files in the firmware, followed the guide for that. Asked for help in the Discord. I could never get it working though and returned it after a week. I was going to hang on to it and harass people on Discord, but didn't want to lose track of time and go beyond the return window. Believe me, I really wanted it to work.

Looks like it was added to the supported list before the explicit support I guess

https://github.com/commaai/openpilot/commit/e3275e918354945d...

michaelmior

an hour ago

I beleive in the US OBDII ports are required in all vehicles 1996 and later.

bdavbdav

5 hours ago

I suspect it may be even more than that theoretically. A lot of VAG cars based on the same platform are missing.

akgrd

5 hours ago

This seems to be a mix of C++ and Python, including a script called "realtime.py" (oxymoron?). So am I now exposed to other people using Python on the roads to operate heavy machinery?

AyyEye

an hour ago

The interface between the openpilot and the car is a standalone device (the panda) that provides and enforces the safety model. All code is written in C to automotive safety standards including ISO26262, ISO11270, ISO15622, and MISRA-C. 100% line coverage for all safety unt tests.

They also run pretty extensive tests (regression, unit, hardware/software-in-the-loop, mutation, and vehicle specific) on every commit and have actual hardware devices continually running real routes looking for regressions.

https://github.com/commaai/openpilot?tab=readme-ov-file#safe...

https://github.com/commaai/panda?tab=readme-ov-file#code-rig...

enragedcacti

42 minutes ago

> So am I now exposed to other people using Python on the roads to operate heavy machinery?

yes in the sense that python is running the ML models and deciding what the vehicle should do, but it is heavily bounded in what it can do by the safety model which is implemented in bare-metal MISRA C running on the microcontroller that interfaces between openpilot and the CAN bus (panda). It enforces things like accel/braking limits and steering rate limits along with consistency checks, heartbeats, vehicle status checks, etc.

Level 2 self driving is already only a best effort system so if python caused an issue it would just fall back to the safety model on the panda and ultimately the driver to operate the vehicle safely.

traverseda

4 hours ago

Well NIST says don't use C++ either: https://www.nist.gov/itl/ssd/software-quality-group/safer-la...

So what, you want everything written in RUST on a linux kernel with hard real-time patches? It uses machine vision anyway, which has no hard guarantees at all. The software it uses to detect lanes or cars is probabilistic by it's very nature.

Python does pretty good at soft real time if you manage your own event loop and disable the garbage collector, and you're a lot less likely to get "crash the entire stack" style memory allocation bugs. Sure, GO or RUST would be better, I think CPP could be worse if handled inexpertly.

thisismyswamp

3 hours ago

exception handling in GO makes it unusable IMO

tmarkman

4 hours ago

Python has segfault issues, surprising exceptions and version incompatibility.

I've been using Linux/BSD for over a decade now. No C or C++ application has ever crashed, I cannot say the same about Python applications. Outright segfaults are rare but happen. Rogue exceptions are much more common and could basically have the same detrimental effect on a self-driving system as a segfault. And let's not talk about logic bugs due to version incompatibility and the obsessive rewriting of those who took control over CPython.

traverseda

4 hours ago

Ahh, you've been running some grad students first python project as if it was a serious project like curl with 20 years of history, and expecting it to have the same quality. But you've somehow avoided the tons of grad-student CPP programs with similar quality issues, or the broken code pushed by companies like crowdstrike or IBM.

Fair enough, your experience may vary. I'd suggest not judging the language by the standards of some hobbyist code that just so happened to end up on github. I've had tons of bugs in c/cpp programs over the years, some more critical than others.

I've seen a lot of shitty and unreliable python code, and a lot of good and mature C/CPP projects. I've also seen really bad security issues and crashes with bad C code, heartbleed, crowdstrike, etc.

For what it's worth I've never had youtube-dl hard crash on me, and I could argue that it's a more complicated problem to solve than what curl is solving. In an apples-to-apples comparison I think it does pretty well.

No matter what language you use for this you're going to be relying on an AI vision model with no hard guarantees.

bee_rider

3 hours ago

Actually Python was insufficient for the sort of grad student bugs I wanted to write, I was able to just wrap everything up in giant try blocks and then,

    except:
      print(“Something happened”, i) 
(Where I might be an index. Or an element).

Fortran is able to generate better bugs, because it has allocate/free.

plrandk

3 hours ago

You have much more control over a pure C/C++ application because it does not involve the Python runtime. Crowdstrike etc. are exploits that don't really matter here: If you are on the CAN bus it's game over already.

That said, I'm pretty sure CPython has exploits, too. They'll be harder to find and trigger though.

traverseda

2 hours ago

Sure, runtimes exist and have engineering trade-offs. You avoid a whole class of memory related bugs but you lose a lot of control over memory allocation. You can do soft real-time as long as you manually manage the garbage collection and accept that there will be some (bounded) jitter on memory allocations.

The first rule of the tautology club is the first rule of the tautology club. Things have trade-offs. Python removes (or at least significantly reduces) a whole class of bugs that appear when using lower-level languages, that's part of why it's a good glue language.

jedberg

2 hours ago

Sort of. It operates after your vehicle safety systems, so yes, python is controlling the car to an extent, but only within the limits of the built in safety systems.

bks

5 hours ago

I chose the Hyundai Ioniq 5 as my current car specifically because it’s compatible with OpenPilot. It’s been a total game-changer for my driving experience. Just like their tagline says, “make driving chill,” and for me, it truly delivers on that promise.

sofixa

4 hours ago

This reminds me of Waymo's approach to self-driving cars. Paraphrasing, but basically they found that progressively adding self-driving to help human drivers is bad, because it leads to the humans becoming complacent and not paying enough attention. Therefore they decided on an all or nothing approach, where their cars would be only and entirely self-driven.

typewithrhythm

3 hours ago

This always seemed like a bit of bull from waymo. It's not an easy problem to work with existing manufacturers to give a better and or cheaper solution... Especially when there are established competitors with efficient verification and validation processes (that every manufacturer requires).

They decided it wasn't worth explaining that their techniques don't generalise to a driver assist. It would not be good or cheap enough to be worth developing the compliance and integration frameworks.

jowday

3 hours ago

The first thing Waymo tried building (way back when, circa 2010 or so) was highway based driver assist in the style of Autopilot. They did a ton of testing with it and didn’t like how quickly their testers stopped paying attention despite a ton of instruction not to. I’ve seen clips from these tests.

It’s also possible they shifted direction because the long term vision of robotaxis is much more lucrative.

warble

2 hours ago

Maybe I'm being ignorant about something here but isn't paying less attention the whole point?

sofixa

22 minutes ago

Unless you assume that self-driving software is perfect, no, it really isn't. That's the whole problem - the drivers would get complacent, so when there's an issue, they'd be caught by surprise and wouldn't be able to react.

mook

an hour ago

Isn't the point to pay _no_ attention? The difference is when an accident occurs, was the person in the car at fault for not vigilantly watching everything.

zrt1019

5 hours ago

I'm confused:

"THIS IS ALPHA QUALITY SOFTWARE FOR RESEARCH PURPOSES ONLY. THIS IS NOT A PRODUCT. YOU ARE RESPONSIBLE FOR COMPLYING WITH LOCAL LAWS AND REGULATIONS. NO WARRANTY EXPRESSED OR IMPLIED."

Where can this be used? In a private parking lot?

rogerrogerr

5 hours ago

The driver takes liability, of course, and this can be used wherever the driver deems it safe and useful.

diggan

4 hours ago

> this can be used wherever the driver deems it safe and useful

With the disclaimer that this depends on the location of course. For example, I think in Spain (and probably EU wide?) modifications that affect steering and throttle control would need to undergo local homologation before you're legally allowed to drive with that on public roads at all.

Which, to be honest, makes a lot of sense. I don't think anyone would be happy if cars start using software MVPs automatically controlling throttle and steering while in real traffic.

SkyPuncher

4 hours ago

They’re just trying to scare away people who thing they can chuck this on their car and suddenly have a self-driving robot that they don’t have to pay attention to.

torlok

3 hours ago

You pay a thousand bones for a device you have to babysit. What's confusing about that?

thatgerhard

5 hours ago

Is it like an app you install on the car or is it a custom integration?

rvnx

5 hours ago

It's a dashcam that you put on the windshield with 2 cameras pointing forward and one inward (filming the driver).

ozzyphantom

5 hours ago

It’s my understanding that in addition to the cameras it also uses the sensors already built in to the car which would include blind-spot detection, no?

diggan

4 hours ago

It says my car is supported and my car doesn't have any blind-spot detection, nor does the requirements list that as needed, so maybe it's optional but not required?

falcor84

4 hours ago

What is filming the driver used for? Can you disable that?

siliconc0w

an hour ago

Been following Openpilot for awhile, would totally use it if I had a supported car. Though these days most cars come with 'pretty good' ADAS, even 'hands free' in some situations, so I wonder how much it's "worth it" to DIY compared to factory default.

colesantiago

5 hours ago

I am not surprised that comma is still around.

Minimal VC funding, less than 100 employees, not outrageously increasing headcount each month, profitable and sells a product with good margins.

Not many startups do this anymore, they are just chasing funding every 3 months using OpenAI’s API, comma has their own models before the AI hype.

solarkraft

3 hours ago

It’s cool to see that it’s possible to innovate sustainably, in a niche. So refreshing to see something stick around rather than become a bubble with a billion dollar valuation that either takes over the world and enshittifies or implodes and takes the product with it.

Make business chill.

That said: The larger they get, the more regulator attention they’ll attract. If some government entity wanted to, they could probably easily kill them.

ModernMech

2 hours ago

I wonder why the website doesn't say "operating system" but instead calls it an "advanced driver assistance system"

drivingmenuts

5 hours ago

If I was in law enforcement, I’d be rubbing my hands in glee to get ahold of that saved video.

xipix

an hour ago

I'd be even more gleeful if I was in car insurance. "Openpilot's not covered, your insurance is invalidated."

torlok

3 hours ago

I always wondered who would pay $1000 for a device you have to babysit, just so that you can become an unpaid beta tester and data drone for SV millionaires. I guess all you have to do is tell people they're a part of a revolution?

FullGarden_S

4 hours ago

for a second, I though this was the ROS alternative I've been forever waiting for smh

guyfromfargo

3 hours ago

I’m surprised to see so many people disliking Open Pilot on HackerNews. I have one of these, and it’s a total game changer on long trips. I drove from Texas to California using my Comma 3 and I didn’t have to overtake it a single time on the interstate.

Sure you have to actively be alert your entire drive, but it’s still significantly better than actually doing the work of driving.