The hidden JTAG in a Qualcomm/Snapdragon device’s USB port

231 pointsposted a day ago
by denysvitali
(linaro.org)

36 Comments

mmastrac

a day ago

This is a much better experience than the previous Qualcomm debug experience, which was a hand-rolled set of read/write/execute primitives exposed over USB. It was hilariously undersecured, allowing a few of us to continually get root on various Qualcomm models.

In seriousness, these debug ports are seriously lacking in most mobile chipsets. MediaTek still has the old-style approach in many of their devices, requiring some incantations which expose serial over USB, but not in the way you think -- it's serial over USB pins!

I've done tonnes of work with mobile chipsets and security and this seems like they've finally started down the road to making this functionality accessible. Don't be surprised if you don't see this supported out of the box in most places, though. Most OEMs will certainly disable this once they've adapted their bootloaders to it. The big G doesn't like debuggability in end user devices.

IAmLiterallyAB

a day ago

Google exposes serial Serial over the SBU pins on all the Pixel devices

twojacobtwo

a day ago

What are the effective implications of this?

bri3d

a day ago

It's just a UART; you can use the UART to debug the device in various ways.

On Pixel devices, the UART is not configured or brought up by default in locked production mode (as things should be), but by unlocking the device and then using `fastboot oem uart enable` you can flip the bits to turn it on. On early Pixel devices it was on the headphone jack and on newer ones it's on the SBU pins.

By default I think it's still configured as the kernel console in the kernel command line, so once it's enabled it will show the kernel debug output and present a TTY. But of course you can subsequently configure it to do whatever you'd want a UART for: kgdb for kernel-debugging, earlier stuff in the bootloader, and so on.

So, the implications are just: there's a convenient debugging interface available to you that turns on if you unlock the device and ask for it.

On Chromebook devices there's a more complicated and fancy debugging system where the SBU pins can be muxed to the security processor's USB host interface by presenting a debug cable called a SuzyQ, which presents a whole suite of debugging facilities. This used to be used quite frequently for unbricking purposes.

Tharre

a day ago

On the newer pixel phones (starting with the ones containing the titan chip) you can also mux the SBU pins to the security chip USB interface with "fastboot oem citadel suzyq".

And BTW, the SuzyQ cable is nothing more then two pull up resistors and a USB hub connected to the normal usb D+/D- pins on one port and the SBU pins on the 2nd port. Nothing fancy about it, people have even made their own (minus the hub) by soldering some wires and resistors to a usb-c breakout board. Google has also published the schematics for it:

https://www.chromium.org/chromium-os/ccd/951-00273-01_201806...

Veserv

a day ago

Most of those boards have a separate physical JTAG connector (at least in development kits, this article indicates JTAG over USB is disabled in production systems anyways so no difference there) which is what they are expecting you to use for low-level debugging. It only costs like 1,000 $ for a JTAG probe which is like 1 fully-burdened engineer-day of cost. Even fully featured probes enabling hardware trace and time-travel debugging only cost like 1 engineer-week.

bri3d

a day ago

> Most of those boards have a separate physical JTAG connector (at least in development kits, this article indicates JTAG over USB is disabled in production systems anyways so no difference there

There's generally an entire phase of prototyping where engineers will be using production boards but still need JTAG, which is why it's fused and why these kinds of features exist. It's a lot easier to have your lower-level software team (drivers/BSP, perf, etc.) sitting with production-ready units provisioned with engineering keys and debug enabled than to have them having to use some kind of case-off JTAG header setup, cost aside.

AlotOfReading

a day ago

The probes cost enough to exceed individual purchasing limits at hardware companies, which means you need to go through the requisition process. That takes long enough that you have to plan ahead and you don't order more as your needs increase. Then everyone's fighting for the limited probes right before a ship date and they get jealously guarded like priceless jewels.

JTAG also isn't usually exposed through enclosures, so using the probe on a field unit might require destructive entry depending on the application.

Veserv

a day ago

Well the problem there is companies who are too stupid to invest in cheap tooling with massive ROI for their developers. A pretty constant problem in software development.

And I am not knocking JTAG over USB. It is certainly convenient and beneficial since you can enable it in production or deployed units. I was commenting on how the GP (and even article) was making it out to be missing capability. They just do not have the cheap tools that are the intended way to access that capability.

edit: The article even mentions how the "Qualcomm Landing Team at Linaro", which seems to be the team that works with pre-production hardware to get them working on launch day, has a development process where "debuggers have never been a staple of our work for all the typical reasons you'd expect (cost and complexity being the main ones)". That is literally the team that should have pre-production units in the lab which will have debug connectors and where JTAG probes should be par for the course, yet they are apparently hardly using them partly because of "cost".

nrclark

10 hours ago

The landing team's job is to "land" patches into upstream. They take Qualcomm code and spin straw into gold until it's eventually good enough to contribute to projects like the Linux kernel.

Having read a lot of Qualcomm code myself, I don't envy their job.

phkahler

12 hours ago

There's something wrong if you're using Jtag on a field unit. In my experience, once a chip is up and running we implement higher level communication and debug methods.

AlotOfReading

4 hours ago

To give an automotive example, a field unit may fail and be RMA'd to the tier 1 to improve their production line. They don't have access/documentation for the production software, and it wasn't developed to interact with their diagnostic tooling. They'll need to reliably reflash their own test software onto a board with potential hardware failures. JTAG is sometimes what they request to do that.

Hizonner

a day ago

> this article indicates JTAG over USB is disabled in production systems anyways

Well, should be. I bet there've been screwups now and then...

mrheosuper

19 hours ago

Nope, Many phases you have to debug on real product (no access or hard to access to jtag/swd header).

rangestransform

20 hours ago

You bring me back to the olden days of QXDM in my adolescence, when I used it to save myself from bricking my phone, and again when I had my first internship working on an MDM9x50 device

immibis

11 hours ago

You say "undersecured", I say "allows people to own the things they bought".

jon-wood

10 hours ago

Sadly these two things can be true at the same time. As a hacker I love devices which (maybe accidentally) expose a way for me to run arbitrary software on them. At the same time as a person carrying a computer in my pocket containing my entire life I'm not a fan if that allows anyone grabbing said computer to run arbitrary software on it.

The ideal here is the standard approach where a user has to explicitly take some sort of action which can't be found accidentally and done without understanding the implications to activate debug mode. On Android that's tapping a specific item in the about menu way more times than you might think would be necessary. On iOS it's hooking up to a computer, then authorising the connection on the phone. Either way you get both the ability to run arbitrary(ish) software without someone picking up your device when left on a train being able to do so.

mystified5016

7 hours ago

Also there's "allow anything on the other side of this public 'charging port' own the things you bought"

userbinator

21 hours ago

MediaTek still has the old-style approach in many of their devices, requiring some incantations which expose serial over USB, but not in the way you think -- it's serial over USB pins!

Wasn't that only in their old feature-phone (MT62xx) SoCs? All the smartphone ones AFAIK (at least since the MT657x days) use USB CDC in their BootROM and preloader.

user

a day ago

[deleted]

mrheosuper

19 hours ago

at work we adapted to "usb c debug accessory mode", which should be a standard everyone should follow.

tripdout

a day ago

It will be really interesting to see what production devices this is enabled on - It mentions the OnePlus 6 at least which has it fused out but is still accessible.

Edit: How are they reading the eFuses on a production OnePlus 6? Do they have a Qualcomm-signed EL3 EDL loader?

It seems to exist as qcom,msm-eud in the device tree of a (unfortunately production) SM4350 device I have along with an eud_enable_reg. Time to recompile the kernel with `/dev/mem`.

tripdout

a day ago

Well, no luck.

In the device tree I see (snippet):

  qcom,msm-eud@1628000 {
   compatible = "qcom,msm-eud";
   interrupt-names = "eud_irq";
   interrupts = <0x00 0xbd 0x04>;
   reg = <0x1628000 0x2000 0x162a000 0x1000 0x3e5018 0x04>;
   reg-names = "eud_base", "eud_mode_mgr2", "eud_tcsr_check_reg";
   qcom,secure-eud-en;
   qcom,eud-tcsr-check-enable;
   status = "ok";
  };

  qusb@162b000 {
   compatible = "qcom,qusb2phy-v2";
   reg = <0x162b000 0x400 0x1b40268 0x04 0x162f014 0x04 0x162a000 0x04>;
   reg-names = "qusb_phy_base", "efuse_addr", "refgen_north_bg_reg_addr", "eud_enable_reg";
   qcom,efuse-bit-pos = <0x19>;
   qcom,efuse-num-bits = <0x03>;
but `devmem 0x162A000 4 0x1` causes the system to lock up and I see the following in ramoops:

    [  433.720232] msm_watchdog f410000.qcom,wdt: Causing a QCOM Apps Watchdog bite!
    [  433.727381] msm_watchdog f410000.qcom,wdt: Wdog - STS: 0xb01a6, CTL: 0x3, BARK TIME: 0x57fdf, BITE TIME: 0x6ffd6
I'm not at all sure on the interpretation of this, but the reading at the efuse_addr (so I guess certain ones can be read from EL0?) is 0x0e000000 which has bits 25-27 set and QFPROM fuses seem to have a blown value of 1 according to Qualcomm docs, so it might be fused out?

zorgmonkey

a day ago

yeah EDL loaders for a bunch of production devices exist here [0] also more on various XDA Forum posts for stuff like unbricking guides. It is worth noting for people who don't

[0]: https://github.com/bkerler/Loaders

tripdout

a day ago

But reading QFUSES specifically requires an EL3 loader "edl qfp qfp.bin -> To dump qfprom fuses (only on EL3 loaders)" and I don't believe most devices programmers (especially as relatively new as the OnePlus 6) run under that privilege level.

dazhbog

a day ago

So just to get this straight, Qualcomm has a piece of custom silicon, as a peripheral controlled by registers, that when enabled reroutes the ARMs USB pins through it (adding a USB hub in the middle), and on that hub it adds a SWD programmer and a serial port that connect back to the ARM core's IOs? Amazing!

ethan_smith

20 hours ago

This USB-based debug approach is becoming standard practice across SoC vendors (MediaTek's preloader, Apple's special cables) as it eliminates dedicated debug headers while maintaining controlled access through eFuse-gated authentication.

jeffreygoesto

13 hours ago

On some SoCs we could get to the ETM port and it was even parallel (what a nice waste of pins ;)), little down-clocking and you can get a full program trace out, i.e. with a Lauterbach. Can debug application together with the OS and all drivers down to single assembler instruction level.

jauntywundrkind

19 hours ago

Different topic, but I was crazy impressed to see Qualcomm's dedication on getting USB audio offload going, having the audio device forward data to the USB host controller, for it to send it to the USB audio device.

Feels like a weird thing to spend so much effort optimizing but neat as heck to see. https://www.phoronix.com/news/Linux-6.16-USB-Audio-Offload

londons_explore

12 hours ago

> This feature offers major power savings on embedded devices where a USB audio stream can continue to flow while the rest of the system is sleeping, something that devices running on battery power really care about.

How many shipping devices use USB for audio? I was under the impression that most phones/tablets/laptops support usb audio, but it is a niche/power user feature, and not the way the main device speakers are connected.

Do these same systems have bluetooth audio offload so the system can sleep for multiple minutes while a song is playing via bluetooth? (which seems to be a far more common usecase)

jauntywundrkind

8 hours ago

I admit to mystification myself at the work out in here for what feels like a pretty rare use case!

With 3.5mm stereo jacks gone, I do think there are some folks who use USB audio a lot now! But it also feels like it's gotta be a pretty small crowd! This sure feels like a long long way to go to give these select few a battery life boost! It'd be interesting to know how big the impact is, to at least weigh this long effort.

Like you, would love to know if there's anything cute folks have done for offloading Bluetooth! my gut says there are vastly vastly vastly more Bluetooth users.

tecleandor

7 hours ago

USB-C headphones for phones have their own audio interfaces, yep. Headsets and webcams have usb audio interfaces too.

Also, I guess there are a good bunch of devices that have audio interfaces that, internally, might be connected to the USB bus.

But I'm also thinking about sound "processing" oriented hardware that could run Linux. Sound recorders, mixers, synths, smart speakers... that could be based on an SBC or an ARM or similar processor.