Zigbee vs. Matter over Thread:Understanding IoT Protocol Performance in Practice

95 pointsposted 11 hours ago
by teleforce
(arxiv.org)

79 Comments

cpressland

4 hours ago

I’m in the process of migrating from Zigbee (Z2M) to Matter over Thread (Apple HomePods + Matter Server).

My Zigbee network was a mishmash of different vendors all of which had their own failure rates and quirks, Aqara having by far the highest failure rate and complexity.

My new Ikea (Thread) based setup has had absolutely zero issues so far and fingers crossed it doesn’t start suffering as I scale it out. That said, it is noticeably slower than Zigbee, but we’re talking 50ms > 250ms. Which is more than acceptable.

solarkraft

an hour ago

> That said, it is noticeably slower than Zigbee, but we’re talking 50ms > 250ms. Which is more than acceptable.

I’m taking your “but” to mean “it is acceptable.

I would at least deem it noticeable to wait a quarter second for my light to turn on. May just be my own impatience, but I get an insta-headache whenever I have to wait to find out whether something worked or I have to try again. Every bit of jank is a downgrade in quality of life to me.

Now: May this be due to that specific setup or is the delay somehow inherent to the technology?

tetris11

4 hours ago

What's to stop Ikea from pushing an OTA and crippling your whole setup 2 years from now?

HerAnt

4 hours ago

You don't need to update your devices, and they don't need to be connected to any cloud. That's the whole point of these local communication protocols, after all.

AnonymousPlanet

3 hours ago

Sorry to pop your bubble, but the moment you add any IP connectivity it will be eventually used to mandate internet access. The devices you have now might not yet demand it, but you will have to replace them at some point or extend your setup in the future. We've seen this play out so many times now, it's baffling people still think this time will be different.

There's a saying: If you set up a trough, the pigs will come.

pshirshov

an hour ago

I run 100-device zigbee network with zigbee2mqtt. Philips, Bosch, Sonoff and some others. No internet access.

Zigbee devices are slow and low power, they cannot access internet directly.

close04

3 hours ago

For IPv6 networks you're in luck. The hubs are not proxies so the IPv6 enabled IoT device in an IPv6 network can still be blocked at firewall level from accessing the internet. Less that the separation Zigbee provides but better than nothing.

For IPv4 networks you're out of luck. The internal mesh traffic is still IPv6 to the hub and anything crossing into the IPv4 network is done through the hub's IP. No more blocking available unless it's a feature of the hub. I'm not aware of any hub having this kind of FW capability.

jnovacho

an hour ago

Firewall blocking is not the point. The point is, that over time through enshitiffication the new device will require call home to unlock. Even when it does not need internet to work at all. But through this the vendor will get a foothold.

Sure you can block it, but then you have a pricey paperweight.

This is not unique to Thread/Matter, they can pull the rug on ZigBee as well, ofc.

homeonthemtn

24 minutes ago

The assumption of enshitification it that is receives updates. What you and a few others aren't understanding is that these don't need updates. These aren't Alexa or Google devices. They just get setup and passively work

Now, will they eventually build devices that don't work like that? Maybe.

treffer

2 hours ago

That they would never see me buying again from them. That I can stop OTA. That matter is interoperable and I can run multiple gateways before cutting them out.

Most IoT stuff I own had a good track record. The only stuff that screwed me over was the logitech harmony smart remote.

But I usually pick the more open ecosystems and vendors. E.g.heating is on shelly.

darkwater

an hour ago

If we start from GP premises where they were multi-vendor on ZigBee and now (allegedly) are living a much better mono-vendor life with Matter, you cannot play the multi-vendors card. I'm on ZigBee multi-vendor orchestrated by Zigbee2MQTT and while some devices are quirky sometimes (worse offender... IKEA) I'm not planning to move over to Matter any time soon at all.

XiS

4 hours ago

`establishing it as a more robust foundation for larger and more heterogeneous deployments.`

Based on.... 6 devices?

Quite honestly I would have been interested in measurements with more than only 6 devices. Maybe upto 50 or so,even better if they'd mixed different hardware nodes for a more realistic scenario

8fingerlouie

19 minutes ago

I have 47 thread/matter devices, alongside some 10-15 matter over wifi devices, so not quite 50, but "close enough".

It's not as fast as Zigbee, not in day to day usage, and not in terms of reconnecting, which often takes 30 seconds (ie. after a power outage) where my Zigbee devices just seem to reconnect instantly. On day to day usage there's a slight delay when turning something on, but it's maybe 100ms or less.

I initially had quite a few problems, but after "standardizing" on a single thread network (I had 3 before for some reason), as well as adding some wired thread border routers, things have become much more stable.

My zigbee to thread is still a "work in progress", replacing devices as they fail (ie. lightbulbs), and I'm also still adding new Zigbee devices like the Aqara T1 Valve controller, but I prefer thread over zigbee today.

One thing I will note, people tend to underestimate the range of thread (and zigbee for that matter). Each mains powered device will function as a repeater just like zigbee, and I initially thought I might need a couple of smart plugs as repeaters, but despite having a somewhat large house (by european standards) as well as double brick walls dividing it, my thread devices have no problems "seeing" a thread border router in the opposite end of the house.

8-prime

4 hours ago

Absolutely. My setup which is about 1 month old already has 14 devices and I'm just getting started. Only having 6 devices really feels comically small for a study investigating the performance of communication protocols where device count can really make a differnce, as observed by even then changes from 1 to 6 devices.

timvdalen

2 hours ago

I'm running both now, and while my Matter network seems to be solid, I can't get over the fact that it's so much harder to set up devices than on Zigbee. Why does Google need to get involved for me to set up a local IKEA device with my local border router?

SEJeff

10 hours ago

I’m a bit surprised they didn’t add zwave into this comparison.

margalabargala

6 hours ago

They're probably only comparing open protocols. If one doesn't care if their protocol is proprietary then sure one might consider zwave. But it's hardly "surprising" that it's left out.

brianaker

5 hours ago

The funny part is that people think that Zigbee and Matter are open :)

All three of them have their shortcomings. Matter requires an unsecured IPv6 network to operate; I have my doubts about it having any longterm future.

preisschild

3 hours ago

> The funny part is that people think that Zigbee and Matter are open :)

How is Matter (APL2) over OpenThread (BSD-3-Clause) not "open"?

> Matter requires an unsecured IPv6 network to operate

How is this network "unsecured" exactly? Thread IS encrypted and authenticated.

SEJeff

5 hours ago

What? The entire specification is in this git repo maintained by the home assistant team. Also, they’re on the zwave board now. It’s wholly open, but wasn’t maybe 4-5 years ago.

https://github.com/zwave-js/specs

waste_monk

5 hours ago

I'd like to use z-wave but as an Australian we have different allowed frequency bands (specifically the USA-market gear uses frequencies that are reserved for telecom use over here, and anything that can interfere with the ability to call emergency services is Very Illegal), and thus the few devices that get built supporting the AU rf bands are overly expensive due to the (lack of) economy of scale.

As a result my home automation stack is a mix of zigbee, Matter (over both Wifi and Thread), and just plain WiFi. All of which uses the 2.4Ghz (and 5Ghz but mostly 2.4 for the IoT stuff) ISM band(s) and is thus cheap and plentiful.

Not sure how many other regions suffer the same issues, but I suspect that z-wave will be outcompeted by the more modern alternatives in the coming years.

/rant

walrus01

8 hours ago

yes, and also that if someone is buying stuff new, it should be exclusively zwave800 stuff. example hardware on the "hub" end would be:

https://www.home-assistant.io/connect/zwa-2/

https://www.getzooz.com/zooz-zst39-z-wave-long-range-usb-sti...

the zooz is this internally: https://www.silabs.com/wireless/z-wave/800-series-modem-soc/...

mrheosuper

7 hours ago

another protocol/wireless stack? Why can't we settle down to something that is at least "good enough"?

margalabargala

6 hours ago

Because zwave is proprietary. There's money to be made here! Can you imagine the lost profit potential if people standardize on an open protocol like ZigBee?

cryptoegorophy

7 hours ago

Do you really need long antenna? I noticed zwave can act as a mesh.

SuperMouse

6 hours ago

Most likely it's not a super robust transmission mode like LoRa (that works down to -145dBm). Having a well designed antenna with a good pattern help a lot then.

asveikau

9 hours ago

Doesn't zwave use a different frequency, and zigbee and thread are both 2.4ghz? Perhaps that makes it a more fair comparison?

SEJeff

8 hours ago

Yes, zwave is in the much less congested 900mhz range. It is a fair bit more reliable as it doesn’t compete with:

- WiFi

- Bluetooth

- microwave ovens

Additionally, it’s a bit more strict with certification so you can easily use multiple vendors for zwave whereas with zigbee it’s often a terrible experience since vendors implement things differently in ways that are not as interoperable. In fact, for a device to be certified by the zwave alliance, it must pass strict conformance testing to ensure backwards compatibility and interoperability.

torginus

5 hours ago

This is both good and bad, and no surprise its an American standard. Lower frequencies have higher range, which is good if you live in a house (because range), and bad if you live in an apartment (because interference and crowded channels). So it will compete with itself, and everything on the ISM band (which you have no control over).

parineum

7 hours ago

All it costs is significantly more money and less options.

If you limit your pool of zigbee devices to just more reputable companies, you'll end up with around the same amount of options as zwave and they'll be reliable but significantly cheaper.

You also have the option to find some super cheap stuff that works just fine. That usually requires some research but not much. You can also build ESPHome devices for zigbee but that's pretty niche, to say the least.

Having previously been all zwave then moved and tried to go matter but ended up with zigbee, I wish I never listened to people saying what you are. As far as I'm concerned, zwave doesn't know it's dead yet and Matter isn't going to fulfill it's promises because the corporate interests that wouldn't give you local control with zigbee aren't suddenly going to change their minds because it says matter now.

wao0uuno

5 hours ago

My entire “smart” home setup is built with Matter over Thread. It’s fully local. No internet access required. With Home Assistant onboarding can be done without a smartphone now. Devices without official certification work but probably only in Home Assistant.

iamacyborg

5 hours ago

My Matter over Thread stuff keeps occasionally falling over in Home Assistant and becoming unavailable, probably a router issue as I’m using a homepod mini but still endlessly frustrating.

wao0uuno

5 hours ago

I’m using ZBT-2 with HAOS on RPi 4 4GB. It’s rock solid in a crowded apartment complex. Devices are 90% IKEA and 10% Aqara. Definitely try that antenna and don’t connect additional border routers. Let HA own everything.

iamacyborg

4 hours ago

My whole setup is, suboptimal, to put it delicately. It’s just running on a NUC that’s crammed inside my AV cabinet, antennas would be crammed in there as well. I mostly buy zigbee stuff but it’s hard to ignore how cheap the Ikea Matter over Thread kit is.

torginus

5 hours ago

Dunno, personally I have found HA + MoT to be strictly worse than Z2M. It's the classic case of the Douglas Adams quote:

“The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at or repair."

HA MoT has a simplified UI, that hides hellish complexity underneath, that's held together by loose pieces of gum, which means its difficult to troubleshoot. HA has a rather laissez faire policy to stability, for example, they rolled out an update just recently that made it impossible to add new devices.

I just readded yday evening my Ikea temp sensors as they mysteriously stopped working for no reason. Why? Who knows, but you can't ship stuff like this to a non-technical audience, especially if they rely on it. I had it connected as the temp sensor of my AC, and was surprised to find the AC was not working, thats how I found out it suicided itself.

wao0uuno

4 hours ago

I never had any problems with Thread that couldn’t be fixed by pulling batteries from a crashed Ikea device and I have very little interest in how the protocol works.

Automatic updates in Home Assistant are disabled by default for a reason. It’s not an ultra polished, safe corporate environment for non technical wives and children.

Which sensor? Battery powered temp/humidity sensor or USB-C powered air quality monitor? I’m asking because the battery powered one is known to be bad. They disconnect and refuse to connect again until their batteries are pulled for ~10 seconds. It’s a device problem, not Thread problem.

BHSPitMonkey

6 hours ago

But Matter _is_ locally-controlled. Matter-over-Thread is better as there's no side channels to abuse; Matter-over-Wifi also guarantees local control though the presence of Wifi means a high probability of some additional onboard software bloat the manufacturer can use to enshittify the product if you're not careful.

Tor3

5 hours ago

Zwave frequencies, as well as regulations, vary between countries/regions. This means that equipment isn't interchangeable, something I'm currently running into - I can't get a zwave electronic door lock for Japan while I could easily have bought one in my home country. But I wouldn't be able to bring it if I wanted to.

Zigbee is basically the same everywhere, however the 2.4GHz band is of course somewhat congested.

unsnap_biceps

8 hours ago

some zigbee controllers even support being converted to thread via a firmware update

tehlike

5 hours ago

smlights do that! they are great.

vdfs

11 hours ago

These protocols add so much complications it makes wifi/tcp/ip a blaze

They fix some problems but add more, you endup just like that xckd about protocols

Forge36

10 hours ago

Having switched to zigbee devices recently I regret trying to start with WiFi. While I'm not sure how it looks under the hood, adding devices with home assistant was miles easier.

baby_souffle

10 hours ago

I've helped lots of people with home automation Network build outs l; the biggest factor is the RF environment.

If you're fortunate enough to not be battery constrained at most endpoints and can put a high quality Wi-Fi access point in an optimal location than Wi-Fi is incredibly reliable.

The tooling for inspecting all layers of a Wi-Fi network is infinitely more accessible and mature and that certainly helps; I can't go to My local big box store and get a USB thread dongle for 20 bucks that will just work on a Lennox computer and integrate nicely with Wireshark out of the box...

hiitsmyaccount

9 hours ago

I have a Unifi home network with two Wifi 7 access points and gigabit ethernet. For home automation, I went with ZWave and do not regret it at all. It's been rock solid and I've never felt the need to get a dongle to debug the traffic. I don't want to be working through all of the WiFi baggage with my home automation devices. I recently introduced some Zigbee bulbs (hue and cheapos from Amazon) and they've been mostly "fine", albeit with lower range.

Gigachad

10 hours ago

Thread is fairly similar to Zigbee, just standardized and doesn't require brand specific hubs. You can actually control thread devices direct from an iphone with no hub at all.

Wifi for anything but cameras is a mistake, it only exists for maximum compatibility since everyone has wifi already.

esseph

9 hours ago

Wifi for cameras is arguably also a mistake

nomel

9 hours ago

Cameras that require internet to connect to are a mistake, but cameras that use Wifi for local link are fine (wifi is perfect for this). And, having some way to connect remotely is a requirement, for most everyone.

rob-olmos

9 hours ago

I took their point to be about wifi being jammable

fragmede

9 hours ago

Yeah first thing burglars are gonna do is jam your wifi camera so you don't have any video of them. Run cat-5 and get a PoE camera.

kps

9 hours ago

And be sure your outdoor cameras are on an isolated VLAN so people can't just plug in to your home network.

inigyou

3 hours ago

I read this as sarcastic but it's a thing. Simplisafe, for example, is trivially jammed.

mlrtime

an hour ago

Yeah, not sure why people are downvoting you. I suspect because most people run wifi cameras and don't see the reason to run a ethernet cable.

If you are halfway serious about your home security, you should use ethernet. It's trivial for thieves to jam wifi now.

wao0uuno

5 hours ago

Security system that can be jammed is a terrible idea. Cameras should be wired. Ideally on separate network and with battery backup.

zer00eyz

9 hours ago

> Wifi for anything but cameras

Buttons (key presses, remotes - think rapid input), Knobs, sliders, Pixel LED' strips, some MM wave sensors, ir/radio transmitters, really need wifi.

There is products where you will want a bluetooth proxy, plenty of kitchen/grill thermometers and scales for kitchen and bathroom are great examples.

Where range is an issue LoRa is a thing that has plenty of devices and is easy to DIY.

unsnap_biceps

8 hours ago

My matter over thread devices (inovelli) for light switches work faster and better then my old wifi devices (cync)

torginus

5 hours ago

A bit of clarification: Thread is a layer 2 protocol, for the user, it looks like standard IPv6. Matter is application level, and just expect standard TCP/IP underneath. There's nothing binding Matter to Thread, and in fact a lot of Matter devices use Wifi

amarcheschi

6 hours ago

I studied zigbee protocol for an exam and while everything is there for a reason, it truly feels diabolical

preisschild

3 hours ago

Thats exactly what Matter over Thread solves. It just uses standard ipv6, udp, tcp.

WiFi is not designed for lots of low energy devices that can be repeaters.

deadmutex

10 hours ago

Some specifics would be a great addition to this comment.

jauntywundrkind

11 hours ago

A lot of graphs where OpenThread & zigbee dance around each other, are within 2x of each other.

Two huge notables to me: message throughput for openthreads scales up, zigbee seems to hit a wall early. Not sure when that matters but very clear. The last table was a fright through: it takes half a minute for OpenThreads to recover after a node drops where-as Zigbee a quarter second! Wow.

richwater

9 hours ago

Is there a situation in which your smarthome taking 30s to recover from a node failure is impacting your life? Seems like a distinction without a difference

denkmoon

5 hours ago

Do you have any smarthome stuff? Standing around waiting 30 seconds for your light to come on would be moronic. A complete regression over the electric switch invented in 1884. I'd rip everything out right then and there. Your home should be reliable, smart or no.

diffeomorphism

2 hours ago

Buttons etc. have direct binding. So even if your hub goes up in flames those will just keep working.

The automated stuff like "starting at sunrise gradually adjust the lights" might be delayed by 30 seconds. Annoying but hardly rage inducing.

8-prime

3 hours ago

Whilst I agree a smart home offers some benefits that your 1884 switch simply cannot match. And the cruicial point is that the 30 seconds are the worst case situation where something went wrong on the network. Its not the usual operational experience you'll get.

For me, having to wait 30 seconds once every blue moon is a tradeoff I'm willing to make.

4chandaily

9 hours ago

Standing outside with an armfull of groceries, and a door is now our of range of your hub temporarily due to the failed node. 30s can seem like quite a while standing in the rain.

AlotOfReading

8 hours ago

Not exactly a strong argument if you need like 5 qualifications to demonstrate the issue. If the right node fails, if it happens right as you walk up, if it's raining, if your porch is uncovered, if your door doesn't have an immediate fallback... I'm sure someone will be affected because of the law of large numbers, but it seems like an unlikely scenario to base a purchasing decision on.

codeflo

5 hours ago

A light switch not working for 30 seconds, or a door not opening for 30 seconds, even if only sporadically, is a significant loss of life quality, no qualifiers needed.

NekkoDroid

3 hours ago

> A light switch not working for 30 seconds, or a door not opening for 30 seconds

I dunno if its just me, but having a manual override for such important/basic stuff is kind of a must for me. As much as I enjoy smart home stuff, having anything critical only available via that is just stupid.

solarkraft

an hour ago

Oh, absolutely. That is the hot path.

alexjurkiewicz

7 hours ago

Because IOT devices are cheap and networks are ad hoc, flaky connectivity is a likely occurrence. Thread must be reliable in these situations.

BrandoElFollito

3 hours ago

I have tons of devices and I wish everything was Wi-Fi.

Wi-Fi has its problems but at least it is easily measurable, debugable, ... Zigbee is a voodoo protocol: you initiate the pairing and hope for the best. And then you hope for the best to have it stay on.