All Z-wave functionality suddenly died on my IQ Panel 2+ late this afternoon. This is a mesh with over 30 nodes that has been stable, and with no hardware or panel configuration actions. As all nodes show as “Malfunctioning”, it seems pretty clear that the panel/controller is the common denominator.
As far as I can tell and have tried, everything unrelated to Z-wave functionality on the panel appears to be functioning properly.
I have tried rebooting the panel, but no difference in behavior. If I go Z-wave Tests, it’s clear that the panel is unable to communicate with any nodes. If I look at the error counters of individual nodes after the reboot, all I see is failures incrementing. If I go to Panel Test and select Z-wave, the test here consistently fails.
This is really weird, the three ADC temp sensors seem to not be reporting a malfunction. Are those up to date in your ADC account? Is temperature still being reported?
No, I haven’t done any firmware update for some time. I have been running the latest available (2.5.1?) for quite some time now.
No power outages nor brownouts, or other environmental issues that I am aware of. I was actually working from home all day, and my first sign of trouble was when my wife returned from work in the late afternoon and opened doors that triggered certain light rules (which were successfully triggered at many earlier points in the day). These lights went offline/reported malfunction, and I could not reach them no matter what I did at the panel (or ADC app), so I eventually punted and rebooted the panel. At this point, it became clear that whatever was going on extended well passed just those initial ~3 lights.
Yes, I noticed that the RTSes were not explicitly reporting Malfunction, but suspected that this might simply because they were in sleep mode and would not normally report such an issue, even if they were affected. They are otherwise active devices. The one in the basement is not physically mounted to anything, so maybe I’ll just bring it upstairs where its warmer to see if I can catch it registering a temperature change…
Yeah you are right, I think since the RTS don’t respond to pings it’s a moot point, but may as well test if it is easy.
It sounds like your Z-wave radio did indeed just fail altogether. It is strange and extremely rare, and typically it is a software error that a reboot fixes.
I’ve sent a couple commands to the panel, reboot the radio, check the Z-wave network. Looks like still a malfunction on everything.
One thing you might try is to go to Settings > Advanced Settings > Power Down.
After powering it down in the software, unplug it, open and unplug the battery.
Then fully remove and reinsert the Z-wave module. It should be the middle of the three expansion modules. Make sure when reinstalled that it is seated firmly and the set screw is tightened down. Any luck?
Yep, doesn’t look like the RTSes are updating at all. I haven’t seen a temp change in the one I brought up from the basement. Additionally, if I look at the History of each of them at the panel, the very latest update between the three was a bit before 1pm yesterday, a few hours before it became apparent there was an issue.
I had tried powering down and pulling AC power, but not quite to the point of pulling off the panel and disconnecting battery or touching the Z-wave module. I’ll try this next and report back.
Is there any way that the Z-wave radio might have somehow gotten disabled somewhere in the panel’s software/configuration that I could check (not sure if this is even configurable)?
I tried “tickling” the Z-wave toggle, but that didn’t help. So, I shut down the panel, disconnected it from AC power, disconnected the battery, and the proceeded to disconnect/reconnect the Z-wave module. The Z-wave module appeared to have been securely attached, but I disconnected / reconnected it anyway, then reinstalled the set screw.
I gather that we’re looking at a hardware problem, from the sounds of things… What comes next?
Yes, it sounds like a hardware issue, though the latest firmware versions for Qolsys have had some Z-wave issues and a patch is being rolled out soon. I doubt this issue is related, it is more likely the radio failed.
We are working with Qolsys and seeing if there are any additional potential troubleshooting steps or known issues, but my guess is it will come down to warranty replacement. If that is the result from Qolsys, we will get an RMA number and instructions to you ASAP.
Thanks, Jason, I truly appreciate whatever can be done to try and expedite turning this around quickly. For what it’s worth, I’ll be at home Friday and can easily pursue any further testing or requests for info. Thank you!!
I just tried that a couple different times with the same device. Both times it error’ed out pretty quickly with:
“Unable to process request: Operation failed. Please ensure your device is cleared and in range before including in the network. Please try again - Status code: 514”.
Wait, did the Clear work? That error response sounds like what you would see when adding. Did the panel actually recognize and clear a device when you used Clear Device and pressed the device learn button?
I really don’t think so. The Leviton light switch LED flashed red rather than green, which I recall indicates an error. I still see the device listed on Alarm.com, as well as on the panel’s list of devices.
I went ahead and tried re-adding it, anyway, and got the same red LED pattern on the switch. The panel looks like it is just waiting to time out on the device inclusion at the moment.
No luck, I’m afraid. I completely powered down the HEM, then powered down the panel, kept it off for at least 5 minutes, then powered it back up and tried to turn on lights individually, as well as by entire groupings from both ADC app, as well as via panel.
Alright, that is probably a hardware issue with the radio then. It may be firmware related, and I know a new version is supposed to drop soon which fixes some other Z-wave issues.
I’m checking with Qolsys for availability of the Z-wave daughtercard by itself and whether this would be possible. It looks like this panel may be outside of the standard warranty period, (purchased April 2019, is that right?) so I am verifying with Qolsys what all options we have.
Actually good news, looks like Qolsys can go ahead an do full warranty since it is within 2 years of production. No worries there. (FYI Z-wave daughtercard is just not available by itself regardless)
We’ll get this RMA process started. I apologize I might not be able to get an RMA number until Monday.