What do we do with a company that regularly pumps metric tons of virtual toxic sludge onto the Internet and yet refuses to clean up their act? If ever there were a technology giant that deserved to be named and shamed for polluting the Web, it is Xiongmai — a Chinese maker of electronic parts that power a huge percentage of cheap digital video recorders (DVRs) and Internet-connected security cameras.
In late 2016, the world witnessed the sheer disruptive power of Mirai, a powerful botnet strain fueled by Internet of Things (IoT) devices like DVRs and IP cameras that were put online with factory-default passwords and other poor security settings.
Security experts soon discovered that a majority of Mirai-infected devices were chiefly composed of components made by Xiongmai (a.k.a. Hangzhou Xiongmai Technology Co., Ltd.) and a handful of other Chinese tech firms that seemed to have a history of placing product market share and price above security.
Since then, two of those firms — Huawei and Dahua — have taken steps to increase the security of their IoT products out-of-the-box. But Xiongmai — despite repeated warnings from researchers about deep-seated vulnerabilities in its hardware — has continued to ignore such warnings and to ship massively insecure hardware and software for use in products that are white-labeled and sold by more than 100 third-party vendors.
On Tuesday, Austrian security firm SEC Consult released the results of extensive research into multiple, lingering and serious security holes in Xiongmai’s hardware.
SEC Consult said it began the process of working with Xiongmai on these problems back in March 2018, but that it finally published its research after it became clear that Xiongmai wasn’t going to address any of the problems.
“Although Xiongmai had seven months notice, they have not fixed any of the issues,” the researchers wrote in a blog post published today. “The conversation with them over the past months has shown that security is just not a priority to them at all.”
PROBLEM TO PROBLEM
A core part of the problem is the peer-to-peer (P2P) communications component called “XMEye” that ships with all Xiongmai devices and automatically connects them to a cloud network run by Xiongmai. The P2P feature is designed so that consumers can access their DVRs or security cameras remotely anywhere in the world and without having to configure anything.
To access a Xiongmai device via the P2P network, one must know the Unique ID (UID) assigned to each device. The UID is essentially derived in an easily reproducible way using the device’s built-in MAC address (a string of numbers and letters, such as 68ab8124db83c8db).
Electronics firms are assigned ranges of MAC address that they may use, but SEC Consult discovered that Xiongmai for some reason actually uses MAC address ranges assigned to a number of other companies, including tech giant Cisco Systems, German printing press maker Koenig & Bauer AG, and Swiss chemical analysis firm Metrohm AG.
SEC Consult learned that it was trivial to find Xiongmai devices simply by computing all possible ranges of UIDs for each range of MAC addresses, and then scanning Xiongmai’s public cloud for XMEye-enabled devices. Based on scanning just two percent of the available ranges, SEC Consult conservatively estimates there are around 9 million Xiongmai P2P devices online.
[For the record, KrebsOnSecurity has long advised buyers of IoT devices to avoid those advertise P2P capabilities for just this reason. The Xiongmai debacle is yet another example of why this remains solid advice].
BLANK TO BANK
While one still needs to provide a username and password to remotely access XMEye devices via this method, SEC Consult notes that the default password of the all-powerful administrative user (username “admin”) is blank (i.e, no password).
The admin account can be used to do anything to the device, such as changing its settings or uploading software — including malware like Mirai. And because users are not required to set a secure password in the initial setup phase, it is likely that a large number of devices are accessible via these default credentials.
Even if a customer has changed the default admin password, SEC Consult discovered there is an undocumented user with the name “default,” whose password is “tluafed” (default in reverse). While this user account can’t change system settings, it is still able to view any video streams.
Normally, hardware devices are secured against unauthorized software updates by requiring that any new software pushed to the devices be digitally signed with a secret cryptographic key that is held only by the hardware or software maker. However, XMEye-enabled devices have no such protections.
In fact, the researchers found it was trivial to set up a system that mimics the XMEye cloud and push malicious firmware updates to any device. Worse still, unlike with the Mirai malware — which gets permanently wiped from memory when an infected device powers off or is rebooted — the update method devised by SEC Consult makes it so that any software uploaded survives a reboot.
CAN XIONGMAI REALLY BE THAT BAD?
In the wake of the Mirai botnet’s emergence in 2016 and the subsequent record denial-of-service attacks that brought down chunks of the Internet at a time (including this Web site and my DDoS protection provider at times), multiple security firms said Xiongmai’s insecure products were a huge contributor to the problem.
Among the company’s strongest critics was New York City-based security firm Flashpoint, which pointed out that even basic security features built into Xiongmai’s hardware had completely failed at basic tasks.
For example, Flashpoint’s analysts discovered that the login page for a camera or DVR running Xiongmai hardware and software could be bypassed just by navigating to a page called “DVR.htm” prior to login.
Flashpoint’s researchers also found that any changes to passwords for various user accounts accessible via the Web administration page for Xiongmai products did nothing to change passwords for accounts that were hard-coded into these devices and accessible only via more obscure, command-line communications interfaces like Telnet and SSH.
Not long after Xiongmai was publicly shamed for failing to fix obvious security weaknesses that helped contribute to the spread of Mirai and related IoT botnets, Xiongmai lashed out at multiple security firms and journalists, promising to sue its critics for defamation (it never followed through on that threat, as far as I can tell).
At the same time, Xiongmai promised that it would be issuing a product recall on millions of devices to ensure they were not deployed with insecure settings and software. But according to Flashpoint’s Zach Wikholm, Xiongmai never followed through with the recall, either. Rather, it was all a way for the company to save face publicly and with its business partners.
“This company said they were going to do a product recall, but it looks like they never got around to it,” Wikholm said. “They were just trying to cover up and keep moving.”
Wikholm said Flashpoint discovered a number of additional glaring vulnerabilities in Xiongmai’s hardware and software that left them wide open to takeover by malicious hackers, and that several of those weaknesses still exist in the company’s core product line.
“We could have kept releasing our findings, but it just got really difficult to keep doing that because Xiongmai wouldn’t fix them and it would only make it easier for people to compromise these devices,” Wikholm said.
The Flashpoint analyst said he believes SEC Consult’s estimates of the number of vulnerable Xiongmai devices to be extremely conservative.
“Nine million devices sounds quite low because these guys hold 25 percent of the world’s DVR market,” to say nothing of the company’s share in the market for cheapo IP cameras, Wikholm said.
What’s more, he said, Xiongmai has turned a deaf ear to reports about dangerous security holes across its product lines principally because it doesn’t answer directly to customers who purchase the gear.
“The only reason they’ve maintained this level of [not caring] is they’ve been in this market for a long time and established very strong regional sales channels to dozens of third-party companies,” that ultimately rebrand Xiongmai’s products as their own, he said.
Also, the typical consumer of cheap electronics powered by Xiongmai’s kit don’t really care how easily these devices can be commandeered by cybercriminals, Wikholm observed.
“They just want a security system around their house or business that doesn’t cost an arm and leg, and Xiongmai is by far the biggest player in that space,” he said. “Most companies at least have some sort of incentive to make things better when faced with public pressure. But they don’t seem to have that drive.”
A PHANTOM MENACE
SEC Consult concluded its technical advisory about the security flaws by saying Xiongmai “does not provide any mitigations and hence it is recommended not to use any products associated with the XMeye P2P Cloud until all of the identified security issues have been fixed and a thorough security analysis has been performed by professionals.”
While this may sound easy enough, acting on that advice is difficult in practice because very few devices made with Xiongmai’s deeply flawed hardware and software advertise that fact on the label or product name. Rather, the components that Xiongmai makes are sold downstream to vendors who then use it in their own products and slap on a label with their own brand name.
How many vendors? It’s difficult to say for sure, but a search on the term XMEye via the e-commerce sites where Xiongmai’s white-labeled products typically are sold (Amazon, Aliexpress.com, Homedepot.com and Walmart) reveals more than 100 companies that you’ve probably never heard of which brand Xiongmai’s hardware and software as their own. That list is available here (PDF) and is also pasted at the conclusion of this post for the benefit of search engines.
SEC Consult’s technical advisory about their findings lists a number of indicators that system and network administrators can use to quickly determine whether any of these vulnerable P2P Xiongmai devices happen to be on your network.
For end users concerned about this, one way of fingerprinting Xiongmai devices is to search Amazon.com, aliexpress.com, walmart.com and other online merchants for the brand on the side of your device and the term “XMEye.” If you get a hit, chances are excellent you’ve got a device built on Xiongmai’s technology.
Another option: open a browser and navigate to the local Internet address of your device. If you have one of these devices on your local network, the login page should look like the one below:
Another giveaway on virtually all Xiongmai devices is pasting “http://IP/err.htm” into a browser address bar should display the following error message (where IP= the local IP address of the device):
According to SEC Consult, Xiongmai’s electronics and hardware make up the guts of IP cameras and DVRs marketed and sold under the company names below.
What’s most remarkable about many of the companies listed below is that about half of them don’t even have their own Web sites, and instead simply rely on direct-to-consumer product listings at Amazon.com or other e-commerce outlets. Among those that do sell Xiongmai’s products directly via the Web, very few of them seem to even offer secure (https://) Web sites.
SEC Consult’s blog post about their findings has more technical details, as does the security advisory they released today.
In response to questions about the SEC Consult reports, Xiongmai said it is now using a new encryption method to generate the UID for its XMEye devices, and will not longer be relying on MAC addresses.
Xiongmai also said users will be asked to change a devices default username and password when they use the XMEye Internet Explorer plugin or mobile app. The company also said it had removed the “default” account in firmware versions after August 2018. It also disputed SEC Consult’s claims that it doesn’t encrypt traffic handled by the devices.
In response to criticism that any settings changed by the user in the Web interface will not affect user accounts that are only accessible via telnet, Xiongmai said it was getting ready to delete telnet completely from its devices “soon.”
KrebsOnSecurity is unable to validate the veracity of Xiongmai’s claims, but it should be noted that this company has made a number of such claims and promises in the past that never materialized.
Johannes Greil, head of SEC Consult Vulnerability Lab, said as far as he could tell none of the proclaimed fixes have materialized.
“We are looking forward for Xiongmai to fix the vulnerabilities for new devices as well as all devices in the field,” Greil said.
Here’s the current list of companies that white label Xiongmai’s insecure products, according to SEC Consult:
WNK Security Technology
Update, 3:44 p.m.: Updated story to include Xiongmai’s statement.