AV-TEST Product Overview “Smart Locks”

Hardly any other product category in the Internet of Things makes the need for adequate security as obvious to everyone as smart locks – the direct consequence of neglected security and an exploitable vulnerability can jeopardize your own property and private security. So it’s no wonder that customers take a close look before making a purchase decision.

In the past, we have repeatedly tested products from this category and will continue to do so in the future. The majority of the most popular products in the “Smart Locks” category have also been awarded our certificate, giving customers the chance to recognize secure products directly from the label on the packaging.

In the following overview, we list all the products that we have already tested and will also carry out regular updates to include newly tested products, remove outdated tests or add new findings to products that have already been tested. We are also planning further product overviews for other relevant product categories.

Certified!

As already mentioned, over the years we have awarded our certificate to some of the most important representatives in the Smart Locks category for passing our extensive security tests. In terms of security, all of them are absolutely worthy guardians of your own front door. There are also no major differences in terms of data protection. And where there were any, they were generally rectified within a few weeks as a result of our test.

While some of the products have now reached the end of their life cycle or have simply not been recertified, the following representatives are currently still certified (with the date of the last test):

Netatmo

The latest new certification entry in the Smart Locks category is the product from French manufacturer Netatmo. With the Smart Door Lock and Keys combination, Netatmo presented its vision for a secure and interference-resistant smart locking system last year. As we were able to witness for ourselves, security was a top priority during development from the very beginning and ultimately culminated in an absolutely solidly secured Smart Lock, which provided no evidence of potential weak points in the test. We were also able to award top marks in the area of data protection: The privacy policy informs the user in an exemplary manner and data collection can also be objected too, simply and easily via the system itself.

In contrast to all the other Smart Locks we have tested so far, the Netatmo solution takes a slightly different approach to operation. Whereas virtually all competitors rely on motor-controlled operation of the locking cylinder, the Netatmo solution always has to be operated manually. The system therefore does not close and open the door directly, but merely releases or blocks the opening. This certainly only has advantages in terms of susceptibility to faults, but it could be an important factor for some interested customers. The fact that, in addition to classic control via a mobile application, the system can also be operated more or less like a normal lock using the smart keys supplied, which can be programmed and unprogrammed as required, is probably just as interesting.

Nuki

Nuki Smart Locks have been undergoing our annual certification test since the first version was released in 2017. Right from the start, we were able to observe the focus on security and data protection and certify this with our certificate. In the following years, the new, more advanced versions 2.0, 3.0 and, most recently, version 4.0 with Matter support were added, which also passed our certification process without any problems. In addition, the integrated version in the form of the Smart Door and the Opener peripheral product (not listed in the table) have also been awarded the “Approved IoT Product” rating.

Overall, the Nuki ecosystem is probably one of the most mature and widespread solutions in the smart lock sector, which is still reflected in the high level of customer popularity. The already high level of security and the continuous improvement and expansion of the products and product range also promise a high level of security for years to come.

tedee

Last year, the smart door lock from Polish manufacturer tedee was also certified for the third time, together with the tedee bridge required for remote control and the GO product variant, which is identical to the tedee lock in terms of security. From the very first test, the comparatively small smart lock impressed not only with its stylish, slim design, but also with its absolutely solid security concept. As with all other certified Smart Locks, communication via the Internet and local communication via Bluetooth is, of course, fully protected against the most common attacks. Even the mobile application is additionally protected against manipulation of the source code and, by implementing suitable integrity checks, prevents an attacker with access to the user’s smartphone from installing a modified version. In practice, this is certainly not a scenario that occurs all too frequently and the tedee app does not offer 100% protection against manipulation (we bypassed the signature checks in the test), but the fact that thought was given to this at all shows the dedication to security.

ABUS

As part of the new, digitalized ABUS One product range, the popular German manufacturer now has a new smart lock in its product portfolio, which we tested and certified in June 2024 – the Loxeris One.
This is a classic smart lock that can basically do everything you would expect in this product category. As usual, the Smart Lock is mounted on a conventional locking cylinder, which is motorized via an inserted key. As already mentioned, no special cylinder is required here, only the key used must be cut to the correct length before use.

Once the Smart Lock has been installed, it is configured, calibrated and controlled via a mobile application for Android and iOS, as usual. When used standalone, this control takes place exclusively via Bluetooth directly between the app and the device. If the user also values remote control via the Internet, Bridge One is also required, which enables and implements remote access to all devices in the ABUS One ecosystem.

In terms of security, there were no problems worth mentioning in the certification test: The mobile applications, and above all the security-relevant parts of these, are designed and implemented absolutely adequately. The particularly critical Bluetooth communication, which in the case of the ABUS One devices runs via a protocol specially developed by ABUS, provided no evidence of serious problems in the test and after a detailed review. Here, ABUS relies on strong authentication together with AES 128-bit encryption and thus adequately secures local communication. Communication via the Internet, between the bridge and the cloud and between the app and the cloud, also showed no reason for serious criticism in the test – here too, the security level can be classified as high.

When it comes to data protection, the ABUS solution leaves nothing to be desired. The privacy policy, which is available via the apps themselves and the app stores, provides everything that a good privacy policy should provide. The essential information on data collection, storage and sharing is included and explained in sufficient detail. The two Google trackers included (CrashLytics and Firebase) are mentioned and the user has the option to explicitly object to data collection and still use the app. However, even with data collection activated, the applications are quite data-efficient – no unnecessary or even excessive collection of user and usage data was observed.

The Contenders

As in virtually all popular Internet of Things product categories, new products are constantly coming onto the market in the smart lock sector, competing with the established giants. However, there are also products that have been on the market for a long time and enjoy great popularity but have not yet been examined in detail by us. We make our selection here largely on the basis of the general popularity of the products, i.e. based on sales figures and customer reviews, for example. Testing is carried out in our Quick Check format, which always provides a solid initial assessment of the level of security and data protection.

This section will also be updated and expanded as soon as new products have been subjected to our tests. The following table lists all tested products with test result and test date.

Homematic IP Door Lock Drive

Like the other competitors, the door lock drive for eQ-3 Homematic IP system also offers a convenient way to control a door lock remotely. This also requires a Homematic IP Access Point (HAP) or a central control unit (CCU). The central control unit can then be used to call up a WebUI to control the Homematic IP devices. In this case, we used the access point for our Quick Check. It is important to note that the Homematic IP door lock drive requires a cloud connection in order to be able to operate the device even in close proximity – operation via short-range radio such as Bluetooth or NFC is not possible with the Homematic IP device. This certainly has disadvantages for operation and availability, but of course it also reduces the attack surface in terms of security.

The eQ-3 device is also quite solid – we did not find any explicit weak points during our Quick Check, but we do have a few comments. For example, we have to criticize the Homematic IP app, which can be installed on outdated devices and Android versions down to version 6.0 of the Google OS. As we have mentioned before, we would definitely avoid this for security-critical applications.

Another note concerns the “Cleartext Traffic” setting, which basically allows the application to carry out unencrypted communication via HTTP instead of HTTPS. Although we were unable to detect any unencrypted communication in the test, we would also recommend changing the configuration here so that unencrypted communication is prevented from the system side.

Everything looks quite solid in terms of communication. We would only recommend integrating a time component into the communication for requests to the cloud in order to increase security and ensure increased robustness against replay attacks. Currently, only the AUTHTOKEN, PIN (if specified by the user), DeviceID and State parameters are taken into account.

With regard to data protection and the privacy policy, there are also only minor points to note: Overall, the information content of the declaration is absolutely sufficient and all important information on data collection, processing and sharing is included. Compared to the other two candidate Smart Locks, the very modest use of trackers is particularly positive. Only one single tracker can be reliably identified in the Homematic IP application: Google Firebase Analytics. According to our dynamic analysis, this is only active sporadically and is also correctly mentioned in the privacy policy. Furthermore, the eQ-3 solution is the only one of the three Smart Locks in Quick Check format that provides a German privacy policy directly in the Google Play Store. For products that are sold on the German market, this should of course be the minimum standard.

Last but not least, a few general points for an overview: The privacy policy comprises 2,056 words with an average of 19.77 words per sentence. It takes the user around 3.3 minutes to read the entire privacy policy. The privacy policy in the Google Play Store is displayed in German. The Flesch-Kincaid readability index was used to assess readability, which in our example indicates that a degree is required to understand the privacy policy.

Overall, the eQ-3 solution can therefore also be regarded as absolutely adequately secured. Here and there, the level of security could certainly be improved, especially in comparison to the top representatives of the “Smart Locks” product category, in order to rigorously eliminate even theoretical weak points. In terms of data protection, however, the solution is already exemplary and is rated 2 out of 3 stars according to our Quick Check format.

Lockin G30 Smart Lock

The G30 Smart Lock from Lockin is one of the relatively new representatives at the start – a classic smart lock that can control the door lock via Bluetooth and, with the included bridge, also via Wi-Fi. Operation directly in the immediate vicinity of the device and remote control of the door lock are therefore also possible with the G30, as usual. However, unlike the Nuki products, for example, an account is required to use the associated mobile application with the Lockin G30. From a security point of view, the applications look okay, but from a configuration point of view, we only noticed that the automatic update for the bridge is deactivated by default. It is really questionable why such an essential function is deactivated by default. The user should definitely activate the auto-update function here in order to benefit from security updates as quickly as possible.

Speaking of mobile apps: In our tests, it is relatively often necessary to make changes to the application code itself. There can be several reasons for this. For example, we may incorporate logging in order to understand certain functions more quickly or to be able to trace entire process chains more easily. We also repeatedly come across applications that try to prevent the app code from being manipulated. The tedee app is a previously mentioned example. However, the Lockin application tried to make life particularly difficult for us here. Not only were functions implemented here that were intended to prevent manipulation and dynamic analysis of the application, but there were also functions that only appeared to serve this purpose but were never actually called. These are either development relics or deliberate attempts to create false trails – we rarely see this in commercial products.

Apart from this, however, some functions are also implemented that actually serve to prevent manipulation and analysis, including reverse hooks that aim to detect dynamic analysis tools such as Frida, Xposed or Saurik (see following code snippet), key hash checks to ensure file integrity, detection of root access on rooted Android devices or emulator detection to determine if the app is running in a virtualized environment.

Of course, you can hardly accuse a developer of implementing too much security, but this unconditional prevention of analysis is suspicious, as it often indicates that an attempt is being made to conceal potentially inadequate security by preventing analysis – security by obscurity, so to speak. Apart from that, the whole thing is not particularly useful on Android – of course we were still able to carry out our analysis, although admittedly it was more complicated than usual.

During the dynamic analysis of the app, we noticed that opening via the app offers the option of repeating the same opening request several times in a replay. In the test, it was possible to replay the same request twice within a period of at least 10 minutes and thus successfully open the lock. When the request is sent for the fourth time, it is finally recognized as outdated. Improvements should definitely be made here. This potential vulnerability also seems particularly unnecessary, as a time component is already transmitted with the opening request anyway, but is apparently not checked. On a positive note, every opening (including those carried out via replay) is logged in the viewable log, allowing complete traceability.

As far as communication via Bluetooth is concerned, we were unable to identify any obvious weak points, at least during the Quick Check, even if there are indications of at least potential weak points. For example, the lock can only be operated via Bluetooth when there is an active connection to the Internet. With solid and secure Bluetooth communication, this should not really be necessary. We may investigate this further at a later date.

An additional note regarding the app is that it can be installed from Android version 6. We can of course understand that developers want to support as wide a range of devices as possible, but for a security-critical application we would advise against supporting such old versions. Android version 6, for example, has not received any security updates since 2018 and contains a number of critical vulnerabilities which, in the worst-case scenario, could also affect the installed applications.

Of course, we have also checked Lockin’s privacy policy. It contains most of the important information. During the analysis, however, we found two trackers in the code: Google Firebase Analytics and Pangle. These are not mentioned in any way in the privacy policy. Our dynamic analysis could not prove any clear activity of the trackers either, but it can be safely assumed that this takes place at least occasionally and then this should also be mentioned accordingly in the privacy policy. It should also be noted that for requests outside China, such as in the EU, a period of up to 30 days is granted for a response – customer patience needed here.

Last but not least, a few general points for an overview: The privacy policy comprises 3,263 words with an average of 29.4 words per sentence. It takes the user around 4.41 minutes to read the entire privacy policy. The privacy policy in the Google Play Store is displayed in English. The Flesch-Kincaid readability index was used to evaluate readability, which in our example indicates that a degree is required to understand the privacy policy.

Overall, the Lockin G30 is rated with 2 out of 3 possible stars according to our Quick Check format: In principle, the solution is quite solid, but there are cutbacks here and there when it comes to implementation and the privacy policy.

Yale Linus L2

Like all other solutions, the Linus Smart Lock from Yale promises simple and secure remote control of the door lock. Thanks to the cooperation with BOSCH, uncomplicated integration into the German manufacturer’s smart home and operation via the BOSCH applications is also possible without any problems. Without the appropriate hardware, the lock can either be operated via Bluetooth only or the additional Yale Connect Wi-Fi Bridge can be used, which then also offers use as a completely independent solution with its own mobile application and makes it accessible from anywhere via the Internet. However, an account is also required for this. It is also worth noting that two-factor authentication (via SMS and email) is available, which can further increase the level of security here if desired.

Our Quick Check revealed that it is a solid product. The only note from the static analysis is that the app can be installed for Android versions 6.0 and higher – as already mentioned, these old Android versions no longer receive security updates and should therefore no longer be used for security-critical applications, even if this means that the system can no longer be operated with older smartphones.

The communication itself is absolutely adequately secured. Messages are protected against manipulation and unauthorized access using AES encryption. A secret remoteOperateSecret is used for this purpose and the transmission time is also encrypted so that replay attacks can also be effectively prevented here.

Here too, we have taken a closer look at the privacy policy. Almost all the required information points of a privacy policy are covered. However, the trackers found in both the static and dynamic analysis are an exception. These were not mentioned in the privacy policy. These are two widely used trackers called Google CrashLytics, which reports crashes, and Google Firebase Analytics, which generates other reports. Nevertheless, they should definitely be mentioned in the privacy policy.

Last but not least, a few general points for an overview: The privacy policy comprises 2,434 words with an average of 27.35 words per sentence. It takes the user around 2.96 minutes to read the entire privacy policy. The privacy policy in the Google Play Store is displayed in English. The Flesch-Kincaid readability index was used to assess readability, which in our example indicates that a degree is required to understand the privacy policy.

The successor model Yale Linus Smart Lock L2 offers improved performance and an integrated WLAN module. Although Wi-Fi integration improves connectivity, it is crucial to ensure that robust security measures are implemented to prevent potential cyberattacks. However, in testing, we also found no evidence of potentially serious issues in this area.

Overall, the Yale Linus L2 presents itself as an absolutely solid solution in its test debut. Of course, the test depth in our Quick Check is still on the shallower side, so problems could still occur here on closer inspection. For now, however, there are no signs of this. The solution is also solid in terms of data protection and privacy. A rating of the full 3 out of 3 stars is therefore absolutely justified here.