Add the ability to mark dps as optional.

Issue #179.

A number of configs with known dps that were commented out because of not being consistently available have had this option enabled.

Should resolve #187 as well, without adding a specific config.

custom_components/tuya_local/devices/README.md

@@ -77,7 +77,7 @@ secondary entities, so only basic functionality is implemented.
 
 ### `deprecated`
 
-*Optional, deprecated*
+*Deprecated, DO NOT USE for new devices.*
 
 This is used to mark an entity as deprecated.  This is mainly
 for older devices that were implemented when only climate devices were
@@ -162,12 +162,21 @@ to use a secondary entity for that.
 
 ### `readonly`
 
-*Optional.*
+*Optional, default false.*
 
 A boolean setting to mark attributes as readonly. If not specified, the
 default is `false`.  If set to `true`, the attributes will be reported
 to Home Assistant, but no functionality for setting them will be exposed.
 
+### `optional`
+
+*Optional, default false.*
+
+A boolean setting to mark attributes as optional.  This allows a device to be
+matched even if it is not sending the dp at the time when adding a new device.
+It can also be used to match a range of devices that have variations in the extra
+attributes that are sent.
+
 ### `mapping`
 
 *Optional.*
@@ -182,7 +191,7 @@ defined in their own section below.
 
 ### `hidden`
 
-*Optional.*
+*Optional, default false.*
 This can be used to define DPs that do not directly expose Home Assistant
 attributes.  When set to **true**, no attribute will be sent. A `name` should
 still be specified and the attribute can be referenced as a `constraint`
@@ -198,7 +207,7 @@ is set to Eco.
 
 ### `range`
 
-*Optional.*
+*Optional, may be required in some contexts, may have defaults in others.*
 
 For integer attributes that are not readonly, a range can be set with `min`
 and `max` values that will limit the values that the user can enter in the
@@ -206,7 +215,7 @@ Home Assistant UI.  This can also be set in a `mapping` or `conditions` block.
 
 ### `unit`
 
-*Optional. default="C" for temperature dps on climate devices, None for sensors.*
+*Optional, default="C" for temperature dps on climate devices.*
 
 For temperature dps, some devices will use Fahrenhiet.  This needs to be
 indicated back to HomeAssistant by defining `unit` as "F".  For sensor 
@@ -216,7 +225,7 @@ equivalents, other units are currently ASCII so can be easily entered directly).
 
 ### `class`
 
-*Optional.  default=None.*
+*Optional.*
 
 For sensors, this sets the state class of the sensor (measurement, total
 or total_increasing)
@@ -224,7 +233,7 @@ or total_increasing)
 
 ### `format`
 
-*Optional. default=None*
+*Optional.*
 
 For base64 and hex types, this specifies how to decode the binary data (after hex or base64 decoding).
 This is a container field, the contents of which should be a list consisting of `name`, `bytes` and `range` fields.  `range` is as described above.  `bytes` is the number of bytes for the field, which can be `1`, `2`, or `4`.  `name` is a name for the field, which will have special handling depending on
@@ -266,7 +275,7 @@ level to override all values, but I can't imagine a useful purpose for that.
 
 ### `scale`
 
-*Optional, default=1*
+*Optional, default=1.*
 
 This can be used in an `integer` dp mapping to scale the values.  For example
 some climate devices represent the temperature as an integer in tenths of
@@ -277,7 +286,7 @@ of 0.03.
 
 ###`invert`
 
-*Optional, default=False*
+*Optional, default=False.*
 
 This can be used in an `integer` dp mapping to invert the range.  For example,
 some cover devices have an opposite idea of which end of the percentage scale open
@@ -286,7 +295,7 @@ must also be specified for the dp.
 
 ### `step`
 
-*Optional, default=1*
+*Optional, default=1.*
 
 This can be used in an `integer` dp mapping to make values jump by a specific
 step.  It can also be set in a conditions block so that the steps change only
@@ -346,7 +355,7 @@ have a mapping that mirrors the value of the configuration dp.
 
 ### `invalid`
 
-*Optional. Boolean, default false.*
+*Optional, default false.*
 
 Invalid set to true allows an attribute to temporarily be set read-only in
 some conditions.  Rather than passing requests to set the attribute through
@@ -360,14 +369,14 @@ control when the preset is in sleep mode (since sleep mode should force low).
 
 ### `constraint`
 
-*Optional. Always paired with `conditions`.*
+*Optional, always paired with `conditions`.*
 
 If a rule depends on an attribute other than the current one, then `constraint`
 can be used to specify the element that `conditions` applies to.
 
 ### `conditions`
 
-*Optional. Always paired with `constraint.`*
+*Optional, always paired with `constraint.`*
 
 Conditions defines a list of rules that are applied based on the `constraint`
 attribute. The contents are the same as Mapping Rules, but `dps_val` applies
@@ -464,7 +473,16 @@ Humidifer can also cover dehumidifiers (use class to specify which).
    If the light mixes in color modes in the same dp, **color_mode** should be used instead.
 
 ### lock
-- **lock** (required, boolean): a dp to control the lock state: true = locked, false = unlocked
+- **lock** (optional, boolean): a dp to control the lock state: true = locked, false = unlocked
+- **unlock_fingerprint** (optional, integer): a dp to identify the fingerprint used to unlock the lock.
+- **unlock_password** (optional, integer): a dp to identify the password used to unlock the lock.
+- **unlock_temp_pwd** (optional, integer): a dp to identify the temporary password used to unlock the lock.
+- **unlock_dynamic_pwd** (optional, integer): a dp to identify the dynamic password used to unlock the lock.
+- **unlock_card** (optional, integer): a dp to identify the card used to unlock the lock.
+- **unlock_app** (optional, integer): a dp to identify the app used to unlock the lock.
+- **request_unlock** (optional, integer): a dp to signal that a request has been made to unlock, the value should indicate the time remaining for approval.
+- **approve_unlock** (optional, boolean): a dp to unlock the lock in response to a request.
+- **jammed** (optional, boolean): a dp to signal that the lock is jammed.
 
 ### number
 - **value** (required, number): a dp to control the number that is set.

custom_components/tuya_local/devices/bresser_weather_station.yaml

@@ -19,9 +19,10 @@ primary_entity:
           - scale: 10
         class: measurement
         readonly: true
-#    - id: 22
-#      type: string
-#      name: fault_type
+    - id: 22
+      type: string
+      name: fault_type
+      optional: true
     - id: 58
       type: string
       name: wind_direct
@@ -134,27 +135,30 @@ secondary_entities:
     dps:
       - id: 30
         <<: *battery
-#  - entity: binary_sensor
-#    name: Battery Ch1
-#    class: battery
-#    category: diagnostic
-#    dps:
-#      - id: 31
-#        <<: *battery
-#  - entity: binary_sensor
-#    name: Battery Ch2
-#    class: battery
-#    category: diagnostic
-#    dps:
-#      - id: 32
-#        <<: *battery
-#  - entity: binary_sensor
-#    name: Battery Ch3
-#    class: battery
-#    category: diagnostic
-#    dps:
-#      - id: 33
-#        <<: *battery
+  - entity: binary_sensor
+    name: Battery Ch1
+    class: battery
+    category: diagnostic
+    dps:
+      - id: 31
+        optional: true
+        <<: *battery
+  - entity: binary_sensor
+    name: Battery Ch2
+    class: battery
+    category: diagnostic
+    dps:
+      - id: 32
+        optional: true
+        <<: *battery
+  - entity: binary_sensor
+    name: Battery Ch3
+    class: battery
+    category: diagnostic
+    dps:
+      - id: 33
+        optional: true
+        <<: *battery
   - entity: sensor
     name: Temperature
     class: temperature
@@ -167,42 +171,48 @@ secondary_entities:
     dps:
       - id: 39
         <<: *humidity
-#  - entity: sensor
-#    name: Temperature Ch1
-#    class: temperature
-#    dps:
-#      - id: 40
-#        <<: *temperature
-#  - entity: sensor
-#    name: Humidity Ch1
-#    class: humidity
-#    dps:
-#      - id: 41
-#        <<: *humidity
-#  - entity: sensor
-#    name: Temperature Ch2
-#    class: temperature
-#    dps:
-#      - id: 42
-#        <<: *temperature
-#  - entity: sensor
-#    name: Humidity Ch2
-#    class: humidity
-#    dps:
-#      - id: 43
-#        <<: *humidity
-#  - entity: sensor
-#    name: Temperature Ch3
-#    class: temperature
-#    dps:
-#      - id: 44
-#        <<: *temperature
-#  - entity: sensor
-#    name: Humidity Ch3
-#    class: humidity
-#    dps:
-#      - id: 45
-#        <<: *humidity
+  - entity: sensor
+    name: Temperature Ch1
+    class: temperature
+    dps:
+      - id: 40
+        optional: true
+        <<: *temperature
+  - entity: sensor
+    name: Humidity Ch1
+    class: humidity
+    dps:
+      - id: 41
+        optional: true
+        <<: *humidity
+  - entity: sensor
+    name: Temperature Ch2
+    class: temperature
+    dps:
+      - id: 42
+        optional: true
+        <<: *temperature
+  - entity: sensor
+    name: Humidity Ch2
+    class: humidity
+    dps:
+      - id: 43
+        optional: true
+        <<: *humidity
+  - entity: sensor
+    name: Temperature Ch3
+    class: temperature
+    dps:
+      - id: 44
+        optional: true
+        <<: *temperature
+  - entity: sensor
+    name: Humidity Ch3
+    class: humidity
+    dps:
+      - id: 45
+        optional: true
+        <<: *humidity
   - entity: sensor
     name: Pressure
     class: pressure

custom_components/tuya_local/devices/kyvol_e30_vacuum.yaml

@@ -85,14 +85,16 @@ primary_entity:
           value: gentle
         - dps_val: "4"
           value: closed
-#    - id: 15
-#      type: string
-#      name: clean_record
-#      readonly: true
-#    - id: 16
-#      type: integer
-#      name: clean_area
-#      unit: m2
+    - id: 15
+      type: string
+      name: clean_record
+      readonly: true
+      optional: true
+    - id: 16
+      type: integer
+      name: clean_area
+      unit: m2
+      optional: true
     - id: 18
       type: bitfield
       name: error

custom_components/tuya_local/devices/lefant_m213_vacuum.yaml

@@ -65,11 +65,11 @@ primary_entity:
     - id: 13
       type: boolean
       name: locate
-# Listed at iot.tuya.com, but does not seem to be returned by default
-#    - id: 15
-#      type: string
-#      name: clean_record
-#      readonly: true
+    - id: 15
+      type: string
+      name: clean_record
+      readonly: true
+      optional: true
     - id: 18
       type: bitfield
       name: error
@@ -107,16 +107,14 @@ primary_entity:
     - id: 104
       type: integer
       name: unknown_104
-# dp 106 returned only by Lefant M213 and M213S, not by APOSEN A550
-# seems to be some encoded ASCII descriptor related to charging status
-#    - id: 106
-#      type: string
-#      name: unknown_106
-# dp 108 returned only sometimes?
-# seems to be some encoded ASCII descriptor of battery voltage
-#    - id: 108
-#      type: string
-#      name: unknown_108
+    - id: 106
+      type: string
+      name: charge_status
+      optional: true
+    - id: 108
+      type: string
+      name: battery_status
+      optional: true
 secondary_entities:
   - entity: sensor
     name: Clean Area

custom_components/tuya_local/devices/orion_smart_lock.yaml

@@ -35,16 +35,18 @@ primary_entity:
     - id: 15
       type: integer
       name: unlock_app
-# Suspect the following are not always reported, as they are large hex fields
-#    - id: 25
-#      type: hex
-#      name: fingers_enrolled
-#    - id: 26
-#      type: hex
-#      name: passwords_enrolled
-#    - id: 27
-#      type: hex
-#      name: cards_enrolled
+    - id: 25
+      type: hex
+      name: fingers_enrolled
+      optional: true
+    - id: 26
+      type: hex
+      name: passwords_enrolled
+      optional: true
+    - id: 27
+      type: hex
+      name: cards_enrolled
+      optional: true
 secondary_entities:
   - entity: sensor
     name: alert

custom_components/tuya_local/devices/rgbcw_lightbulb.yaml

@@ -31,6 +31,7 @@ primary_entity:
     - id: 23
       name: color_temp
       type: integer
+      optional: true
       range:
         min: 0
         max: 1000
@@ -39,6 +40,7 @@ primary_entity:
     - id: 24
       name: rgbhsv
       type: hex
+      optional: true
       format:
         - name: h
           bytes: 2
@@ -58,23 +60,27 @@ primary_entity:
     - id: 25
       name: scene_data
       type: hex
-#  The following were removed because they aren't always present,
-#  and anyway they are complex hex data fields that we cannot handle.
-#   - id: 27
-#     name: music_data
-#     type: hex
-#   - id: 28
-#     name: control_data
-#     type: hex
-#   - id: 30
-#     name: rhythm_mode
-#     type: hex
-#   - id: 31
-#     name: sleep_mode
-#     type: hex
-#   - id: 32
-#     name: wakeup_mode
-#     type: hex
+      optional: true
+    - id: 27
+      name: music_data
+      type: hex
+      optional: true
+    - id: 28
+      name: control_data
+      type: hex
+      optional: true
+    - id: 30
+      name: rhythm_mode
+      type: hex
+      optional: true
+    - id: 31
+      name: sleep_mode
+      type: hex
+      optional: true
+    - id: 32
+      name: wakeup_mode
+      type: hex
+      optional: true
 secondary_entities:
   - entity: number
     name: Timer

custom_components/tuya_local/helpers/device_config.py

@@ -112,12 +112,16 @@ class TuyaDeviceConfig:
     def matches(self, dps):
         """Determine if this device matches the provided dps map."""
         for d in self.primary_entity.dps():
-            if d.id not in dps.keys() or not _typematch(d.type, dps[d.id]):
+            if (d.id not in dps.keys() and not d.optional) or (
+                d.id in dps.keys() and not _typematch(d.type, dps[d.id])
+            ):
                 return False
 
         for dev in self.secondary_entities():
             for d in dev.dps():
-                if d.id not in dps.keys() or not _typematch(d.type, dps[d.id]):
+                if (d.id not in dps.keys() and not d.optional) or (
+                    d.id in dps.keys() and not _typematch(d.type, dps[d.id])
+                ):
                     return False
         _LOGGER.debug("Matched config for %s", self.name)
         return True
@@ -136,8 +140,8 @@ class TuyaDeviceConfig:
             True if all dps in entity could be matched to dps, False otherwise
         """
         for d in entity.dps():
-            if (d.id not in keys and d.id not in matched) or not _typematch(
-                d.type, dps[d.id]
+            if (d.id not in keys and d.id not in matched and not d.optional) or (
+                (d.id in keys or d.id in matched) and not _typematch(d.type, dps[d.id])
             ):
                 return False
             if d.id in keys:
@@ -290,6 +294,10 @@ class TuyaDpsConfig:
     def name(self):
         return self._config["name"]
 
+    @property
+    def optional(self):
+        return self._config.get("optional", False)
+
     @property
     def format(self):
         fmt = self._config.get("format")
@@ -485,6 +493,7 @@ class TuyaDpsConfig:
             except ValueError:
                 self.stringify = False
         else:
+
             self.stringify = False
 
         result = value

custom_components/tuya_local/helpers/mixin.py

@@ -73,7 +73,9 @@ class TuyaLocalEntity:
         """Get additional attributes that the platform itself does not support."""
         attr = {}
         for a in self._attr_dps:
-            attr[a.name] = a.get_value(self._device)
+            value = a.get_value(self._device)
+            if value is not None or not a.optional:
+                attr[a.name] = value
         return attr
 
     async def async_update(self):

tests/devices/base_device_tests.py

@@ -47,7 +47,7 @@ class TuyaDeviceTestCase(IsolatedAsyncioTestCase):
         self.addCleanup(device_patcher.stop)
         self.mock_device = device_patcher.start()
         self.dps = payload.copy()
-        self.mock_device.get_property.side_effect = lambda id: self.dps[id]
+        self.mock_device.get_property.side_effect = lambda id: self.dps.get(id)
         cfg = TuyaDeviceConfig(config_file)
         self.conf_type = cfg.legacy_type
         type(self.mock_device).has_returned_state = PropertyMock(return_value=True)