8000 Add Smart Meter B Route integration by SeraphicRav · Pull Request #123446 · home-assistant/core · GitHub
[go: up one dir, main page]

Skip to content

Add Smart Meter B Route integration #123446

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 70 commits into
base: dev
Choose a base branch
from

Conversation

SeraphicRav
Copy link
Contributor
@SeraphicRav SeraphicR 8000 av commented Aug 9, 2024

Proposed change

This PR adds a new integration to use Bルートサービス (B Route Service) which uses Echonet Lite over Wi-Sun to provide readings from standard Smart Meters in Japan using a compatible serial device.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.

To help with the load of incoming pull requests:

@SeraphicRav SeraphicRav changed the title Creating new integration Add Smart Meter B Route integration Aug 9, 2024
@SeraphicRav
Copy link
Contributor Author

It is my first time working with serial communication.
In the device I use, https://www-ratoc--e2estore-com.translate.goog/blog/2023/06/wsuha-01?_x_tr_sl=ja&_x_tr_tl=en&_x_tr_hl=en&_x_tr_pto=sc it is expected to set manually WOPT to 01.
The library I am using is supposing the operation has been manually done: https://github.com/nbtk/momonga?tab=readme-ov-file#preparation

I am wondering if:

  • I should check in Home Assistant during the config flow for instance if the option has been set and set it if not
    
  • I should I write a library communicating directly in binary
    
  • I should tell the user to do it himself (I found it not really user friendly to do)
    

Which road should I follow ?

Asked in: https://discord.com/channels/330944238910963714/1271130001995071561

@SeraphicRav
Copy link
Contributor Author

I am making an integration to be able to access Smart Meter data using a USB dongle to communicate via Echonet Lite over Wisun (I think it is only for the Japanese market).

The library I am using is https://github.com/nbtk/momonga?tab=readme-ov-file#ログを有効にした例
My issue is that the library is really noisy by default and I wonder if I should fix it and how. The provided link leads you to the documentation to configure the logging.

It basically exposes 3 loggers for which I can set up the logging level and formater.

Should I just suppress the logs or is there a way to align the log level of the integration with the library ?

Log samples

2024-08-09 10:28:10.771 INFO (MainThread) [homeassistant.setup] Setting up smart_meter_b_route
2024-08-09 10:28:10.773 INFO (SyncWorker_0) [momonga.momonga] Opening Momonga.
2024-08-09 10:28:10.773 INFO (SyncWorker_0) [momonga.momonga_session_manager] Opening a Momonga session...
2024-08-09 10:28:12.998 INFO (SyncWorker_0) [momonga.momonga_session_manager] The Route-B ID and the password were registered.
2024-08-09 10:28:12.998 INFO (SyncWorker_0) [momonga.momonga_session_manager] Scanning PAN channels...
2024-08-09 10:29:05.439 INFO (SyncWorker_0) [momonga.momonga_session_manager] A PAN was found.
2024-08-09 10:29:05.461 INFO (SyncWorker_0) [momonga.momonga_session_manager] Joining the PAN...
2024-08-09 10:29:07.860 INFO (SyncWorker_0) [momonga.momonga_session_manager] A PANA session has been established.
2024-08-09 10:29:07.861 INFO (SyncWorker_0) [momonga.momonga_session_manager] A Momonga session is open.
2024-08-09 10:29:12.861 INFO (SyncWorker_0) [momonga.momonga] Momonga is open.
2024-08-09 10:29:12.866 INFO (MainThread) [homeassistant.components.sensor] Setting up smart_meter_b_route.sensor
2024-08-09 10:29:12.868 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.smart_meter_b_route entity: sensor.instantaneous_current_r_phase
2024-08-09 10:33:48.373 INFO (SyncWorker_0) [momonga.momonga] Successfully transmitted a packet for "E7" request.
2024-08-09 10:33:49.770 INFO (SyncWorker_2) [momonga.momonga] Successfully transmitted a packet for "E0" request.
2024-08-09 10:33:50.630 INFO (SyncWorker_2) [momonga.momonga] Successfully received a packet for "E0" response.
2024-08-09 10:34:59.778 INFO (SyncWorker_0) [momonga.momonga] Successfully received a packet for "E0" response.
2024-08-09 10:35:10.440 INFO (Recorder) [homeassistant.components.sensor.recorder] Compiling initial sum statistics for sensor.total_consumption, zero point set to 7203.8

Asked in https://discord.com/channels/330944238910963714/1271127972103065695

@SeraphicRav
Copy link
Contributor Author

@joostlek Thanks for the review !

Oh, I have also a question about the brand to use, it is not a brand per see but a standard. Do I need to create a brand in the core project and add a logo (which logo) in the brand project ?

@joostlek
Copy link
Member
joostlek commented Aug 9, 2024

A brand is only used for grouping integrations for a brand (like with switchbot switchbot cloud). In this case it could be like iron_os where the main integration has no logo or brand, because there is none, but then you create a virtual integration per company that supports it

@SeraphicRav
Copy link
Contributor Author

Hum... the device, id and password are actually at the HA device level, not at the HA integration level, I need to change the config flow

@SeraphicRav
Copy link
Contributor Author
SeraphicRav commented Aug 15, 2024

I fixed the previous issues:

  • Better UX to select USB device
  • Setting ASCII mode if not active

The current issue is that in the dev container, async_step_usb never kicks in, any idea about what I miss ?

@joostlek
Copy link
Member

You probably need to expose the USB port to the devcontainer

@SeraphicRav
Copy link
Contributor Author
SeraphicRav commented Aug 16, 2024

You probably need to expose the USB port to the devcontainer

It is exposed, I updated the Dockerfile.dev to add:

RUN usermod -aG dialout vscode

.devcontainer/devcontainer.json to add

"--priviledged"

And in the terminal in the dev container, I can see:

vscode ➜ /workspaces/home-assistant-core (Smart-Meter-B-Route) $ ls /dev
acpi_thermal_rel fb0 loop-control net pts tty1 tty22 tty35 tty48 tty60 urandom vcsa3 vga_arbiter
autofs fd loop0 ng0n1 random tty10 tty23 tty36 tty49 tty61 usb vcsa4 vhci
btrfs-control full loop1 null rfkill tty11 tty24 tty37 tty5 tty62 userfaultfd vcsa5 vhost-net
bus fuse loop2 nvme0 rtc0 tty12 tty25 tty38 tty50 tty63 vcs vcsa6 vhost-vsock
core gpiochip0 loop3 nvme0n1 shm tty13 tty26 tty39 tty51 tty7 vcs1 vcsa7 video0< 8000 br> cpu gpiochip1 loop4 nvme0n1p1 snapshot tty14 tty27 tty4 tty52 tty8 vcs2 vcsu video1
cpu_dma_latency hidraw0 loop5 nvme0n1p2 snd tty15 tty28 tty40 tty53 tty9 vcs3 vcsu1 watchdog
cuse hidraw1 loop6 nvme0n1p3 stderr tty16 tty29 tty41 tty54 ttyS0 vcs4 vcsu2 watchdog0
dri hidraw2 loop7 nvme0n1p4 stdin tty17 tty3 tty42 tty55 ttyS1 vcs5 vcsu3 zero
drm_dp_aux0 hpet mapper nvram stdout tty18 tty30 tty43 tty56 ttyS2 vcs6 vcsu4
drm_dp_aux1 hwrng media0 port tpm0 tty19 tty31 tty44 tty57 ttyS3 vcs7 vcsu5
drm_dp_aux2 input mei0 ppp tpmrm0 tty2 tty32 tty45 tty58 ttyUSB0 vcsa vcsu6
drm_dp_aux3 kmsg mem psaux tty tty20 tty33 tty46 tty59 uhid vcsa1 vcsu7
drm_dp_aux4 kvm mqueue ptmx tty0 tty21 tty34 tty47 tty6 uinput vcsa2 vfio
vscode ➜ /workspaces/home-assistant-core (Smart-Meter-B-Route) $

ttyUSB0 is present and I can use it in Home Assistant to do running tests but the discovery does not work after I deleted the integration from the UI, even after a HA reboot.

@SeraphicRav
Copy link
Contributor Author

And the logs of the used library are still noisy :D

2024-08-16 13:21:39.633 INFO (SyncWorker_0) [momonga.momonga] Opening Momonga.
2024-08-16 13:21:39.633 INFO (SyncWorker_0) [momonga.momonga_session_manager] Opening a Momonga session...
2024-08-16 13:21:41.856 INFO (SyncWorker_0) [momonga.momonga_session_manager] The Route-B ID and the password were registered.
2024-08-16 13:21:41.856 INFO (SyncWorker_0) [momonga.momonga_session_manager] Scanning PAN channels...
2024-08-16 13:22:34.293 INFO (SyncWorker_0) [momonga.momonga_session_manager] A PAN was found.
2024-08-16 13:22:34.315 INFO (SyncWorker_0) [momonga.momonga_session_manager] Joining the PAN...
2024-08-16 13:22:43.909 INFO (SyncWorker_0) [momonga.momonga_session_manager] A PANA session has been established.
2024-08-16 13:22:43.910 INFO (SyncWorker_0) [momonga.momonga_session_manager] A Momonga session is open.
2024-08-16 13:22:48.910 INFO (SyncWorker_0) [momonga.momonga] Momonga is open.
2024-08-16 13:22:49.817 INFO (SyncWorker_2) [momonga.momonga] Successfully transmitted a packet for "E8" request.
2024-08-16 13:22:51.007 INFO (SyncWorker_2) [momonga.momonga] Successfully received a packet for "E8" response.
2024-08-16 13:22:51.254 INFO (SyncWorker_2) [momonga.momonga] Successfully transmitted a packet for "E7" request.
2024-08-16 13:22:52.583 INFO (SyncWorker_2) [momonga.momonga] Successfully received a packet for "E7" response.
2024-08-16 13:22:53.008 INFO (SyncWorker_2) [momonga.momonga] Successfully transmitted a packet for "D3" request.
2024-08-16 13:22:53.058 INFO (SyncWorker_2) [momonga.momonga] Successfully received a packet for "D3" response.
2024-08-16 13:22:58.354 INFO (SyncWorker_2) [momonga.momonga] Successfully transmitted a packet for "E1" request.
2024-08-16 13:22:59.084 INFO (SyncWorker_2) [momonga.momonga] Successfully received a packet for "E1" response.
2024-08-16 13:23:04.444 INFO (SyncWorker_2) [momonga.momonga] Successfully transmitted a packet for "E0" request.
2024-08-16 13:23:05.336 INFO (SyncWorker_2) [momonga.momonga] Successfully received a packet for "E0" response.
2024-08-16 13:23:05.341 INFO (MainThread) [homeassistant.components.sensor] Setting up smart_meter_b_route.sensor
2024-08-16 13:23:05.343 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.smart_meter_b_route entity: sensor.none_instantaneous_current_r_phase
2024-08-16 13:23:05.349 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.smart_meter_b_route entity: sensor.none_instantaneous_current_t_phase
2024-08-16 13:23:05.350 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.smart_meter_b_route entity: sensor.none_instantaneous_power
2024-08-16 13:23:05.352 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.smart_meter_b_route entity: sensor.none_total_consumption
2024-08-16 13:24:05.388 INFO (SyncWorker_1) [momonga.momonga] Successfully transmitted a packet for "E8" request.
2024-08-16 13:24:06.534 INFO (SyncWorker_1) [momonga.momonga] Successfully received a packet for "E8" response.
2024-08-16 13:24:06.607 INFO (SyncWorker_1) [momonga.momonga] Successfully transmitted a packet for "E7" request.
2024-08-16 13:24:07.785 INFO (SyncWorker_1) [momonga.momonga] Successfully received a packet for "E7" response.
2024-08-16 13:24:07.985 INFO (SyncWorker_1) [momonga.momonga] Successfully transmitted a packet for "E0" request.
2024-08-16 13:24:08.534 INFO (SyncWorker_1) [momonga.momonga] Successfully received a packet for "E0" response.
2024-08-16 13:25:08.387 INFO (SyncWorker_0) [momonga.momonga] Successfully transmitted a packet for "E8" request.
2024-08-16 13:25:09.782 INFO (SyncWorker_0) [momonga.momonga] Successfully received a packet for "E8" response.
2024-08-16 13:25:10.126 INFO (SyncWorker_0) [momonga.momonga] Successfully transmitted a packet for "E7" request.
2024-08-16 13:25:10.272 INFO (Recorder) [homeassistant.components.sensor.recorder] Compiling initial sum statistics for sensor.none_total_consumption, zero point set to 7370.2
2024-08-16 13:25:11.258 INFO (SyncWorker_0) [momonga.momonga] Successfully received a packet for "E7" response.
2024-08-16 13:25:11.571 INFO (SyncWorker_0) [momonga.momonga] Successfully transmitted a packet for "E0" request.
2024-08-16 13:25:12.333 INFO (SyncWorker_0) [momonga.momonga] Successfully received a packet for "E0" response.

@SeraphicRav
Copy link
Contributor Author

Failing tests are outside my scope:

  • motionblinds_ble

@SeraphicRav SeraphicRav marked this pull request as ready for review August 21, 2024 13:11
@SeraphicRav
Copy link
Contributor Author

The maintainer of Momonga added CI for releases.

@SeraphicRav SeraphicRav marked this pull request as draft February 20, 2025 15:21
@SeraphicRav SeraphicRav marked this pull request as ready for review February 22, 2025 10:56
@SeraphicRav
Copy link
Contributor Author

Note: Tests are failing in unrelated code

@SeraphicRav
Copy link
Contributor Author

Fixed issues

Comment on lines +18 to +46
def is_duplicate(
hass: HomeAssistant,
entry: ConfigEntry,
) -> bool:
"""Filter duplicate entries."""
existing_entries = hass.config_entries.async_entries(
DOMAIN, include_disabled=False, include_ignore=False
)
entry_bid = entry.data[CONF_ID]
for existing_entry in existing_entries:
existing_entry_bid = existing_entry.data[CONF_ID]
if (
existing_entry_bid == entry_bid
and existing_entry.unique_id != entry.unique_id
):
_LOGGER.warning(
"Duplicate entry found (Skipping): existing_entry.unique_id=%s, entry.unique_id=%s, entry.runtime_data.bid=%s",
existing_entry.unique_id,
entry.unique_id,
entry_bid,
)
return True
return False


async def async_setup_entry(hass: HomeAssistant, entry: BRouteConfigEntry) -> bool:
"""Set up Smart Meter B Route from a config entry."""
if is_duplicate(hass, entry):
return False
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we do this here? If we check this during the config flow then there's no problem right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The config flow does not check if the user input for bid is duplicated or not.

This was implemented following this comment: #123446 (comment)

Comment on lines +97 to +101
if not self.context.get("unique_id"):
await self.async_set_unique_id(
_generate_unique_id(device), raise_on_progress=False
)
self._abort_if_unique_id_configured()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in theory the unique id wouldnt be set right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, not sure I understand the comment :D

It is set just above

10000
@home-assistant home-assistant bot marked this pull request as draft April 8, 2025 12:56
],
"quality_scale": "bronze",
"requirements": ["pyserial==3.5", "momonga==0.1.4"],
"usb": [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The USB discovery is pretty broad, it will match any USB-serial chip produced by FTDI. Does the device have any USB descriptors populated? What does it look like with lsusb?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the result of lsusb:
Bus 003 Device 002: ID 0403:6015 Future Technology Devices International, Ltd Bridge(I2C/SPI/UART/FIFO)

I am not knowledgeable about USB serials, if it can be more precise, I would be glad to restrict it.

They have multiple models: https://www.ratocsystems.com/products/wisun/usb-wisun/rs-wsuha/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked with @puddly and that means we can't discover this with too many false positives

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Contacted the USB device maker, maybe a suggestion will be made.

@SeraphicRav SeraphicRav marked this pull request as ready for review May 20, 2025 15:37
@home-assistant home-assistant bot requested a review from joostlek May 20, 2025 15:38
],
"quality_scale": "bronze",
"requirements": ["pyserial==3.5", "momonga==0.1.4"],
"usb": [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked with @puddly and that means we can't discover this with too many false positives

@home-assistant home-assistant bot marked this pull request as draft May 20, 2025 18:33
Sign up for free to join this conve 6F85 rsation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0