Recorder errors #206

peterg79 · 2023-09-13T07:33:40Z

peterg79
Sep 13, 2023

Every once in a while I'm getting weird data on the energy dashboard (see screenshot)

The corresponding errors in the log are the following:

2023-09-12 08:06:22.876 WARNING (MainThread) [aioesphomeapi.connection] emporiavue2 @ 172.27.201.245: Connection error occurred: [Errno 104] Connection reset by peer
2023-09-12 08:10:10.149 WARNING (Recorder) [homeassistant.components.sensor.recorder] Entity sensor.vue2_ac_daily_energy from integration esphome has state class total_increasing, but its state is not strictly increasing. Triggered by state 391081 (391083.0) with last_updated set to 2023-09-12T15:06:27.401919+00:00. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+esphome%22
2023-09-12 08:10:10.149 WARNING (Recorder) [homeassistant.components.sensor.recorder] Entity sensor.vue2_spa_daily_energy from integration esphome has state class total_increasing, but its state is not strictly increasing. Triggered by state 84 (86.0) with last_updated set to 2023-09-12T15:06:27.438460+00:00. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+esphome%22
2023-09-12 08:10:10.150 WARNING (Recorder) [homeassistant.components.sensor.recorder] Entity sensor.vue2_solar_daily_energy from integration esphome has state class total_increasing, but its state is not strictly increasing. Triggered by state 1464194 (1490696.0) with last_updated set to 2023-09-12T15:06:27.528440+00:00. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+esphome%22

I suspect the device has been reset (maybe power fluctuations?) and it may have reverted to the last stored values in flash. I use the flash_write_interval: "48h" as recommended in the readme. Did anyone else experience the same? Is there a recommended workaround?

Answered by flaviut

Sep 13, 2023

Interesting--my power is quite reliable, so I haven't gotten the chance to experience this.

What if you configure your total_daily_energy sensors like this, the difference being that each one has restore: false set?


  - { power_id:  cir1, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name:  "Circuit 1 Daily Energy", filters: *throttle_time }
  - { power_id:  cir2, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name:  "Circuit 2 Daily Energy", filters: *throttle_time }
  - { power_id:  cir3, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name:  "Circuit 3 Daily Energy", filters: *throttle_time }
  - { power_id:  cir4, platfor…

View full answer

flaviut · 2023-09-13T22:20:59Z

flaviut
Sep 13, 2023
Maintainer

Interesting--my power is quite reliable, so I haven't gotten the chance to experience this.

What if you configure your total_daily_energy sensors like this, the difference being that each one has restore: false set?


  - { power_id:  cir1, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name:  "Circuit 1 Daily Energy", filters: *throttle_time }
  - { power_id:  cir2, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name:  "Circuit 2 Daily Energy", filters: *throttle_time }
  - { power_id:  cir3, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name:  "Circuit 3 Daily Energy", filters: *throttle_time }
  - { power_id:  cir4, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name:  "Circuit 4 Daily Energy", filters: *throttle_time }
  - { power_id:  cir5, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name:  "Circuit 5 Daily Energy", filters: *throttle_time }
  - { power_id:  cir6, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name:  "Circuit 6 Daily Energy", filters: *throttle_time }
  - { power_id:  cir7, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name:  "Circuit 7 Daily Energy", filters: *throttle_time }
  - { power_id:  cir8, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name:  "Circuit 8 Daily Energy", filters: *throttle_time }
  - { power_id:  cir9, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name:  "Circuit 9 Daily Energy", filters: *throttle_time }
  - { power_id: cir10, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name: "Circuit 10 Daily Energy", filters: *throttle_time }
  - { power_id: cir11, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name: "Circuit 11 Daily Energy", filters: *throttle_time }
  - { power_id: cir12, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name: "Circuit 12 Daily Energy", filters: *throttle_time }
  - { power_id: cir13, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name: "Circuit 13 Daily Energy", filters: *throttle_time }
  - { power_id: cir14, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name: "Circuit 14 Daily Energy", filters: *throttle_time }
  - { power_id: cir15, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name: "Circuit 15 Daily Energy", filters: *throttle_time }
  - { power_id: cir16, platform: total_daily_energy, accuracy_decimals: 0, restore: false, name: "Circuit 16 Daily Energy", filters: *throttle_time }

You can then remove the entire preferences section.

Do you still see the same problematic behavior?

0 replies

clowrey · 2023-09-14T00:50:23Z

clowrey
Sep 14, 2023

I know these types of sensors are only supposed to go up in value - so maybe if you're getting random negative values on some of those it could cause issues? maybe try using the positive only value filter on them just to be sure?

2 replies

flaviut Sep 14, 2023
Maintainer

I think the issue here is an interaction between ESPHome and Home Assistant.

When Home Assistant sees the value of the total daily energy drop to a value that's not 0, I assume that it thinks a bunch of energy has been fed back into the grid. With the change I suggested above, the sensor will reset to 0 when the Vue reboots & hopefully resolve this issue.

clowrey Sep 14, 2023

Yeah that definitely might fix this error too.. hope so!

clowrey · 2023-09-14T00:51:17Z

clowrey
Sep 14, 2023

Oh and also the method: left should be set ideally for this type of sensor. I was having all sorts of issues like this upon restart with this when the method was anything but left in HA.

3 replies

peterg79 Sep 14, 2023
Author

Thanks for the tip! I briefly looked at the code here:

esphome/esphome/components/total_daily_energy/total_daily_energy.cpp

Line 62 in afbb900

case TOTAL_DAILY_ENERGY_METHOD_LEFT:

and I need more time to investigate how it makes a difference. Can you share what was your issue?

clowrey Sep 14, 2023

It was with the home assistant integration sensors - to do with if they received a data point that was say 500w and then no data for 6 hours on the default they would assume the data was 500w for that whole time even with no updates. The left setting makes it not use the last known value for the integration. Maybe not applicable here since it's all being done in the ESPhome..

peterg79 Sep 14, 2023
Author

Thanks for the details. This seems like a separate issue, as values are continuously reported even if they are not changing (this could be optimized in the future). I see it can cause a problem with measurement state class, but should not cause any trouble with total and total increasing. Once I sort out the daily_total_energy measures, I'll take a look at the power sensors to see if I have this problem. Thanks again!

peterg79 · 2023-09-14T06:36:39Z

peterg79
Sep 14, 2023
Author

Thanks for the tips! I see that the restore defaults to true here:

esphome/esphome/components/total_daily_energy/sensor.py

Line 50 in afbb900

cv.Optional(CONF_RESTORE, default=True): cv.boolean,

I'll do some testing (primarily to replicate the problem to confirm the reboots are causing the issue, so I can verify the final solution).

Looking at home-assistant code the reset detection does not rely on 0, but looks for more than 10% drop:
https://github.com/home-assistant/core/blob/c265d3f3ccf464c32154219369453974f13b5cd3/homeassistant/components/sensor/recorder.py#L340

Looks like the total_daily_energy platform does not reset the value daily, therefore a 2 days setback will soon be smaller than 10% drop. After debugging, I've found this happens due my firewall filters out ntp traffic. I changed the time source to home-assistant.
The relevant code which should reset the sensor each midnight (when there's proper time source) is here:

esphome/esphome/components/total_daily_energy/total_daily_energy.cpp

Line 35 in afbb900

if (t.day_of_year != this->last_day_of_year_) {

Still, restore=false sounds like a better solution. I even consider it should be the default for any sensors with total_increasing state class, especially when using flash_write_interval: "48h" which can revert to a saved value 1-2 days ago.

I'll follow-up once I tried the recommendations and confirmed the solution.

3 replies

tzapu Sep 22, 2023

~~i can confirm that with a big enough drop, something like 33% i did not see any problems in the home assistant energy graphs.~~

/edit i spoke too soon, the graphs did not update yet when i checked, i do have problems with a 33% drop on reset as well

i have moved to restore: false though to not have issues for smaller differences either.
i wonder if moving to method: left is desirable as well

peterg79 Sep 23, 2023
Author

restore: false solved the issue for me, I could not reproduce the symptom after that. Also, configuring the time source fixes the daily reset (see the attached chart). Thanks for all the help!

tzapu Sep 23, 2023

been running today with restore: false as well and kept resetting it. no adverse ill effects observed on home assistant graphs. nice

will run with : method: left after gathering data for a few days to see how it affects it

in photo, same solar array from the inverter meter and from the vue, almost identical

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Recorder errors #206

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 4 comments 8 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Recorder errors #206

peterg79 Sep 13, 2023

Replies: 4 comments · 8 replies

flaviut Sep 13, 2023 Maintainer

clowrey Sep 14, 2023

flaviut Sep 14, 2023 Maintainer

clowrey Sep 14, 2023

clowrey Sep 14, 2023

peterg79 Sep 14, 2023 Author

clowrey Sep 14, 2023

peterg79 Sep 14, 2023 Author

peterg79 Sep 14, 2023 Author

tzapu Sep 22, 2023

peterg79 Sep 23, 2023 Author

tzapu Sep 23, 2023

peterg79
Sep 13, 2023

Replies: 4 comments 8 replies

flaviut
Sep 13, 2023
Maintainer

clowrey
Sep 14, 2023

flaviut Sep 14, 2023
Maintainer

clowrey
Sep 14, 2023

peterg79 Sep 14, 2023
Author

peterg79 Sep 14, 2023
Author

peterg79
Sep 14, 2023
Author

peterg79 Sep 23, 2023
Author