Crash When Transmitting Data to Server-V2 During Wi-Fi Roaming

[kezarjg]

While sending data over a TCP connection using tcp_send(), the system crashes with a LoadProhibited exception on core 1. This issue occurred around the time the module is roaming from one Wi-Fi network to another, which may affect the state of the TCP connection.

The crash occurs when invoking the tcp_send() function in the following code:

OVMS.V3/components/mongoose/mongoose/mongoose.c:2776
len = nc->iface->vtable->tcp_send(nc, buf, len);

The crash occurred during the event ticker.1@ovms-server-v2.

[dexterbg]

When you see a crash, please always report the full crash debug info, especially the backtrace and, if undecoded, firmware version & source.

To decode the backtrace, use the support/a2l utility.

[kezarjg]

Last boot was 50102 second(s) ago
Time at boot: 2024-10-04 17:08:39 EDT
This is reset #3 since last power cycle
Detected boot reason: Crash (12/12)
Reset reason: Exception/panic (4)
Crash counters: 3 total, 0 early

Last crash: LoadProhibited exception on core 1
Registers:
PC : 0x401339f2 PS : 0x00060530 A0 : 0x801695e0 A1 : 0x3ffc3f00
A2 : 0x3f851bb0 A3 : 0x3f8e16f0 A4 : 0x000000ae A5 : 0x0000ff00
A6 : 0x00ff0000 A7 : 0xff000000 A8 : 0x00000081 A9 : 0x3039332c
A10 : 0x67005953 A11 : 0x41d9c016 A12 : 0x0000001f A13 : 0x67005953
A14 : 0x7ff00000 A15 : 0x7ff9c016 SAR : 0x00000001 EXCCAUSE: 0x0000001c
EXCVADDR: 0x000000a5 LBEG : 0x400029ac LEND : 0x400029cb LCOUNT : 0x00000000
Current task on core 0: IDLE0, 516 stack bytes free
Current task on core 1: OVMS Events, 4544 stack bytes free
Backtrace:
0x401339f2 0x401695dd 0x4016b665 0x4016da8e 0x401690c2 0x401190a2 0x40119271 0x40119358 0x401193e1
Event: ticker.1@ovms-server-v2 0 secs
Version: 3.3.004-321-gbb821cfc-dirty/ota_0/main (build idf v3.3.4-851-gfa4f07fb3 Sep 30 2024 17:27:30)

Hardware: OVMS WIFI BLE BT cores=2 rev=ESP32/3; MODEM SIM7600
Firmware: 3.3.004-321-gbb821cfc-dirty/ota_0/main (build idf v3.3.4-851-gfa4f07fb3 Sep 30 2024 17:27:30)
Running partition: ota_0
Boot partition: ota_0
Factory image: 3.3.001-33-g58d01654
OTA_O image: 3.3.004-321-gbb821cfc-dirty
OTA_1 image: 3.3.004-320-gc2ef512b-dirty

$ ../support/a2l /home/jerry/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/build/ovms3.elf 0x401339f2 0x401695dd 0x4016b665 0x4
016da8e 0x401690c2 0x401190a2 0x40119271 0x40119358 0x401193e1
Using elf file: /home/jerry/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/build/ovms3.elf
0x401339f2 is in mg_send (E:/OVMS/home/jerry/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:2776).
2771 size_t mg_send(struct mg_connection nc, const void buf, int len) {
2772 nc->last_io_time = (time_t) mg_time();
2773 if (nc->flags & MG_F_UDP) {
2774 len = nc->iface->vtable->udp_send(nc, buf, len);
2775 } else {
2776 len = nc->iface->vtable->tcp_send(nc, buf, len);
2777 }
2778
2779 if (!(nc->flags & MG_F_SEND_IMMEDIATELY) ||
2780 (nc->flags & MG_F_CLOSE_IMMEDIATELY) ||
0x401695dd is in OvmsServerV2::Transmit(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) (E:/OVMS/home/jerry/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_server_v2/src/ovms_server_v2.cpp:801).
796 RC4_crypt(&m_crypto_tx1, &m_crypto_tx2, (uint8_t)s, len);
797
798 char buf = new char[(len2)+4];
799 base64encode((uint8_t)s, len, (uint8_t*)buf);
800 strcat(buf,"\r\n");
801 mg_send(m_mgconn, buf, strlen(buf));
802
803 delete [] buf;
804 delete [] s;
805 return true;
0x4016b665 is in OvmsServerV2::TransmitMsgStat(bool) (E:/OVMS/home/jerry/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_server_v2/src/ovms_server_v2.cpp:1115).
1110 << ","
1111 << std::setprecision(1)
1112 << StandardMetrics.ms_v_bat_range_speed->AsFloat(0, units_speed)
1113 ;
1114
1115 Transmit(buffer.str().c_str());
1116 }
1117
1118 void OvmsServerV2::TransmitMsgGen(bool always)
1119 {
0x4016da8e is in OvmsServerV2::Ticker1(std::__cxx11::basic_string<char, std::char_traits, std::allocator >, void*) (E:/OVMS/home/jerry/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/ovms_server_v2/src/ovms_server_v2.cpp:2042).
2037 bool caron = StandardMetrics.ms_v_env_on->AsBool();
2038 int now = StandardMetrics.ms_m_monotonic->AsInt();
2039 int next = (m_peers==0) ? m_updatetime_idle : m_updatetime_connected;
2040 if ((m_lasttx==0)||(now>(m_lasttx+next)))
2041 {
2042 TransmitMsgStat(true); // Send always, periodically
2043 TransmitMsgEnvironment(true); // Send always, periodically
2044 TransmitMsgGPS(m_lasttx==0);
2045 TransmitMsgGroup(m_lasttx==0);
2046 TransmitMsgTPMS(m_lasttx==0);
0x401690c2 is in std::_Function_handler<void (std::__cxx11::basic_string<char, std::char_traits, std::allocator >, void*), std::_Bind<std::_Mem_fn<void (OvmsServerV2::)(std::__cxx11::basic_string<char, std::char_traits, std::allocator >, void)> (OvmsServerV2*, std::_Placeholder<1>, std::_Placeholder<2>)> >::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator >&&, void*&&) (e:\ovms\opt\xtensa-esp32-elf\xtensa-esp32-elf\include\c++\5.2.0/functional:600).
595 template<typename... _Args, typename _Req
596 = _Require<typename _Traits::__lvalue,
597 _CheckArgs<_Pack<_Args...>>>>
598 result_type
599 operator()(_Class* __object, _Args&&... __args) const
600 { return (__object->_M_pmf)(std::forward<_Args>(__args)...); }
601
602 // Handle smart pointers, references and pointers to derived
603 template<typename _Tp, typename... _Args, typename _Req
604 = _Require<_NotSame<_Class, _Tp>, _NotSame<_Class, _Tp>,
0x401190a2 is in std::function<void (std::__cxx11::basic_string<char, std::char_traits, std::allocator >, void*)>::operator()(std::__cxx11::basic_string<char, std::char_traits, std::allocator >, void*) const (e:\ovms\opt\xtensa-esp32-elf\xtensa-esp32-elf\include\c++\5.2.0/functional:2271).
2266 function<_Res(_ArgTypes...)>::
2267 operator()(_ArgTypes... __args) const
2268 {
2269 if (_M_empty())
2270 __throw_bad_function_call();
2271 return _M_invoker(_M_functor, std::forward<_ArgTypes>(__args)...);
2272 }
2273
2274 #if __cpp_rtti
2275 template<typename _Res, typename... _ArgTypes>
0x40119271 is in OvmsEvents::HandleQueueSignalEvent(event_queue_t*) (E:/OVMS/home/jerry/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:310).
305 {
306 for (EventCallbackList::iterator itc=el->begin(); itc!=el->end(); ++itc)
307 {
308 m_current_started = monotonictime;
309 m_current_callback = itc;
310 m_current_callback->m_callback(m_current_event, msg->body.signal.data);
311 m_current_callback = NULL;
312 }
313 }
314 }
0x40119358 is in OvmsEvents::EventTask() (E:/OVMS/home/jerry/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:262).
257 {
258 case EVENT_none:
259 break;
262 HandleQueueSignalEvent(&msg);
263 esp_task_wdt_reset(); // Reset WATCHDOG timer for this task
262 HandleQueueSignalEvent(&msg);
263 esp_task_wdt_reset(); // Reset WATCHDOG timer for this task
264 m_current_event.clear();
265 break;
266 default:
0x401193e1 is in EventLaunchTask(void) (E:/OVMS/home/jerry/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_events.cpp:89).
84
85 void EventLaunchTask(void pvParameters)
86 {
87 OvmsEvents me = (OvmsEvents*)pvParameters;
88
89 me->EventTask();
90 }
91
92 void event_trace(int verbosity, OvmsWriter* writer, OvmsCommand* cmd, int argc, const char* const* argv)
93 {

[kezarjg]

Disabling ovms-server-v2 has provided stability, with no crashes occurring even during Wi-Fi roaming. In this configuration, the application has been running smoothly for several days.

Additional network traffic is being sent through a JavaScript Plugin, which does not produce the same error, indicating that the issue may be specific to the ovms-server-v2 handling of TCP connections during network transitions.

I'm continuing to investigate, but could you clarify the intended mechanism for monitoring network connection status within the module?