Experimental ESP-IDF v5

[llange]

This PR is more or less ready for review.

It aims to bring compatibility with ESP-IDFv5, with the following caveats:

• Same limitations as the ESP-IDFv4 branch on which it is based, see Experimental ESP-IDF v4 #806
• Factory reset is absent (esp_flash_erase_region needs some work)
• Mongoose is not (yet) ready to compile with TLS enabled.
• There are still some warnings during compilation (mainly ADC which needs conversion + some others)
• wolfSSL can't be (yet) compiled with OPENSSL defines (see Compilation error when defining OPENSSL_ALL or OPENSSL_EXTRA on ESP-IDFv5 : ssl.c:18011:22: error: size of array 'sha_test' is negative wolfSSL/wolfssl#6028)

Please note that:

• wolfSSL has been updated to tag v5.5.4-stable, and needs a small patch to compile (see [Bug]: (sha512.c:1309:29: error: 'sha512' undeclared) wolfSSL/wolfssl#6024, [Bug]: esp32_sha.c:511:26: error: lvalue required as left operand of assignment wolfSSL/wolfssl#6026, ESP-IDFv5: Warning related to the uint32_t definition change wolfSSL/wolfssl#6029 fixed in master by ESP32 SHA384 fixes wolfSSL/wolfssl#6034)

diff --git a/wolfcrypt/src/port/Espressif/esp32_sha.c b/wolfcrypt/src/port/Espressif/esp32_sha.c
index 557bde8a9..0706fa7b1 100644
--- a/wolfcrypt/src/port/Espressif/esp32_sha.c
+++ b/wolfcrypt/src/port/Espressif/esp32_sha.c
@@ -507,8 +507,7 @@ int wc_esp_digest_state(WC_ESP32SHA* ctx, byte* hash)
 
     #if defined(WOLFSSL_SHA384)
         case SHA2_384:
-            SHA_LOAD_REG = SHA_384_LOAD_REG;
-            SHA_BUSY_REG = SHA_384_BUSY_REG;
+            DPORT_REG_WRITE(SHA_384_LOAD_REG, 1);
             break;
     #endif
 
@@ -551,7 +550,7 @@ int wc_esp_digest_state(WC_ESP32SHA* ctx, byte* hash)
      *    DPORT_SEQUENCE_REG_READ(address + i * 4);
      */
     esp_dport_access_read_buffer(
-        (word32*)(hash), /* the result will be found in hash upon exit     */
+        (uint32_t*)(hash), /* the result will be found in hash upon exit     */
         SHA_TEXT_BASE,   /* there's a fixed reg addy for all SHA           */
         wc_esp_sha_digest_size(ctx->sha_type) / sizeof(word32) /* # 4-byte */
     );
diff --git a/wolfcrypt/src/sha512.c b/wolfcrypt/src/sha512.c
index d42806976..deb60222d 100644
--- a/wolfcrypt/src/sha512.c
+++ b/wolfcrypt/src/sha512.c
@@ -1306,7 +1306,7 @@ static int InitSha384(wc_Sha384* sha384)
     sha384->ctx.isfirstblock = 1;
     if(sha384->ctx.mode == ESP32_SHA_HW) {
         /* release hw */
-        esp_sha_hw_unlock(&(sha512->ctx));
+        esp_sha_hw_unlock(&(sha384->ctx));
     }
     /* always set mode as INIT
     *  whether using HW or SW is determined at first call of update()

• wolfSSH has been updated to tag v1.4.6-stable
• mongoose has not been updated but needs the following patch:

diff --git a/mongoose.c b/mongoose.c
index b12cff18..60a7f62e 100644
--- a/mongoose.c
+++ b/mongoose.c
@@ -9160,7 +9160,7 @@ static void mg_send_file_data(struct mg_connection *nc, FILE *fp) {
 static void mg_do_ssi_include(struct mg_connection *nc, struct http_message *hm,
                               const char *ssi, char *tag, int include_level,
                               const struct mg_serve_http_opts *opts) {
-  char file_name[MG_MAX_PATH], path[MG_MAX_PATH], *p;
+  char file_name[MG_MAX_PATH], path[MG_MAX_PATH+2], *p;
   FILE *fp;
 
   /*
diff --git a/mongoose.h b/mongoose.h
index 3bcf8147..5649e1a7 100644
--- a/mongoose.h
+++ b/mongoose.h
@@ -1768,7 +1768,7 @@ typedef struct {
 
 void cs_md5_init(cs_md5_ctx *c);
 void cs_md5_update(cs_md5_ctx *c, const unsigned char *data, size_t len);
-void cs_md5_final(unsigned char *md, cs_md5_ctx *c);
+void cs_md5_final(unsigned char md[16], cs_md5_ctx *c);
 
 #ifdef __cplusplus
 }

To be able to "test" this PR, you'll need:

• An ESP-IDF v5 installation (from git) tested and configured, set to tag v5.0

Ideally this may need to be broken down for better review - note that you should be able to review commit per commit as I tried to keep them consistent and compilable.

[llange]

Cross-referencing #360 and #263

[jetpax]

Thanks, this is a great step forward, and works pretty well for me with esp-idf=5.0.1, given your caveats, as long as I have CONFIG_OVMS_COMP_CELLULAR=y in sdkconfig.

Without that, it fails to compile with

.../main/ovms_command.cpp:1249:15: error: aggregate 'stat st' has incomplete type and cannot be defined
 1249 |   struct stat st;

Not clear to me where this error is coming from, any ideas?

[llange]

Hello @jetpax,
Thank you very much for taking the time (and the risk...) to test this branch. I'm glad that you managed to have it work, and I look forward to your opening of many issue tickets so that we can improve it.
I confirm that CONFIG_OVMS_COMP_CELLULAR is y in my builds, and I'll check why it doesn't compile without it. Stay tuned !

[llange]

I added a missing header, which fixes the compilation. Tell me if it works for you !
(This branch is updated with the change, you can pull it to retrieve it)

[jetpax]

Thanks @llange , that works great, and system seems much more stable.

I do also get a compile error when I tried to turn off Duktape with CONFIG_OVMS_SC_JAVASCRIPT_NONE=y

In file included from /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/powermgmt/powermgmt_web.cpp:31:
/Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_metrics.h:168:29: error: 'OvmsWriter' was not declared in this scope
  168 | int OvmsMetricUnit_Validate(OvmsWriter* writer, int argc, const char* token, bool complete, metric_group_t group = GrpNone);

The reason I was doing that was that I had a crash that I though might have been memory allocation related when I tried to turn on CAN tcpserver network streaming to use SavvyCAN, (though maybe Mongoose related?)

In case it's of interest, here's the commands to replicate that crash:

OVMS> enable
Password:
Secure mode
OVMS# can can1 start listen 500000
Can bus can1 started in mode listen at speed 500000bps
I (193297) events: Signal(power.can1.on)
OVMS# can log start tcpserver discard gvret-b :23
CAN logging as TCP server: Type:tcpserver Format:gvret-b(discard) Vehicle:DEMO Path::23
I (205507) canlog-tcpserver: Launching TCP server at :23
I (273977) canlog-tcpserver: Log service connection from 192.168.1.153

Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x400fe49a  PS      : 0x00060630  A0      : 0x800fe514  A1      : 0x3fff89c0  
A2      : 0x3ffb5da0  A3      : 0x3fff89e0  A4      : 0x3ffff124  A5      : 0x3fff4430  
A6      : 0x00000003  A7      : 0x00060023  A8      : 0x00000000  A9      : 0x00000000  
A10     : 0x00000000  A11     : 0x00000001  A12     : 0x3ffb5da8  A13     : 0x00000001  
A14     : 0x3ff5f07c  A15     : 0x00000001  SAR     : 0x00000013  EXCCAUSE: 0x0000001c  
EXCVADDR: 0x00000010  LBEG    : 0x4008b86d  LEND    : 0x4008b87d  LCOUNT  : 0xfffffffc  


Backtrace: 0x400fe497:0x3fff89c0 0x400fe511:0x3fff89e0 0x401026ef:0x3fff8a20 0x4010283b:0x3fff8aa0 0x4013beb9:0x3fff8ac0 0x4013d21a:0x3fff8b00 0x4013d5da:0x3fff8b20 0x4013d8a2:0x3fff8b80 0x4013a70d:0x3fff8be0 0x400edb47:0x3fff8c00 0x400edbe5:0x3fff8c50

Here's the Stack Trace decode

0x400fe49a: canformat_gvret_binary::PopulateBusList12(gvret_replymsg_t*) at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/can/src/canformat_gvret.cpp:401
0x400fe497: canformat_gvret_binary::PopulateBusList12(gvret_replymsg_t*) at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/can/src/canformat_gvret.cpp:401
0x400fe511: canformat_gvret_binary::getheader[abi:cxx11](timeval*) at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/can/src/canformat_gvret.cpp:223
0x401026ef: canlog_tcpserver::MongooseHandler(mg_connection*, int, void*) at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/can/src/canlog_tcpserver.cpp:236
0x4010283b: tsMongooseHandler at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/can/src/canlog_tcpserver.cpp:129
0x4013beb9: mg_call at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:2368
0x4013d21a: mg_if_accept_tcp_cb at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:2764
0x4013d5da: mg_accept_conn at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:3578
0x4013d5da: mg_mgr_handle_conn at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:3842
0x4013d8a2: mg_socket_if_poll at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:4042
0x4013a70d: mg_mgr_poll at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:2538
0x400edb47: OvmsNetManager::MongooseTask() at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_netmanager.cpp:772
0x400edbe5: MongooseRawTask at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_netmanager.cpp:754

[llange]

Hi @jetpax , thanks for the bug report !
I'll try to reproduce it to see how we can mitigate the crash and hopefully fix it.

Regarding the compilation error with CONFIG_OVMS_SC_JAVASCRIPT_NONE=y ; I confirm it happens also on my side, and it's not that easy to fix at the moment. I'll keep you informed when I have a working solution.
Did you try this CONFIG_OVMS_SC_JAVASCRIPT_NONE=y with the (current) master branch ? Does it compile OK ?