RTL-0.15 with CLN and clnrest.py got very slow

[ghost]

Not sure if this is a bug, but I used to run RTL with c-lightning-REST, which was working fine. Now I'm running RTL 0.15 with CLN's clnrest.py plugin and everything got very slow. "Getting node information" and "Initializing node data" take more than a minute to finish, although I'm running it on a faster machine.

[saubyk]

Hi @martinneustein do you have a very busy node with a lot of channels and routing traffic? If you do, have you ever done any maintenance to clean up your database?

[ghost]

IHi @saubyk, it has about 16 channels and is running CLBOSS. I did not do any DB maintenance, I looked for information what I can do and couldn't find anything. I compiled lightingd without Postgres support/it uses sqlite3.

[ghost]

@saubyk I VACUUMed and ANALYZEd the sqlite3 db, no change. Some calls just seem to take forever.

[4/10/2024, 4:00:55 AM] INFO: Peers => List Peers...

[4/10/2024, 4:01:00 AM] INFO: Network => Network Fee Rates Received for perkw.

[4/10/2024, 4:01:00 AM] INFO: OnChain => Funds List Received.

[4/10/2024, 4:01:00 AM] INFO: Onchain => Local Remote Balance.

[4/10/2024, 4:01:00 AM] INFO: Onchain => Onchain Balance Received.

[4/10/2024, 4:01:00 AM] INFO: Channels => Peer Channels List Received.

[4/10/2024, 4:01:00 AM] INFO: Network => Network Fee Rates Received for perkb.

[4/10/2024, 4:01:00 AM] INFO: Payments => List Payments...

[4/10/2024, 4:01:14 AM] INFO: Channels => Peer Channels List With Aliases Received.

[4/10/2024, 4:01:14 AM] INFO: Peers => Sorted Peers List Received.

[4/10/2024, 4:01:26 AM] INFO: Payments => Payments List with Memo Received.

[sha-265]

I have performance issues with CLNRest also, that I didn't have with c-lightning-REST.

Especially when trying to load the routing page, I get a 500 error:

[ShahanaFarooqui]

listforwards being slow with clnrest makes sense. As we were using pagination in c-lightning-REST which has been removed with migration. We can utilise CLN's pagination now.

[sha-265]

Since when there is CLN's pagination?

[ShahanaFarooqui]

It was added in CLN v23.11 PR ElementsProject/lightning#6753.

[ShahanaFarooqui]

Similar issue: #1443

[ShahanaFarooqui]

I could not reproduce this issue on Chrome or Firefox. On Chrome, processing 10,000 Forwarding History records took 287 ms, while 1000 Local Failed Transactions records took 240 ms. Firefox performed slightly better, with times of 231 ms and 222 ms, respectively.

Closing the issue now.