PHP tracking first page reload is slow | Offer ability to not wait for response

[NicoHood]

I have noticed that on the first page reload of the day the website loading is slow. This is caused by the php tracker. Possible reasons:

• DNS resoltion
• Matomo instance is in idle mode

Could you give me a hint why this happens? What would be the best idea to fix this? I thought about using an async call, however I am not very familiar with async code on php.

[tsteur]

Hi @NicoHood I recommend you ask this question in our forum: https://forum.matomo.org/ Maybe someone will be able to help you there. We only manage bug reports and feature requests here but can't help with technical support for specific setups etc unfortunately. Thanks for your understanding. I'm not sure re DNS but maybe you can measure it if that's an issue? Maybe also check otherwise if there are any caches active and also be good to check if the slowness for the first page reload of the day is really because of Matomo or maybe something else?

[NicoHood]

Moved the issue here:
https://forum.matomo.org/t/php-tracking-first-page-reload-is-slow/40220

Any feedback appreciated!

[NicoHood]

I want to raise up this issue again:

I currently do not know why I got the delays sometimes, but I noticed that when the matomo instance is completely offline, the website will also not load. I noticed this while developing the website locally, but the matomo instance was on the www and we loss our internet connection.

So this means, if matomo is down for some reason, the php tracker will block the whole website, which is of course not, what we want. We need a fire and forget mechanism. Within the library itself, or something in my user code.

Now I am no expert here, but I've read that there are curl/socket options to use fire and forget. Maybe that is an option here? Since we have no return value anyways #85 that would make sense to me. What do you think?

https://stackoverflow.com/questions/36171222/async-curl-request-in-php/36171409#36171409
https://stackoverflow.com/questions/14587514/php-fire-and-forget-post-request

[OliverO2]

I second this. The issue should be renamed to something like "Delay or denial of service when tracking request is delayed or fails" and re-opened as a bug.

To me, the solution proposed in the updated OP of PHP Fire and Forget POST Request - Stack Overflow seems perfect. Fire-and-forget is the appropriate strategy here.

Using the current HTTP request/response mechanism asynchronously on extra threads would

• be unnecessarily resource-intensive without providing a clear benefit as there is not much that can be done when a response is missing or signals an error,
• require some sort of sophisticated timeout handling,
• introduce extra failure modes in high load scenarios such as excessive thread creation or thread starvation due to thread pool exhaustion.

[OliverO2]

Another note: The solution described in PHP Fire and Forget POST Request - Stack Overflow uses a timeout of 30 seconds when opening the connection. This would still slow down a web service more than is desirable if the tracking service does not accept the connection quickly enough.

[diosmosis]

@tsteur I think this is worth reopening, making requests non blocking in the php tracker could be rather useful.

[tsteur]

👍 reopened the issue and adjusted the title. It could be added as an option to not wait for the response if the system supports this feature. Some people would still want to wait for a result so we wouldn't change the default to not break the API etc and it's not clear whether more people would rather wait or not wait for the response. Also an opt-in solution will let us test this feature nicely and see how it goes etc.

Feel free to create a pull request if someone is keen to work on this. Be certainly valuable 👍

[teslina]

Since my production website went down only 3 hours after implementing the matomo-php-tracker 😱, I made a quick & dirty implementation of the async request. I won't make a pull request because as I said, the code is quick&dirty ;-) But maybe some would like to use it and go from there:

teslina@5208b3c

It's now working perfectly fine & smooth under heavy load.

A side note: I don't wanna use javascript nor non-async php requests at all, but I needed a visitorID. So I chose to generate the visitor ID directly on my website, and not requesting it from matomo.

You could start the async request like this:

    if( !isset($_SESSION['matomoVisitorID']) ){
     // Code from: 
     // core/Tracker/Visit.php ::  public static function generateUniqueVisitorId()
     //
     $_SESSION['matomoVisitorID'] = substr(bin2hex(random_bytes(16)), 0, 16);
    }


    $matomoTracker = new MatomoTracker((int)MATOMO_SITE_ID, MATOMO_URL);

    $matomoTracker->enableAsyncTracking();
    $matomoTracker->setTokenAuth(MATOMO_TOKEN);
    $matomoTracker->setVisitorId($_SESSION['matomoVisitorID']);
    $matomoPageTitle = '';
    $matomoTracker->doTrackPageView($matomoPageTitle);

[canonex]

I experience the same issue: I share two screeshots of kCacheGrind.

This is a complex query and Matomo tracker is 11% of the total:

In this medium/easy query that become almost 20% as the curl execution time is the same:

For very easy request 30% it is reached.
It makes the whole system very slow and I am struggling to continue using this functionality

Thank you,
Riccardo

[tsteur]

Thanks for this @NicoHood Feel free to issue a PR @NicoHood if any possible and we'll be happy to merge. Not sure it can be done easily in the tracker.

Or if this would need to be done around https://github.com/matomo-org/matomo/blob/4.x-dev/piwik.php#L15 to flush the output and end the connection early if a specific URL parameter is present. Might be worth a try and not too complicated to implement.

[NicoHood]

I dont think that is is a good solution. If the dns response of the matomo server is slow, this is still an issue. And for my usecase, I think that is the cause.

[canonex]

If the dns response of the matomo server is slow

I can't test it now but, in cases where a static ip is available, a solution like the one proposed here, may solve the issue?

[ghost]

Please be careful with the "fire and forget POST request" that was posted above, because it only works for insecure port 80 connections and does not work for HTTPS.

I'm talking about the solution that uses fsockopen/fwrite. In my case, port 80 redirects to port 443, as per PCI specification requirements.

I came here because of bug #105, which I solved by removing the exception at line 1904 and replaced it with a simple return false;.