[Vtigercrm-developers] "Failed - Network error" when downloading report in CSV format

[Martin Allen]

We are seeing the same issue when exporting a CSV report – which only contains 22 records  with a single query (that does not involve any permissions) and PHP has 512mb memory limit….again exporting as Excel works fine…

From: vtigercrm-developers-bounces at lists.vtigercrm.com <vtigercrm-developers-bounces at lists.vtigercrm.com> On Behalf Of nilay khatri
Sent: 22 July 2021 07:23
To: Vtiger Developers List <vtigercrm-developers at lists.vtigercrm.com>
Subject: Re: [Vtigercrm-developers] "Failed - Network error" when downloading report in CSV format

@Ruben, also increase memory limit if you are exporting in CSV format.

On Thu, Jul 22, 2021 at 11:46 AM nilay khatri <nilay.spartan at gmail.com<mailto:nilay.spartan at gmail.com>> wrote:
One more thing, the CSV output generation is not optimized, you can not store the entire resultset into an array and then pipe to a CSV file.

On Thu, Jul 22, 2021 at 11:43 AM nilay khatri <nilay.spartan at gmail.com<mailto:nilay.spartan at gmail.com>> wrote:
Uma, for this you really need to test Vtiger with a large data set where scalability becomes an issue. There are many issues similar to this, which can not be addressed with the current architecture of Vtiger. I am eager to hear that I am wrong.

- Mass edits
- Report export
- Export from List
- Imports

When you work with a large data set, there is no foolproof/predictable way to scale.

For example, this given case of report export. Suppose we had put conditions which involve full scan and is a non-admin with sharing rules etc, the query becomes really complex and can easily go for more than 5-10 minutes (even for small data sets, let us say 30-40K records as well).

Generally php time limits are set to less than 1 min. But anyways you can predefine what could be the time required to execute the various queries. So you might have to leave the configuration on the higher side, which would be a workaround but would again fail if something needs even more resources/time. And it is not good to leave 1 GB memory and 30 min of execution time limit :)

Another challenge is that if you put your CRM behind a OSI layer 3 firewall / filtering, these normally have very short TTFB limits, usually < 100 sec. Which again terminates the request and can lead to the above situation.

That's the same case where there are no event triggers for mass edits. As a workaround Vtiger decided to introduce bulk mode to simply skip events and limit the record selection to 500! Though there is a concept of batch event, but never implemented.


On Thu, Jul 22, 2021 at 11:08 AM Uma S <uma.s at vtiger.com<mailto:uma.s at vtiger.com>> wrote:
Hi Ruben,

I validated this case of Export CSV in Reports, which works perfectly fine.

Can you please be more precise on the case scenario, Did you notice any errors logs and what is the size of your report? I mean the number of records?

On Thu, Jul 22, 2021 at 8:25 AM Rubén A. Estrada Orozco <rulotec1 at gmail.com<mailto:rulotec1 at gmail.com>> wrote:
I'm having and issue where downloading any report in CSV format gives this error:

[cid:image001.png at 01D77ED8.4B6FE8E0]

It doesn't happen when using Excel format.

It happened in vtiger 7.0.1 and still happens after upgrading to 7.4

Any ideas?

Saludos

Rubén
_______________________________________________
http://www.vtiger.com/


--
With
Best Regards
Uma.S
Vtiger Team
_______________________________________________
http://www.vtiger.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.vtigercrm.com/pipermail/vtigercrm-developers/attachments/20210722/d2890c59/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 5478 bytes
Desc: image001.png
URL: <http://lists.vtigercrm.com/pipermail/vtigercrm-developers/attachments/20210722/d2890c59/attachment-0001.png>