[Vtigercrm-developers] slow csv export

[Adam Heinz]

I found a 100x performance improvement [1] in the csv export code today.
 Turns out, if you don't do 1-4 database calls for every one of the 16,000
customers you're exporting, you can get your export time down from 80
minutes to 30 seconds.  The sanitizeValues function [2] is pretty much
useless. It wastes a ton of CPU cycles calling fetchUserRole and
getSubordinateRoleAndUsers (via getAssignedPicklistValues). It populates
the picklistValues member variable, which is never used. It replaces uitype
10 crmids with entity names, which I like less than displaying the crmids,
exporting the related tables, and using Excel VLOOKUP to do the cross
referencing; essentially preserving referential integrity in your exported
spreadsheets.

[1]

Index: include/utils/export.php
===================================================================
--- include/utils/export.php	(revision 3285)
+++ include/utils/export.php	(revision 3943)
@@ -214,6 +214,6 @@
     while($val = $adb->fetchByAssoc($result, -1, false)){
		$new_arr = array();
-		$val = $__processor->sanitizeValues($val);
+// 		$val = $__processor->sanitizeValues($val);
 		foreach ($val as $key => $value){
 			if($type == 'Documents' && $key == 'description'){

[2]
http://trac.vtiger.com/cgi-bin/trac.cgi/browser/vtigercrm/branches/5.4.0/include/utils/export.php#L280
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.vtigercrm.com/pipermail/vtigercrm-developers/attachments/20130315/53501b40/attachment.html>