[Vtigercrm-developers] Thoughts about mass delete and APIs

[Sukhdev Mohan]

Hi,

A very specific use case happened to me and I wanted to share a thought about it. I’ve to import and delete large set of data (we talking about 100k fro accounts, contacts and some custom modules...). The reason I need to to delete doesn’t depend on me but there has been a change in logics. Vtiger’s default mass delete doesn’t handles more than 1000 records at a time, so you need to resort to some “hacks” (mark deleted = 1 in crmentity and then empty the recycle bin, even though that won’t delete all records, so once again you have to resort to SQL).
BUT there is a big big problem and a bug: vtiger_modtracker_details doesn’t get clean when you permanently delete a record (following the standard procedure obviously). I’m trying to come up with a query to delete the records from there since it alone weighs around a Gigabyte… 

IMO since you want to permanently delete a record, the CRM should delete the records from modtracker too, I can understand that you want to keep a trace but you can check if it’s the admin and it’s emptying the bin, then delete records from modtracker_* too. May be implement some constraints and foreign key mechanism, many default tables doesn’t have them.

In general there is a memory exhaustion problem, vtws_create|update handles about 5k elements before SO decides to kill them for requesting more memory: in average SO (linux) kills the process handling when it reaches 700mb of virtual memory and around 400mb of real. I saw there is caching in vows_create: can anybody explain me why it’s there?

Best Regards,
Sukhdev Mohan
———————————
Cel. (+39) 320 7020345
Email s.mohan at myti.it




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.vtigercrm.com/pipermail/vtigercrm-developers/attachments/20200114/7dcbb011/attachment.html>