[Vtigercrm-developers] Speeding up web services sync script?

[Alex Hall]

I've thought about going directly to the tables, yes. My concern is that
web services do all the checks and other behind-the-scenes things I might
not know to do. I could update vtiger_contact, but what about the seemingly
unrelated table that may or may not get updated at the same time? I'd never
know exactly what steps are taken by web services, at least not without a
great deal of time spent tracing the web services code.

A CSV import is my other option. I assumed that 20,000 records would take
just as long with that method as with web services, since both take a
similar approach. At least, I'm guessing they do. Plus, I've not looked
into automating a CSV import, and I'm not sure if there's a way to handle
duplicates and such automatically. This would need to be a cron job, not
something I'd do manually. Do you think this would be a viable route for my
purposes?

On Mon, Apr 23, 2018 at 4:18 PM, Chris Thompson <cthompson at moderas.org>
wrote:

>
>
> Have you considered NOT coding this?  Seems to me a properly formatted
> export of your data could be imported into your application anynumberof
> ways and would be significantly faster and not bog your application down.
>
>
>
> *From: *<vtigercrm-developers-bounces at lists.vtigercrm.com> on behalf of
> Alex Hall <ahall at autodist.com>
> *Reply-To: *<vtigercrm-developers at lists.vtigercrm.com>
> *Date: *Monday, April 23, 2018 at 3:56 PM
> *To: *<vtigercrm-developers at lists.vtigercrm.com>
> *Subject: *[Vtigercrm-developers] Speeding up web services sync script?
>
>
>
> Hello all,
>
> I have a script in PHP that runs once a day, in the morning. This script
> queries the AS400 that stores all our company data, retrieving a list of
> all the organizations and their contacts. There are a little under 13,000
> of them, with anywhere from 0 to 2 contacts per organization. Call it
> 20,000 web service calls total. It then loops through these records, using
> web services to create or update each organization or contact in Vtiger.
> Obviously, this is slow.
>
> My question is whether there's anything I can do to speed it up. It takes
> hours now, and often times out before it completes all the records. I'm
> considering rewriting it in Python or another language that can handle
> multi-threading, but I don't know if that's the best way. Are there any
> ways to speed up web services? Any tips on using some method over another
> because that method is faster, for instance? Are there any server settings
> I might try that would give web services more resources to run faster?
> Thanks for any advice.
>
>
> --
>
> Alex Hall
>
> Automatic Distributors, IT department
>
> ahall at autodist.com
>
> _______________________________________________ http://www.vtiger.com/
>
> _______________________________________________
> http://www.vtiger.com/
>



-- 
Alex Hall
Automatic Distributors, IT department
ahall at autodist.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.vtigercrm.com/pipermail/vtigercrm-developers/attachments/20180423/ec5504f0/attachment.html>