[Vtigercrm-developers] Speeding up web services sync script?

[Brian]

Hi Alex,

We've done something similar but in the reverse direction.  We set up 
triggers on the AS400 files and then used java ODBC drivers to update 
the vTiger tables when the AS400 tables are updated.

Since you have the web services piece built, perhaps a similar solution 
would be to use triggers to output changed data or records to some 
alternate AS400 file and then use that as your input to the web service 
call.  That way you are only updating the records and data that need 
updated versus updating your entire set up 13000 records?

On 4/23/2018 3:54 PM, Alex Hall wrote:
> Hello all,
> I have a script in PHP that runs once a day, in the morning. This 
> script queries the AS400 that stores all our company data, retrieving 
> a list of all the organizations and their contacts. There are a little 
> under 13,000 of them, with anywhere from 0 to 2 contacts per 
> organization. Call it 20,000 web service calls total. It then loops 
> through these records, using web services to create or update each 
> organization or contact in Vtiger. Obviously, this is slow.
>
> My question is whether there's anything I can do to speed it up. It 
> takes hours now, and often times out before it completes all the 
> records. I'm considering rewriting it in Python or another language 
> that can handle multi-threading, but I don't know if that's the best 
> way. Are there any ways to speed up web services? Any tips on using 
> some method over another because that method is faster, for instance? 
> Are there any server settings I might try that would give web services 
> more resources to run faster? Thanks for any advice.
>
> -- 
> Alex Hall
> Automatic Distributors, IT department
> ahall at autodist.com <mailto:ahall at autodist.com>
>
>
> _______________________________________________
> http://www.vtiger.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.vtigercrm.com/pipermail/vtigercrm-developers/attachments/20180423/18116d62/attachment-0001.html>