[Vtigercrm-developers] migration of large database...

[Richard Hills - Technologywise]

Actually, I was unable to ever get a 6.0.0 to 6.1.0 upgrade to finish. 
Had it looking finished but the PBXmanager module was broken in some 
ways, I'm very curious about any progress you make with that if you are 
putting your time into it as we also have around 200k records in this 
module.

I second the requirement for command line processing of migration, this 
would make the system more reliable and a lot easier to log the output 
should the migration go badly / we want to read over all of those 
success reports. Great to hear you've managed to sort this for yourself. 
Is it in a format that you can imagine vtiger ever accepting it as a 
contribution? Not suggesting that you need to give away all of your hard 
work, but it'd be great to have and it's always good to see how far they 
will go with accepting contribs.

On 20/11/14 20:51, Alan Lord (News) wrote:
> On 19/11/14 21:53, Richard Hills - Technologywise wrote:
>> Yes!
>>
>> I have been using Alan's workaround script in order to be able to get
>> our system to upgrade to 6.0.0 as a step in our attempts to move to
>> 6.1.0 however we are yet to see a 6.1.0 migration complete and I would
>> not be surprised if this is the cause (it was the cause of us not being
>> able to update to 6.0.0 earlier..). Our crmentity table is around 3m
>> entries and our server is unfortunately not as nice sounding as Alan's.
>
> There are two particularly bad bits of coding in the current migration 
> scripts:
>
> 1. The Label populator routine, which although works is massively 
> inefficient and slow. (For 5.4.0 you can circumvent it by adding the 
> label column and filling it with something, e.g. a dash '-', before 
> running the migration. For 600_to_610 though it runs irrespectively 
> and you have to comment out the code before starting the migration 
> process.)
>
> 2. The PBXManager code (See recent post "600_610.php migration 
> script") which actually does nothing at all!!! But takes an age to run 
> whilst doing it.
>
> I would also really like to be able to run these migration scripts 
> from the command line...
>
> Using a browser is hopeless when it's a big database. You wait hours, 
> getting no feedback, and then something stops or breaks and you have 
> no idea what happened.
>
> I've hacked one script to work from the cli and it is much better - 
> you get feedback as each routine runs, you can record the output to a 
> text file for later study... You could, if you wanted to, break the 
> migration into stages.
>
>> Hoping that some time can be spent on this by the vtiger team.
>
> Me too.
>
> Al
>
>

-- 

Richard Hills
TechnologyWise Ltd, Tauranga, NZ
richard at tw.co.nz
www.technologywise.co.nz
ph: +64 (0)7 571 1060
fax: +64 (0)7 571 1061