[Vtigercrm-developers] How to import and overwrite data without changing record IDs
nilay khatri
nilay.spartan at gmail.com
Fri Sep 18 01:44:36 GMT 2020
Hi Uma,
coming back on this, does on the cloud version, the record IDs change if a
duplicate match is found?
On Thu, Sep 17, 2020 at 6:16 PM nilay khatri <nilay.spartan at gmail.com>
wrote:
> Okay, so if that is the case I would ask my team to stop the work on this
> and would wait to hear further from the Vtiger team.
>
> On Thu, Sep 17, 2020 at 5:04 PM Uma S <uma.s at vtiger.com> wrote:
>
>> Hi Alan,
>>
>> Yes! We took an approach to review our cloud version and see if we can
>> port the import functionality, which resolves most of the issues we
>> discussed over here.
>>
>> Right now, cloud supports 4 different options in Duplicate Handling as
>> noted below.
>>
>> - Merge & Overwrite, skip blank values from CSV - If a mapped field
>> has blank value in the imported file, then existing value in CRM is retained
>> - Merge & Overwrite, apply any blank values from CSV - If a mapped
>> field has blank value in the imported file, then existing value in CRM will
>> be removed.
>> - Skip Duplicates - Duplicates entries in the file will not be
>> imported if a matching record is already in the CRM.
>> - Don't check Duplicates - All record from import file are imported
>>
>> In addition to this unique field is supported during the field mapping
>> step. So that import is carried out successfully.
>>
>>
>> On Wed, Sep 16, 2020 at 5:32 PM Alan Lord <alanslists at gmail.com> wrote:
>>
>>> FWIW, I think it would be helpful if vtiger discussed how this currently
>>> works in vtiger Cloud. In my experience it seems to work quite well
>>> there.
>>>
>>> Al
>>>
>>>
>>> On 16/09/2020 12:34, nilay khatri wrote:
>>> > This is what I think would be ideal:
>>> >
>>> > 1. In any case the duplicate records in the CRM should not be merged
>>> > 2. If the operation is merge then *update* all the matching records
>>> > fields, with the field values in the CSV file if the values in the CRM
>>> > record is empty or not equal to default value for that field
>>> > 3. If the operation is overwrite then *update* all the matching
>>> records
>>> > fields with the field values in CSV blindly
>>> >
>>> > Since we are not merging duplicate records in CRM into one single
>>> record
>>> > anymore it would also be possible to Map the auto-generated fields.
>>> >
>>> > Our team has started work on this and will share the progress soon.
>>> > Please feel free to share your thoughts.
>>> >
>>> > On Tue, Sep 15, 2020 at 2:57 PM Uma S <uma.s at vtiger.com
>>> > <mailto:uma.s at vtiger.com>> wrote:
>>> >
>>> > Hi Martin,
>>> >
>>> > That's a better idea, We need to improvise on the existing
>>> > options(Merge & overwrite) to import field values in the field
>>> > mapping phase for blank field values and add support to introduce
>>> > unique field in field mapping to resolve this issue.
>>> >
>>> > Thanks! for the insight Martin :) Updated your observation on the
>>> > issue <https://code.vtiger.com/vtiger/vtigercrm/issues/1483>.
>>> >
>>> >
>>> > On Tue, Sep 15, 2020 at 12:54 PM Martin Allen
>>> > <martin.allen at clystnet.com <mailto:martin.allen at clystnet.com>>
>>> wrote:
>>> >
>>> > I don't think you need to make the User choose between 'new' or
>>> > 'update' as then they'd potentially have to do two imports
>>> > instead of being able to do both in 1 step - just need to
>>> ensure
>>> > that the 'update' or 'merge' method is very explicit in how it
>>> > works (and does work correctly).
>>> > Adding the ability to select autoincrement fields as the
>>> > duplicate check (being the only true Unique ID field) is key to
>>> > making this work properly though. Being the only non-editable
>>> > field, it means you can then update ANY existing field even if
>>> > it's a field you would previously have used as a duplicate
>>> check
>>> > field! (Which you can currently do!)
>>> >
>>> > When the record already exists maybe, allowing the User to
>>> > specify per field (in the Field Mapping section) whether to
>>> > always use the supplied value (overwrite), or whichever one is
>>> > not blank (merge), and if both not blank then whether to use
>>> the
>>> > supplied value or the existing value?
>>> >
>>> > Martin Allen
>>> >
>>> > *01392 248692 - Main Office
>>> > 01392 690659 - Direct Line*
>>> >
>>> > *signature2*
>>> >
>>> > Have you visited our website recently? http://www.clystnet.com
>>> > <http://www.clystnet.com/>
>>> >
>>> >
>>> > The information in this email is confidential If you are not
>>> the
>>> > intended recipient, you must not read or use that information.
>>> > This email and any attachments are believed to be virus free
>>> > however no responsibility is accepted by Clystnet for any loss
>>> > or damage arising in any way from receipt or use thereof.
>>> > Clystnet Ltd (company reg number 7164503) is based at
>>> Silverdown
>>> > Park, Fair Oak Close, Clyst Honiton, EX5 2UX
>>> >
>>> >
>>> >
>>> > On Tue, 15 Sep 2020 at 07:37, Uma S <uma.s at vtiger.com
>>> > <mailto:uma.s at vtiger.com>> wrote:
>>> >
>>> > Thanks! for the Idea Ruben. Updated the details on the
>>> issue
>>> > <https://code.vtiger.com/vtiger/vtigercrm/issues/1483>.
>>> >
>>> > On Mon, Sep 14, 2020 at 9:38 PM Rubén A. Estrada Orozco
>>> > <rulotec1 at gmail.com <mailto:rulotec1 at gmail.com>> wrote:
>>> >
>>> > I agree, there should be an "update" option which only
>>> > updates fields in existing records but that doesn't
>>> > create new records. And the merge option that currently
>>> > exists should create records if they do not exist and
>>> > update them if they do (here we would just need to fix
>>> > the existing bugs).
>>> > Saludos
>>> >
>>> > Rubén
>>> >
>>> >
>>> > On Sat, Sep 12, 2020 at 12:12 PM Uma S <
>>> uma.s at vtiger.com
>>> > <mailto:uma.s at vtiger.com>> wrote:
>>> >
>>> > Hi Team,
>>> >
>>> > I think we should enhance the import functionality,
>>> > So that systems should first analyze the user's
>>> need
>>> > for import on either to update the existing records
>>> > or to replace the record (i.e., delete existing and
>>> > creating new one). Based on selected choice
>>> > consequent steps should be taken on records.
>>> >
>>> >
>>> >
>>> > On Fri, Sep 11, 2020 at 10:42 AM nilay khatri
>>> > <nilay.spartan at gmail.com
>>> > <mailto:nilay.spartan at gmail.com>> wrote:
>>> >
>>> > And thanks for the input everyone.
>>> >
>>> >
>>> >
>>> > On Fri, Sep 11, 2020 at 12:59 AM Rubén A.
>>> > Estrada Orozco <rulotec1 at gmail.com
>>> > <mailto:rulotec1 at gmail.com>> wrote:
>>> >
>>> > Naily, I've seen similar issues. When using
>>> > the merge feature for example, we noticed
>>> > that if you have a phone column in your csv
>>> > and its empty and if the record in
>>> > vtgier has already a phone, it gets
>>> deleted.
>>> > What you would expect from a merge is that
>>> > it would keep the non-empty field.
>>> >
>>> > I've opened these issues:
>>> > * merge function deletes fields:
>>> >
>>> https://code.vtiger.com/vtiger/vtigercrm/issues/1483
>>> > * add record number field for updating
>>> > purposes:
>>> >
>>> https://code.vtiger.com/vtiger/vtigercrm/issues/1484
>>> >
>>> > Saludos
>>> >
>>> > Rubén
>>> >
>>> >
>>> > On Thu, Sep 10, 2020 at 9:36 AM Martin
>>> Allen
>>> > <martin.allen at clystnet.com
>>> > <mailto:martin.allen at clystnet.com>> wrote:
>>> >
>>> > We've actually written our own import
>>> > scripts that will read a CSV file and
>>> > will update / create records via
>>> > Webservice, allowing you to do a check
>>> > for an existing record using whatever
>>> > fields you want first (and can identify
>>> > and handle if it returns multiple
>>> > records) and then update only the
>>> fields
>>> > you want to.
>>> >
>>> > Martin Allen
>>> >
>>> > *01392 248692 - Main Office
>>> > 01392 690659 - Direct Line*
>>> >
>>> > *signature2*
>>> >
>>> > Have you visited our website recently?
>>> > http://www.clystnet.com
>>> > <http://www.clystnet.com/>
>>> >
>>> >
>>> > The information in this email is
>>> > confidential If you are not the
>>> intended
>>> > recipient, you must not read or use
>>> that
>>> > information. This email and any
>>> > attachments are believed to be virus
>>> > free however no responsibility is
>>> > accepted by Clystnet for any loss or
>>> > damage arising in any way from receipt
>>> > or use thereof. Clystnet Ltd (company
>>> > reg number 7164503) is based at
>>> > Silverdown Park, Fair Oak Close, Clyst
>>> > Honiton, EX5 2UX
>>> >
>>> >
>>> >
>>> > On Thu, 10 Sep 2020 at 15:28, Martin
>>> > Allen <martin.allen at clystnet.com
>>> > <mailto:martin.allen at clystnet.com>>
>>> wrote:
>>> >
>>> > I believe that in vTiger hosted
>>> > version it is possible to have the
>>> > Lead Number (or any Auto Increment
>>> > field in any other module - UI Type
>>> > 4) field included in an Import
>>> which
>>> > can then be used for checking for
>>> > duplicates (obviously this field
>>> > value is not imported, only used
>>> for
>>> > duplicate checking).
>>> >
>>> > Would be good if this was available
>>> > in the open source version, as it
>>> > would make importing data (to
>>> update
>>> > records rather than creating new
>>> > ones) much simpler and safer, as
>>> any
>>> > other field cannot be guaranteed to
>>> > be unique, and sometimes the field
>>> > you want to update are the fields
>>> > that you would typically use to
>>> > identify duplicates (i.e. First
>>> > Name, Last Name, Email Address for
>>> > Contacts)
>>> >
>>> > Martin Allen
>>> >
>>> > *01392 248692 - Main Office
>>> > 01392 690659 - Direct Line*
>>> >
>>> > *signature2*
>>> >
>>> > Have you visited our website
>>> > recently? http://www.clystnet.com
>>> > <http://www.clystnet.com/>
>>> >
>>> >
>>> > The information in this email is
>>> > confidential If you are not the
>>> > intended recipient, you must not
>>> > read or use that information. This
>>> > email and any attachments are
>>> > believed to be virus free however
>>> no
>>> > responsibility is accepted by
>>> > Clystnet for any loss or damage
>>> > arising in any way from receipt or
>>> > use thereof. Clystnet Ltd (company
>>> > reg number 7164503) is based at
>>> > Silverdown Park, Fair Oak Close,
>>> > Clyst Honiton, EX5 2UX
>>> >
>>> >
>>> >
>>> > On Thu, 10 Sep 2020 at 13:22, nilay
>>> > khatri <nilay.spartan at gmail.com
>>> > <mailto:nilay.spartan at gmail.com>>
>>> wrote:
>>> >
>>> > This is a very common
>>> challenge,
>>> > users want to update certain
>>> > data of records in CRM by
>>> > importing a CSV.
>>> >
>>> > Let us say Leads, we want to
>>> > update Lead Status, so the CSV
>>> > sheet contains 2 columns
>>> Primary
>>> > Email and Status.
>>> >
>>> > If we import the CSV and set
>>> the
>>> > duplicate check criteria in
>>> step
>>> > 2 on Primary Email and import
>>> > following things would happen:
>>> >
>>> > - quite possible all the other
>>> > data in fields will be wiped
>>> out
>>> > - if the system contains more
>>> > than 1 Lead with same email id,
>>> > it would delete them, keeping
>>> > only 1
>>> > - the record IDs will change
>>> >
>>> > If I talk about contact module
>>> > that is full of surprises,
>>> > contact Id is available to
>>> > select in the duplicate
>>> > selection criteria, but not
>>> > available in the field mapping!
>>> >
>>> >
>>> --------------------------------------
>>> >
>>> >
>>> > In the end we want to achieve a
>>> > very simple thing, import a
>>> List
>>> > of Leads and update respective
>>> > status without changing record
>>> > Ids or deleting any other
>>> > matching record in system.
>>> >
>>> >
>>> >
>>> _______________________________________________
>>> > http://www.vtiger.com/
>>> >
>>> >
>>> _______________________________________________
>>> > http://www.vtiger.com/
>>> >
>>> >
>>> _______________________________________________
>>> > http://www.vtiger.com/
>>> >
>>> > _______________________________________________
>>> > http://www.vtiger.com/
>>> >
>>> >
>>> >
>>> > --
>>> > With
>>> > Best Regards
>>> > Uma.S
>>> > Vtiger Team
>>> > _______________________________________________
>>> > http://www.vtiger.com/
>>> >
>>> > _______________________________________________
>>> > http://www.vtiger.com/
>>> >
>>> >
>>> >
>>> > --
>>> > With
>>> > Best Regards
>>> > Uma.S
>>> > Vtiger Team
>>> > _______________________________________________
>>> > http://www.vtiger.com/
>>> >
>>> > _______________________________________________
>>> > http://www.vtiger.com/
>>> >
>>> >
>>> >
>>> > --
>>> > With
>>> > Best Regards
>>> > Uma.S
>>> > Vtiger Team
>>> > _______________________________________________
>>> > http://www.vtiger.com/
>>> >
>>> >
>>> > _______________________________________________
>>> > http://www.vtiger.com/
>>> >
>>> _______________________________________________
>>> http://www.vtiger.com/
>>
>>
>>
>> --
>> With
>> Best Regards
>> Uma.S
>> Vtiger Team
>> _______________________________________________
>> http://www.vtiger.com/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.vtigercrm.com/pipermail/vtigercrm-developers/attachments/20200918/e5de21d4/attachment-0001.html>
More information about the vtigercrm-developers
mailing list