[Vtigercrm-developers] How to import and overwrite data without changing record IDs
nilay khatri
nilay.spartan at gmail.com
Thu Sep 17 12:46:51 GMT 2020
Okay, so if that is the case I would ask my team to stop the work on this
and would wait to hear further from the Vtiger team.
On Thu, Sep 17, 2020 at 5:04 PM Uma S <uma.s at vtiger.com> wrote:
> Hi Alan,
>
> Yes! We took an approach to review our cloud version and see if we can
> port the import functionality, which resolves most of the issues we
> discussed over here.
>
> Right now, cloud supports 4 different options in Duplicate Handling as
> noted below.
>
> - Merge & Overwrite, skip blank values from CSV - If a mapped field
> has blank value in the imported file, then existing value in CRM is retained
> - Merge & Overwrite, apply any blank values from CSV - If a mapped
> field has blank value in the imported file, then existing value in CRM will
> be removed.
> - Skip Duplicates - Duplicates entries in the file will not be
> imported if a matching record is already in the CRM.
> - Don't check Duplicates - All record from import file are imported
>
> In addition to this unique field is supported during the field mapping
> step. So that import is carried out successfully.
>
>
> On Wed, Sep 16, 2020 at 5:32 PM Alan Lord <alanslists at gmail.com> wrote:
>
>> FWIW, I think it would be helpful if vtiger discussed how this currently
>> works in vtiger Cloud. In my experience it seems to work quite well
>> there.
>>
>> Al
>>
>>
>> On 16/09/2020 12:34, nilay khatri wrote:
>> > This is what I think would be ideal:
>> >
>> > 1. In any case the duplicate records in the CRM should not be merged
>> > 2. If the operation is merge then *update* all the matching records
>> > fields, with the field values in the CSV file if the values in the CRM
>> > record is empty or not equal to default value for that field
>> > 3. If the operation is overwrite then *update* all the matching records
>> > fields with the field values in CSV blindly
>> >
>> > Since we are not merging duplicate records in CRM into one single
>> record
>> > anymore it would also be possible to Map the auto-generated fields.
>> >
>> > Our team has started work on this and will share the progress soon.
>> > Please feel free to share your thoughts.
>> >
>> > On Tue, Sep 15, 2020 at 2:57 PM Uma S <uma.s at vtiger.com
>> > <mailto:uma.s at vtiger.com>> wrote:
>> >
>> > Hi Martin,
>> >
>> > That's a better idea, We need to improvise on the existing
>> > options(Merge & overwrite) to import field values in the field
>> > mapping phase for blank field values and add support to introduce
>> > unique field in field mapping to resolve this issue.
>> >
>> > Thanks! for the insight Martin :) Updated your observation on the
>> > issue <https://code.vtiger.com/vtiger/vtigercrm/issues/1483>.
>> >
>> >
>> > On Tue, Sep 15, 2020 at 12:54 PM Martin Allen
>> > <martin.allen at clystnet.com <mailto:martin.allen at clystnet.com>>
>> wrote:
>> >
>> > I don't think you need to make the User choose between 'new' or
>> > 'update' as then they'd potentially have to do two imports
>> > instead of being able to do both in 1 step - just need to ensure
>> > that the 'update' or 'merge' method is very explicit in how it
>> > works (and does work correctly).
>> > Adding the ability to select autoincrement fields as the
>> > duplicate check (being the only true Unique ID field) is key to
>> > making this work properly though. Being the only non-editable
>> > field, it means you can then update ANY existing field even if
>> > it's a field you would previously have used as a duplicate check
>> > field! (Which you can currently do!)
>> >
>> > When the record already exists maybe, allowing the User to
>> > specify per field (in the Field Mapping section) whether to
>> > always use the supplied value (overwrite), or whichever one is
>> > not blank (merge), and if both not blank then whether to use the
>> > supplied value or the existing value?
>> >
>> > Martin Allen
>> >
>> > *01392 248692 - Main Office
>> > 01392 690659 - Direct Line*
>> >
>> > *signature2*
>> >
>> > Have you visited our website recently? http://www.clystnet.com
>> > <http://www.clystnet.com/>
>> >
>> >
>> > The information in this email is confidential If you are not the
>> > intended recipient, you must not read or use that information.
>> > This email and any attachments are believed to be virus free
>> > however no responsibility is accepted by Clystnet for any loss
>> > or damage arising in any way from receipt or use thereof.
>> > Clystnet Ltd (company reg number 7164503) is based at Silverdown
>> > Park, Fair Oak Close, Clyst Honiton, EX5 2UX
>> >
>> >
>> >
>> > On Tue, 15 Sep 2020 at 07:37, Uma S <uma.s at vtiger.com
>> > <mailto:uma.s at vtiger.com>> wrote:
>> >
>> > Thanks! for the Idea Ruben. Updated the details on the issue
>> > <https://code.vtiger.com/vtiger/vtigercrm/issues/1483>.
>> >
>> > On Mon, Sep 14, 2020 at 9:38 PM Rubén A. Estrada Orozco
>> > <rulotec1 at gmail.com <mailto:rulotec1 at gmail.com>> wrote:
>> >
>> > I agree, there should be an "update" option which only
>> > updates fields in existing records but that doesn't
>> > create new records. And the merge option that currently
>> > exists should create records if they do not exist and
>> > update them if they do (here we would just need to fix
>> > the existing bugs).
>> > Saludos
>> >
>> > Rubén
>> >
>> >
>> > On Sat, Sep 12, 2020 at 12:12 PM Uma S <
>> uma.s at vtiger.com
>> > <mailto:uma.s at vtiger.com>> wrote:
>> >
>> > Hi Team,
>> >
>> > I think we should enhance the import functionality,
>> > So that systems should first analyze the user's need
>> > for import on either to update the existing records
>> > or to replace the record (i.e., delete existing and
>> > creating new one). Based on selected choice
>> > consequent steps should be taken on records.
>> >
>> >
>> >
>> > On Fri, Sep 11, 2020 at 10:42 AM nilay khatri
>> > <nilay.spartan at gmail.com
>> > <mailto:nilay.spartan at gmail.com>> wrote:
>> >
>> > And thanks for the input everyone.
>> >
>> >
>> >
>> > On Fri, Sep 11, 2020 at 12:59 AM Rubén A.
>> > Estrada Orozco <rulotec1 at gmail.com
>> > <mailto:rulotec1 at gmail.com>> wrote:
>> >
>> > Naily, I've seen similar issues. When using
>> > the merge feature for example, we noticed
>> > that if you have a phone column in your csv
>> > and its empty and if the record in
>> > vtgier has already a phone, it gets deleted.
>> > What you would expect from a merge is that
>> > it would keep the non-empty field.
>> >
>> > I've opened these issues:
>> > * merge function deletes fields:
>> >
>> https://code.vtiger.com/vtiger/vtigercrm/issues/1483
>> > * add record number field for updating
>> > purposes:
>> >
>> https://code.vtiger.com/vtiger/vtigercrm/issues/1484
>> >
>> > Saludos
>> >
>> > Rubén
>> >
>> >
>> > On Thu, Sep 10, 2020 at 9:36 AM Martin Allen
>> > <martin.allen at clystnet.com
>> > <mailto:martin.allen at clystnet.com>> wrote:
>> >
>> > We've actually written our own import
>> > scripts that will read a CSV file and
>> > will update / create records via
>> > Webservice, allowing you to do a check
>> > for an existing record using whatever
>> > fields you want first (and can identify
>> > and handle if it returns multiple
>> > records) and then update only the fields
>> > you want to.
>> >
>> > Martin Allen
>> >
>> > *01392 248692 - Main Office
>> > 01392 690659 - Direct Line*
>> >
>> > *signature2*
>> >
>> > Have you visited our website recently?
>> > http://www.clystnet.com
>> > <http://www.clystnet.com/>
>> >
>> >
>> > The information in this email is
>> > confidential If you are not the intended
>> > recipient, you must not read or use that
>> > information. This email and any
>> > attachments are believed to be virus
>> > free however no responsibility is
>> > accepted by Clystnet for any loss or
>> > damage arising in any way from receipt
>> > or use thereof. Clystnet Ltd (company
>> > reg number 7164503) is based at
>> > Silverdown Park, Fair Oak Close, Clyst
>> > Honiton, EX5 2UX
>> >
>> >
>> >
>> > On Thu, 10 Sep 2020 at 15:28, Martin
>> > Allen <martin.allen at clystnet.com
>> > <mailto:martin.allen at clystnet.com>>
>> wrote:
>> >
>> > I believe that in vTiger hosted
>> > version it is possible to have the
>> > Lead Number (or any Auto Increment
>> > field in any other module - UI Type
>> > 4) field included in an Import which
>> > can then be used for checking for
>> > duplicates (obviously this field
>> > value is not imported, only used for
>> > duplicate checking).
>> >
>> > Would be good if this was available
>> > in the open source version, as it
>> > would make importing data (to update
>> > records rather than creating new
>> > ones) much simpler and safer, as any
>> > other field cannot be guaranteed to
>> > be unique, and sometimes the field
>> > you want to update are the fields
>> > that you would typically use to
>> > identify duplicates (i.e. First
>> > Name, Last Name, Email Address for
>> > Contacts)
>> >
>> > Martin Allen
>> >
>> > *01392 248692 - Main Office
>> > 01392 690659 - Direct Line*
>> >
>> > *signature2*
>> >
>> > Have you visited our website
>> > recently? http://www.clystnet.com
>> > <http://www.clystnet.com/>
>> >
>> >
>> > The information in this email is
>> > confidential If you are not the
>> > intended recipient, you must not
>> > read or use that information. This
>> > email and any attachments are
>> > believed to be virus free however no
>> > responsibility is accepted by
>> > Clystnet for any loss or damage
>> > arising in any way from receipt or
>> > use thereof. Clystnet Ltd (company
>> > reg number 7164503) is based at
>> > Silverdown Park, Fair Oak Close,
>> > Clyst Honiton, EX5 2UX
>> >
>> >
>> >
>> > On Thu, 10 Sep 2020 at 13:22, nilay
>> > khatri <nilay.spartan at gmail.com
>> > <mailto:nilay.spartan at gmail.com>>
>> wrote:
>> >
>> > This is a very common challenge,
>> > users want to update certain
>> > data of records in CRM by
>> > importing a CSV.
>> >
>> > Let us say Leads, we want to
>> > update Lead Status, so the CSV
>> > sheet contains 2 columns Primary
>> > Email and Status.
>> >
>> > If we import the CSV and set the
>> > duplicate check criteria in step
>> > 2 on Primary Email and import
>> > following things would happen:
>> >
>> > - quite possible all the other
>> > data in fields will be wiped out
>> > - if the system contains more
>> > than 1 Lead with same email id,
>> > it would delete them, keeping
>> > only 1
>> > - the record IDs will change
>> >
>> > If I talk about contact module
>> > that is full of surprises,
>> > contact Id is available to
>> > select in the duplicate
>> > selection criteria, but not
>> > available in the field mapping!
>> >
>> >
>> --------------------------------------
>> >
>> >
>> > In the end we want to achieve a
>> > very simple thing, import a List
>> > of Leads and update respective
>> > status without changing record
>> > Ids or deleting any other
>> > matching record in system.
>> >
>> >
>> >
>> _______________________________________________
>> > http://www.vtiger.com/
>> >
>> >
>> _______________________________________________
>> > http://www.vtiger.com/
>> >
>> >
>> _______________________________________________
>> > http://www.vtiger.com/
>> >
>> > _______________________________________________
>> > http://www.vtiger.com/
>> >
>> >
>> >
>> > --
>> > With
>> > Best Regards
>> > Uma.S
>> > Vtiger Team
>> > _______________________________________________
>> > http://www.vtiger.com/
>> >
>> > _______________________________________________
>> > http://www.vtiger.com/
>> >
>> >
>> >
>> > --
>> > With
>> > Best Regards
>> > Uma.S
>> > Vtiger Team
>> > _______________________________________________
>> > http://www.vtiger.com/
>> >
>> > _______________________________________________
>> > http://www.vtiger.com/
>> >
>> >
>> >
>> > --
>> > With
>> > Best Regards
>> > Uma.S
>> > Vtiger Team
>> > _______________________________________________
>> > http://www.vtiger.com/
>> >
>> >
>> > _______________________________________________
>> > http://www.vtiger.com/
>> >
>> _______________________________________________
>> http://www.vtiger.com/
>
>
>
> --
> With
> Best Regards
> Uma.S
> Vtiger Team
> _______________________________________________
> http://www.vtiger.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.vtigercrm.com/pipermail/vtigercrm-developers/attachments/20200917/7e3e9989/attachment-0001.html>
More information about the vtigercrm-developers
mailing list