[Vtigercrm-developers] How to import and overwrite data without changing record IDs
Uma S
uma.s at vtiger.com
Sat Sep 19 17:20:35 GMT 2020
Hi Nilay,
In the Duplicate handling step if a user opted for merge records, Then
newly created record is retained to overwrite the data.
On Fri, Sep 18, 2020 at 7:18 AM nilay khatri <nilay.spartan at gmail.com>
wrote:
> Hi Uma,
>
> coming back on this, does on the cloud version, the record IDs change if a
> duplicate match is found?
>
> On Thu, Sep 17, 2020 at 6:16 PM nilay khatri <nilay.spartan at gmail.com>
> wrote:
>
>> Okay, so if that is the case I would ask my team to stop the work on this
>> and would wait to hear further from the Vtiger team.
>>
>> On Thu, Sep 17, 2020 at 5:04 PM Uma S <uma.s at vtiger.com> wrote:
>>
>>> Hi Alan,
>>>
>>> Yes! We took an approach to review our cloud version and see if we can
>>> port the import functionality, which resolves most of the issues we
>>> discussed over here.
>>>
>>> Right now, cloud supports 4 different options in Duplicate Handling as
>>> noted below.
>>>
>>> - Merge & Overwrite, skip blank values from CSV - If a mapped field
>>> has blank value in the imported file, then existing value in CRM is retained
>>> - Merge & Overwrite, apply any blank values from CSV - If a mapped
>>> field has blank value in the imported file, then existing value in CRM will
>>> be removed.
>>> - Skip Duplicates - Duplicates entries in the file will not be
>>> imported if a matching record is already in the CRM.
>>> - Don't check Duplicates - All record from import file are imported
>>>
>>> In addition to this unique field is supported during the field mapping
>>> step. So that import is carried out successfully.
>>>
>>>
>>> On Wed, Sep 16, 2020 at 5:32 PM Alan Lord <alanslists at gmail.com> wrote:
>>>
>>>> FWIW, I think it would be helpful if vtiger discussed how this
>>>> currently
>>>> works in vtiger Cloud. In my experience it seems to work quite well
>>>> there.
>>>>
>>>> Al
>>>>
>>>>
>>>> On 16/09/2020 12:34, nilay khatri wrote:
>>>> > This is what I think would be ideal:
>>>> >
>>>> > 1. In any case the duplicate records in the CRM should not be merged
>>>> > 2. If the operation is merge then *update* all the matching records
>>>> > fields, with the field values in the CSV file if the values in the
>>>> CRM
>>>> > record is empty or not equal to default value for that field
>>>> > 3. If the operation is overwrite then *update* all the matching
>>>> records
>>>> > fields with the field values in CSV blindly
>>>> >
>>>> > Since we are not merging duplicate records in CRM into one single
>>>> record
>>>> > anymore it would also be possible to Map the auto-generated fields.
>>>> >
>>>> > Our team has started work on this and will share the progress soon.
>>>> > Please feel free to share your thoughts.
>>>> >
>>>> > On Tue, Sep 15, 2020 at 2:57 PM Uma S <uma.s at vtiger.com
>>>> > <mailto:uma.s at vtiger.com>> wrote:
>>>> >
>>>> > Hi Martin,
>>>> >
>>>> > That's a better idea, We need to improvise on the existing
>>>> > options(Merge & overwrite) to import field values in the field
>>>> > mapping phase for blank field values and add support to introduce
>>>> > unique field in field mapping to resolve this issue.
>>>> >
>>>> > Thanks! for the insight Martin :) Updated your observation on the
>>>> > issue <https://code.vtiger.com/vtiger/vtigercrm/issues/1483>.
>>>> >
>>>> >
>>>> > On Tue, Sep 15, 2020 at 12:54 PM Martin Allen
>>>> > <martin.allen at clystnet.com <mailto:martin.allen at clystnet.com>>
>>>> wrote:
>>>> >
>>>> > I don't think you need to make the User choose between 'new'
>>>> or
>>>> > 'update' as then they'd potentially have to do two imports
>>>> > instead of being able to do both in 1 step - just need to
>>>> ensure
>>>> > that the 'update' or 'merge' method is very explicit in how it
>>>> > works (and does work correctly).
>>>> > Adding the ability to select autoincrement fields as the
>>>> > duplicate check (being the only true Unique ID field) is key
>>>> to
>>>> > making this work properly though. Being the only non-editable
>>>> > field, it means you can then update ANY existing field even if
>>>> > it's a field you would previously have used as a duplicate
>>>> check
>>>> > field! (Which you can currently do!)
>>>> >
>>>> > When the record already exists maybe, allowing the User to
>>>> > specify per field (in the Field Mapping section) whether to
>>>> > always use the supplied value (overwrite), or whichever one is
>>>> > not blank (merge), and if both not blank then whether to use
>>>> the
>>>> > supplied value or the existing value?
>>>> >
>>>> > Martin Allen
>>>> >
>>>> > *01392 248692 - Main Office
>>>> > 01392 690659 - Direct Line*
>>>> >
>>>> > *signature2*
>>>> >
>>>> > Have you visited our website recently?
>>>> http://www.clystnet.com
>>>> > <http://www.clystnet.com/>
>>>> >
>>>> >
>>>> > The information in this email is confidential If you are not
>>>> the
>>>> > intended recipient, you must not read or use that information.
>>>> > This email and any attachments are believed to be virus free
>>>> > however no responsibility is accepted by Clystnet for any loss
>>>> > or damage arising in any way from receipt or use thereof.
>>>> > Clystnet Ltd (company reg number 7164503) is based at
>>>> Silverdown
>>>> > Park, Fair Oak Close, Clyst Honiton, EX5 2UX
>>>> >
>>>> >
>>>> >
>>>> > On Tue, 15 Sep 2020 at 07:37, Uma S <uma.s at vtiger.com
>>>> > <mailto:uma.s at vtiger.com>> wrote:
>>>> >
>>>> > Thanks! for the Idea Ruben. Updated the details on the
>>>> issue
>>>> > <https://code.vtiger.com/vtiger/vtigercrm/issues/1483>.
>>>> >
>>>> > On Mon, Sep 14, 2020 at 9:38 PM Rubén A. Estrada Orozco
>>>> > <rulotec1 at gmail.com <mailto:rulotec1 at gmail.com>> wrote:
>>>> >
>>>> > I agree, there should be an "update" option which only
>>>> > updates fields in existing records but that doesn't
>>>> > create new records. And the merge option that
>>>> currently
>>>> > exists should create records if they do not exist and
>>>> > update them if they do (here we would just need to fix
>>>> > the existing bugs).
>>>> > Saludos
>>>> >
>>>> > Rubén
>>>> >
>>>> >
>>>> > On Sat, Sep 12, 2020 at 12:12 PM Uma S <
>>>> uma.s at vtiger.com
>>>> > <mailto:uma.s at vtiger.com>> wrote:
>>>> >
>>>> > Hi Team,
>>>> >
>>>> > I think we should enhance the import
>>>> functionality,
>>>> > So that systems should first analyze the user's
>>>> need
>>>> > for import on either to update the existing
>>>> records
>>>> > or to replace the record (i.e., delete existing
>>>> and
>>>> > creating new one). Based on selected choice
>>>> > consequent steps should be taken on records.
>>>> >
>>>> >
>>>> >
>>>> > On Fri, Sep 11, 2020 at 10:42 AM nilay khatri
>>>> > <nilay.spartan at gmail.com
>>>> > <mailto:nilay.spartan at gmail.com>> wrote:
>>>> >
>>>> > And thanks for the input everyone.
>>>> >
>>>> >
>>>> >
>>>> > On Fri, Sep 11, 2020 at 12:59 AM Rubén A.
>>>> > Estrada Orozco <rulotec1 at gmail.com
>>>> > <mailto:rulotec1 at gmail.com>> wrote:
>>>> >
>>>> > Naily, I've seen similar issues. When
>>>> using
>>>> > the merge feature for example, we noticed
>>>> > that if you have a phone column in your
>>>> csv
>>>> > and its empty and if the record in
>>>> > vtgier has already a phone, it gets
>>>> deleted.
>>>> > What you would expect from a merge is that
>>>> > it would keep the non-empty field.
>>>> >
>>>> > I've opened these issues:
>>>> > * merge function deletes fields:
>>>> >
>>>> https://code.vtiger.com/vtiger/vtigercrm/issues/1483
>>>> > * add record number field for updating
>>>> > purposes:
>>>> >
>>>> https://code.vtiger.com/vtiger/vtigercrm/issues/1484
>>>> >
>>>> > Saludos
>>>> >
>>>> > Rubén
>>>> >
>>>> >
>>>> > On Thu, Sep 10, 2020 at 9:36 AM Martin
>>>> Allen
>>>> > <martin.allen at clystnet.com
>>>> > <mailto:martin.allen at clystnet.com>>
>>>> wrote:
>>>> >
>>>> > We've actually written our own import
>>>> > scripts that will read a CSV file and
>>>> > will update / create records via
>>>> > Webservice, allowing you to do a check
>>>> > for an existing record using whatever
>>>> > fields you want first (and can
>>>> identify
>>>> > and handle if it returns multiple
>>>> > records) and then update only the
>>>> fields
>>>> > you want to.
>>>> >
>>>> > Martin Allen
>>>> >
>>>> > *01392 248692 - Main Office
>>>> > 01392 690659 - Direct Line*
>>>> >
>>>> > *signature2*
>>>> >
>>>> > Have you visited our website recently?
>>>> > http://www.clystnet.com
>>>> > <http://www.clystnet.com/>
>>>> >
>>>> >
>>>> > The information in this email is
>>>> > confidential If you are not the
>>>> intended
>>>> > recipient, you must not read or use
>>>> that
>>>> > information. This email and any
>>>> > attachments are believed to be virus
>>>> > free however no responsibility is
>>>> > accepted by Clystnet for any loss or
>>>> > damage arising in any way from receipt
>>>> > or use thereof. Clystnet Ltd (company
>>>> > reg number 7164503) is based at
>>>> > Silverdown Park, Fair Oak Close, Clyst
>>>> > Honiton, EX5 2UX
>>>> >
>>>> >
>>>> >
>>>> > On Thu, 10 Sep 2020 at 15:28, Martin
>>>> > Allen <martin.allen at clystnet.com
>>>> > <mailto:martin.allen at clystnet.com>>
>>>> wrote:
>>>> >
>>>> > I believe that in vTiger hosted
>>>> > version it is possible to have the
>>>> > Lead Number (or any Auto Increment
>>>> > field in any other module - UI
>>>> Type
>>>> > 4) field included in an Import
>>>> which
>>>> > can then be used for checking for
>>>> > duplicates (obviously this field
>>>> > value is not imported, only used
>>>> for
>>>> > duplicate checking).
>>>> >
>>>> > Would be good if this was
>>>> available
>>>> > in the open source version, as it
>>>> > would make importing data (to
>>>> update
>>>> > records rather than creating new
>>>> > ones) much simpler and safer, as
>>>> any
>>>> > other field cannot be guaranteed
>>>> to
>>>> > be unique, and sometimes the field
>>>> > you want to update are the fields
>>>> > that you would typically use to
>>>> > identify duplicates (i.e. First
>>>> > Name, Last Name, Email Address for
>>>> > Contacts)
>>>> >
>>>> > Martin Allen
>>>> >
>>>> > *01392 248692 - Main Office
>>>> > 01392 690659 - Direct Line*
>>>> >
>>>> > *signature2*
>>>> >
>>>> > Have you visited our website
>>>> > recently? http://www.clystnet.com
>>>> > <http://www.clystnet.com/>
>>>> >
>>>> >
>>>> > The information in this email is
>>>> > confidential If you are not the
>>>> > intended recipient, you must not
>>>> > read or use that information. This
>>>> > email and any attachments are
>>>> > believed to be virus free however
>>>> no
>>>> > responsibility is accepted by
>>>> > Clystnet for any loss or damage
>>>> > arising in any way from receipt or
>>>> > use thereof. Clystnet Ltd (company
>>>> > reg number 7164503) is based at
>>>> > Silverdown Park, Fair Oak Close,
>>>> > Clyst Honiton, EX5 2UX
>>>> >
>>>> >
>>>> >
>>>> > On Thu, 10 Sep 2020 at 13:22,
>>>> nilay
>>>> > khatri <nilay.spartan at gmail.com
>>>> > <mailto:nilay.spartan at gmail.com>>
>>>> wrote:
>>>> >
>>>> > This is a very common
>>>> challenge,
>>>> > users want to update certain
>>>> > data of records in CRM by
>>>> > importing a CSV.
>>>> >
>>>> > Let us say Leads, we want to
>>>> > update Lead Status, so the CSV
>>>> > sheet contains 2 columns
>>>> Primary
>>>> > Email and Status.
>>>> >
>>>> > If we import the CSV and set
>>>> the
>>>> > duplicate check criteria in
>>>> step
>>>> > 2 on Primary Email and import
>>>> > following things would happen:
>>>> >
>>>> > - quite possible all the other
>>>> > data in fields will be wiped
>>>> out
>>>> > - if the system contains more
>>>> > than 1 Lead with same email
>>>> id,
>>>> > it would delete them, keeping
>>>> > only 1
>>>> > - the record IDs will change
>>>> >
>>>> > If I talk about contact module
>>>> > that is full of surprises,
>>>> > contact Id is available to
>>>> > select in the duplicate
>>>> > selection criteria, but not
>>>> > available in the field
>>>> mapping!
>>>> >
>>>> >
>>>> --------------------------------------
>>>> >
>>>> >
>>>> > In the end we want to achieve
>>>> a
>>>> > very simple thing, import a
>>>> List
>>>> > of Leads and update respective
>>>> > status without changing record
>>>> > Ids or deleting any other
>>>> > matching record in system.
>>>> >
>>>> >
>>>> >
>>>> _______________________________________________
>>>> > http://www.vtiger.com/
>>>> >
>>>> >
>>>> _______________________________________________
>>>> > http://www.vtiger.com/
>>>> >
>>>> >
>>>> _______________________________________________
>>>> > http://www.vtiger.com/
>>>> >
>>>> >
>>>> _______________________________________________
>>>> > http://www.vtiger.com/
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > With
>>>> > Best Regards
>>>> > Uma.S
>>>> > Vtiger Team
>>>> > _______________________________________________
>>>> > http://www.vtiger.com/
>>>> >
>>>> > _______________________________________________
>>>> > http://www.vtiger.com/
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > With
>>>> > Best Regards
>>>> > Uma.S
>>>> > Vtiger Team
>>>> > _______________________________________________
>>>> > http://www.vtiger.com/
>>>> >
>>>> > _______________________________________________
>>>> > http://www.vtiger.com/
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > With
>>>> > Best Regards
>>>> > Uma.S
>>>> > Vtiger Team
>>>> > _______________________________________________
>>>> > http://www.vtiger.com/
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > http://www.vtiger.com/
>>>> >
>>>> _______________________________________________
>>>> http://www.vtiger.com/
>>>
>>>
>>>
>>> --
>>> With
>>> Best Regards
>>> Uma.S
>>> Vtiger Team
>>> _______________________________________________
>>> http://www.vtiger.com/
>>
>> _______________________________________________
> http://www.vtiger.com/
--
With
Best Regards
Uma.S
Vtiger Team
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.vtigercrm.com/pipermail/vtigercrm-developers/attachments/20200919/70db7ed5/attachment-0001.html>
More information about the vtigercrm-developers
mailing list