[Vtigercrm-developers] How to import and overwrite data without changing record IDs
Uma S
uma.s at vtiger.com
Thu Sep 17 11:32:13 GMT 2020
Hi Alan,
Yes! We took an approach to review our cloud version and see if we can port
the import functionality, which resolves most of the issues we discussed
over here.
Right now, cloud supports 4 different options in Duplicate Handling as
noted below.
- Merge & Overwrite, skip blank values from CSV - If a mapped field has
blank value in the imported file, then existing value in CRM is retained
- Merge & Overwrite, apply any blank values from CSV - If a mapped field
has blank value in the imported file, then existing value in CRM will be
removed.
- Skip Duplicates - Duplicates entries in the file will not be imported
if a matching record is already in the CRM.
- Don't check Duplicates - All record from import file are imported
In addition to this unique field is supported during the field mapping
step. So that import is carried out successfully.
On Wed, Sep 16, 2020 at 5:32 PM Alan Lord <alanslists at gmail.com> wrote:
> FWIW, I think it would be helpful if vtiger discussed how this currently
> works in vtiger Cloud. In my experience it seems to work quite well there.
>
> Al
>
>
> On 16/09/2020 12:34, nilay khatri wrote:
> > This is what I think would be ideal:
> >
> > 1. In any case the duplicate records in the CRM should not be merged
> > 2. If the operation is merge then *update* all the matching records
> > fields, with the field values in the CSV file if the values in the CRM
> > record is empty or not equal to default value for that field
> > 3. If the operation is overwrite then *update* all the matching records
> > fields with the field values in CSV blindly
> >
> > Since we are not merging duplicate records in CRM into one single record
> > anymore it would also be possible to Map the auto-generated fields.
> >
> > Our team has started work on this and will share the progress soon.
> > Please feel free to share your thoughts.
> >
> > On Tue, Sep 15, 2020 at 2:57 PM Uma S <uma.s at vtiger.com
> > <mailto:uma.s at vtiger.com>> wrote:
> >
> > Hi Martin,
> >
> > That's a better idea, We need to improvise on the existing
> > options(Merge & overwrite) to import field values in the field
> > mapping phase for blank field values and add support to introduce
> > unique field in field mapping to resolve this issue.
> >
> > Thanks! for the insight Martin :) Updated your observation on the
> > issue <https://code.vtiger.com/vtiger/vtigercrm/issues/1483>.
> >
> >
> > On Tue, Sep 15, 2020 at 12:54 PM Martin Allen
> > <martin.allen at clystnet.com <mailto:martin.allen at clystnet.com>>
> wrote:
> >
> > I don't think you need to make the User choose between 'new' or
> > 'update' as then they'd potentially have to do two imports
> > instead of being able to do both in 1 step - just need to ensure
> > that the 'update' or 'merge' method is very explicit in how it
> > works (and does work correctly).
> > Adding the ability to select autoincrement fields as the
> > duplicate check (being the only true Unique ID field) is key to
> > making this work properly though. Being the only non-editable
> > field, it means you can then update ANY existing field even if
> > it's a field you would previously have used as a duplicate check
> > field! (Which you can currently do!)
> >
> > When the record already exists maybe, allowing the User to
> > specify per field (in the Field Mapping section) whether to
> > always use the supplied value (overwrite), or whichever one is
> > not blank (merge), and if both not blank then whether to use the
> > supplied value or the existing value?
> >
> > Martin Allen
> >
> > *01392 248692 - Main Office
> > 01392 690659 - Direct Line*
> >
> > *signature2*
> >
> > Have you visited our website recently? http://www.clystnet.com
> > <http://www.clystnet.com/>
> >
> >
> > The information in this email is confidential If you are not the
> > intended recipient, you must not read or use that information.
> > This email and any attachments are believed to be virus free
> > however no responsibility is accepted by Clystnet for any loss
> > or damage arising in any way from receipt or use thereof.
> > Clystnet Ltd (company reg number 7164503) is based at Silverdown
> > Park, Fair Oak Close, Clyst Honiton, EX5 2UX
> >
> >
> >
> > On Tue, 15 Sep 2020 at 07:37, Uma S <uma.s at vtiger.com
> > <mailto:uma.s at vtiger.com>> wrote:
> >
> > Thanks! for the Idea Ruben. Updated the details on the issue
> > <https://code.vtiger.com/vtiger/vtigercrm/issues/1483>.
> >
> > On Mon, Sep 14, 2020 at 9:38 PM Rubén A. Estrada Orozco
> > <rulotec1 at gmail.com <mailto:rulotec1 at gmail.com>> wrote:
> >
> > I agree, there should be an "update" option which only
> > updates fields in existing records but that doesn't
> > create new records. And the merge option that currently
> > exists should create records if they do not exist and
> > update them if they do (here we would just need to fix
> > the existing bugs).
> > Saludos
> >
> > Rubén
> >
> >
> > On Sat, Sep 12, 2020 at 12:12 PM Uma S <uma.s at vtiger.com
> > <mailto:uma.s at vtiger.com>> wrote:
> >
> > Hi Team,
> >
> > I think we should enhance the import functionality,
> > So that systems should first analyze the user's need
> > for import on either to update the existing records
> > or to replace the record (i.e., delete existing and
> > creating new one). Based on selected choice
> > consequent steps should be taken on records.
> >
> >
> >
> > On Fri, Sep 11, 2020 at 10:42 AM nilay khatri
> > <nilay.spartan at gmail.com
> > <mailto:nilay.spartan at gmail.com>> wrote:
> >
> > And thanks for the input everyone.
> >
> >
> >
> > On Fri, Sep 11, 2020 at 12:59 AM Rubén A.
> > Estrada Orozco <rulotec1 at gmail.com
> > <mailto:rulotec1 at gmail.com>> wrote:
> >
> > Naily, I've seen similar issues. When using
> > the merge feature for example, we noticed
> > that if you have a phone column in your csv
> > and its empty and if the record in
> > vtgier has already a phone, it gets deleted.
> > What you would expect from a merge is that
> > it would keep the non-empty field.
> >
> > I've opened these issues:
> > * merge function deletes fields:
> >
> https://code.vtiger.com/vtiger/vtigercrm/issues/1483
> > * add record number field for updating
> > purposes:
> >
> https://code.vtiger.com/vtiger/vtigercrm/issues/1484
> >
> > Saludos
> >
> > Rubén
> >
> >
> > On Thu, Sep 10, 2020 at 9:36 AM Martin Allen
> > <martin.allen at clystnet.com
> > <mailto:martin.allen at clystnet.com>> wrote:
> >
> > We've actually written our own import
> > scripts that will read a CSV file and
> > will update / create records via
> > Webservice, allowing you to do a check
> > for an existing record using whatever
> > fields you want first (and can identify
> > and handle if it returns multiple
> > records) and then update only the fields
> > you want to.
> >
> > Martin Allen
> >
> > *01392 248692 - Main Office
> > 01392 690659 - Direct Line*
> >
> > *signature2*
> >
> > Have you visited our website recently?
> > http://www.clystnet.com
> > <http://www.clystnet.com/>
> >
> >
> > The information in this email is
> > confidential If you are not the intended
> > recipient, you must not read or use that
> > information. This email and any
> > attachments are believed to be virus
> > free however no responsibility is
> > accepted by Clystnet for any loss or
> > damage arising in any way from receipt
> > or use thereof. Clystnet Ltd (company
> > reg number 7164503) is based at
> > Silverdown Park, Fair Oak Close, Clyst
> > Honiton, EX5 2UX
> >
> >
> >
> > On Thu, 10 Sep 2020 at 15:28, Martin
> > Allen <martin.allen at clystnet.com
> > <mailto:martin.allen at clystnet.com>>
> wrote:
> >
> > I believe that in vTiger hosted
> > version it is possible to have the
> > Lead Number (or any Auto Increment
> > field in any other module - UI Type
> > 4) field included in an Import which
> > can then be used for checking for
> > duplicates (obviously this field
> > value is not imported, only used for
> > duplicate checking).
> >
> > Would be good if this was available
> > in the open source version, as it
> > would make importing data (to update
> > records rather than creating new
> > ones) much simpler and safer, as any
> > other field cannot be guaranteed to
> > be unique, and sometimes the field
> > you want to update are the fields
> > that you would typically use to
> > identify duplicates (i.e. First
> > Name, Last Name, Email Address for
> > Contacts)
> >
> > Martin Allen
> >
> > *01392 248692 - Main Office
> > 01392 690659 - Direct Line*
> >
> > *signature2*
> >
> > Have you visited our website
> > recently? http://www.clystnet.com
> > <http://www.clystnet.com/>
> >
> >
> > The information in this email is
> > confidential If you are not the
> > intended recipient, you must not
> > read or use that information. This
> > email and any attachments are
> > believed to be virus free however no
> > responsibility is accepted by
> > Clystnet for any loss or damage
> > arising in any way from receipt or
> > use thereof. Clystnet Ltd (company
> > reg number 7164503) is based at
> > Silverdown Park, Fair Oak Close,
> > Clyst Honiton, EX5 2UX
> >
> >
> >
> > On Thu, 10 Sep 2020 at 13:22, nilay
> > khatri <nilay.spartan at gmail.com
> > <mailto:nilay.spartan at gmail.com>>
> wrote:
> >
> > This is a very common challenge,
> > users want to update certain
> > data of records in CRM by
> > importing a CSV.
> >
> > Let us say Leads, we want to
> > update Lead Status, so the CSV
> > sheet contains 2 columns Primary
> > Email and Status.
> >
> > If we import the CSV and set the
> > duplicate check criteria in step
> > 2 on Primary Email and import
> > following things would happen:
> >
> > - quite possible all the other
> > data in fields will be wiped out
> > - if the system contains more
> > than 1 Lead with same email id,
> > it would delete them, keeping
> > only 1
> > - the record IDs will change
> >
> > If I talk about contact module
> > that is full of surprises,
> > contact Id is available to
> > select in the duplicate
> > selection criteria, but not
> > available in the field mapping!
> >
> >
> --------------------------------------
> >
> >
> > In the end we want to achieve a
> > very simple thing, import a List
> > of Leads and update respective
> > status without changing record
> > Ids or deleting any other
> > matching record in system.
> >
> >
> >
> _______________________________________________
> > http://www.vtiger.com/
> >
> >
> _______________________________________________
> > http://www.vtiger.com/
> >
> >
> _______________________________________________
> > http://www.vtiger.com/
> >
> > _______________________________________________
> > http://www.vtiger.com/
> >
> >
> >
> > --
> > With
> > Best Regards
> > Uma.S
> > Vtiger Team
> > _______________________________________________
> > http://www.vtiger.com/
> >
> > _______________________________________________
> > http://www.vtiger.com/
> >
> >
> >
> > --
> > With
> > Best Regards
> > Uma.S
> > Vtiger Team
> > _______________________________________________
> > http://www.vtiger.com/
> >
> > _______________________________________________
> > http://www.vtiger.com/
> >
> >
> >
> > --
> > With
> > Best Regards
> > Uma.S
> > Vtiger Team
> > _______________________________________________
> > http://www.vtiger.com/
> >
> >
> > _______________________________________________
> > http://www.vtiger.com/
> >
> _______________________________________________
> http://www.vtiger.com/
--
With
Best Regards
Uma.S
Vtiger Team
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.vtigercrm.com/pipermail/vtigercrm-developers/attachments/20200917/1f2cd208/attachment-0001.html>
More information about the vtigercrm-developers
mailing list