[Vtigercrm-developers] Email marketing scenario

Fri Aug 12 14:13:43 GMT 2016

Hi Błażej,

On 27/07/16 08:56, Błażej Pabiszczak wrote:
>
> @Alan
>
> for me this is just a discussion, if you prefer switching to emails 
> then I have no problem with that [though this is vtiger engine problem 
> so the list is the perfect place to discuss it]. I don't intend to 
> anyhow reduce the value of your module, it's a module like every other 
> module, I only want to talk about the logic, and point out some 
> problems...
>
> 1.
>
>     We have a module to generate random data [as far as I remember it
>     should support generating data for vtiger by default as well], if
>     you have any problems generating the data, I'll gladly send you
>     the module.
>
sounds interesting, yes that would be helpful, thanks.
>
> 1.
>
>
> 2.
>
>     I don't know how your solution works, but if it's based on
>     Full-Text Search, then you won't get these data, for us it's a
>     significant limitation for a search tool.
>
it runs on the various events, create/save/delete and updates the 
corresponding document in the full text server. The full text server 
basically gets given the column_values array as key value pairs.
>
> 1.
>
>
> 2.
>
>     So your search is not very useful when it comes to large
>     databases? When you send queries to the database, how many records
>     do you get [50, 100, 1000?], if you for example get 100 records,
>     and the system takes away the permissions on all records [because
>     the record that the user can access is on the 2000^th place], then
>     will you show the user that no data was found? The History of
>     Changes widget on the homepage works like that right now – the
>     bigger the base and the less data the user sees, the more often he
>     sees nothing at all in the history of changes. If the mechanism
>     works in the way I described [and I hope I'm wrong] then it has no
>     business value because it falsifies the result.
>
we keep getting more data until we run out of results or fill the page 
we want to send to the user, so if they asked for 100 records we might 
look through 500 until we manage to get the top 100 that they can see.
>
> 1.
>
>
> As far as large number of users and test data are concerned, I'm 
> trying to show you that it doesn't matter if your search engine works 
> fast on large databases or not, because Vtiger as a whole doesn't work 
> well with large databases. I'll give you an example: in what way does 
> vtiger transfer data to select type fields? It transfers data by 
> sending html, instead of sending queries at AJAX database level and 
> sending the users list directly there. Additionally, it uses Chosen 
> instead of Select2; Select 2 is five times faster when it comes to 
> large amounts of data [eg 10,000]. When you load 20,000 users in 
> vtiger then one field will take SEVERAL SECONDS to load. There are 
> hundreds of examples like this, and they result from the fact that 
> nobody has deployed this system for a large company [I don't mean 100 
> users, I mean 2,000 – 20,000 users].
>
yeah, we haven't tried scaling vtiger to thousands of users, but we have 
done things with pretty chunky amounts of data. Scaling the users means 
you hit different problems. The full text index is aimed at large 
amounts of records, it could search gigabytes or terrabytes of data but 
it doesn't solve any of the problems of a large number of users.
>
> For the system to work properly it's not enough that you change the 
> global search engine, you also have to change the logic and the 
> core... because the changes are related to hundreds of files. So my 
> question is – do you want to develop the system on at the highest 
> level as developers, or do you want to make toy modules for micro 
> companies? The core needs changes, and they must be introduced to 
> code.vtiger.com, instead of trying to find a solution by implementing 
> modules, because it can't work like that!
>
yeah, there are plenty of core changes that need doing, I want to get 
the javascript libraries in order, perhaps using bower as a package 
manager for it. Mobile device support is something that can be improved, 
there is lots that can be done! The way I see modules working well are 
things for integrating vtiger with another tool that most people won't 
want, or vertical market solutions - perhaps something for estate agents 
selling houses for example. If it is something that everyone would want 
then it might as well go in the core.

> ---
> Z poważaniem / Regards
> *Błażej Pabiszczak*
> /Chief Executive Officer/
> M: +48.884999123
> E: b.pabiszczak at yetiforce.com <mailto:b.pabiszczak at yetiforce.com>
> ------------------------------------------------------------------------
>
> YetiForce 3.0 LTS has arrived! Test 
> <https://gitdeveloper.yetiforce.com/> the latest, most innovative open 
> source system in the world, and join 
> <https://github.com/YetiForceCompany/YetiForceCRM>our community.
>
> W dniu 2016-07-26 16:42, Alan Bell napisał(a):
>
>>
>> On 26/07/16 15:10, Błażej Pabiszczak wrote:
>>>
>>>  1. Can you fill some test system with data, eg. Add 10.000.000
>>>     records into the Accounts module?
>>>
>> yeah, I might have a go at that. I guess numbered data and some 
>> random stuff should be fairly easy to generate. I am not expecting it 
>> to affect the performance much.
>>> 2. Does your search engine allow to search parts of words, eg. I 
>>> want to search for "EFG" in the word "ABCDEFGHI"?
>> it should do, yes but there may be a performance hit for putting a 
>> wildcard at the start of the query. Looking at it now, I am not 
>> entirely sure we are handling wildcards correctly, but the underlying 
>> engine should work with them.
>>>
>>> 3. Can you add 10.000 users to the tested system?
>>>
>> won't make any different to the search performance as such - however 
>> when we get the search results back we do a vtiger API query to see 
>> if each crmid should  be seen by the current user, that way you can't 
>> find things that are private and assigned to other people. I am not 
>> sure if that scales to 10,000 users however if it doesn't then list 
>> views in vtiger don't work either!
>>
>>> ---
>>> Z poważaniem / Regards
>>> *Błażej Pabiszczak*
>>> /Chief Executive Officer/
>>> M: +48.884999123
>>> E: b.pabiszczak at yetiforce.com <mailto:b.pabiszczak at yetiforce.com>
>>> ------------------------------------------------------------------------
>>>
>>> YetiForce 3.0 LTS has arrived! Test 
>>> <https://gitdeveloper.yetiforce.com/> the latest, most innovative 
>>> open source system in the world, and join 
>>> <https://github.com/YetiForceCompany/YetiForceCRM>our community.
>>>
>>> W dniu 2016-07-20 14:32, Alan Lord napisał(a):
>>>
>>>     On 20/07/16 11:50, Alan Bell wrote:
>>>
>>>
>>>             3. Does the search engine need any special search attributes?
>>>
>>>         not really, we do get a bit of a query language for free by using
>>>         elastic search, but these are users who just want to type in words or
>>>         phone numbers or dates and have the system find stuff.
>>>
>>>
>>>     For anyone interested, you can try it out on our demo server:
>>>
>>>     http://geotools.libertus.co.uk/index.php
>>>
>>>     Just start typing anything in the normal search box at the top of any page.
>>>
>>>     After three characters it will start sending stuff to the
>>>     Elasticsearch server and displaying results in a popup.
>>>
>>>     If you want to you can go to the Vlastic Search view, which
>>>     shows multiple ListViews of results on the same page with
>>>     pagination too.
>>>
>>>     e.g.:
>>>     http://geotools.libertus.co.uk/index.php?module=LSVlasticSearch&view=Search&value=smithf*
>>>
>>>     This demo system has about 120,000 records in it but it could
>>>     have lots more...
>>>
>>>     The manual explains some of the search language capabilities,
>>>     such as AND, OR, parenthesis, wildcards etc...
>>>
>>>     But if you type in "smith" it will find all records that has the
>>>     word smith in them somewhere.
>>>
>>>     If you wanted to find something like smithfields (a meat market
>>>     in London) you could type smith* or smithf* for example.
>>>
>>>     Cheers
>>>
>>>     Al
>>>
>>>
>>>     _______________________________________________
>>>     http://www.vtiger.com/
>>>
>>>
>>>
>>> _______________________________________________
>>> http://www.vtiger.com/
>>
>>
>> _______________________________________________
>> http://www.vtiger.com/
>
>
> _______________________________________________
> http://www.vtiger.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.vtigercrm.com/pipermail/vtigercrm-developers/attachments/20160812/12d589f2/attachment-0001.html>