<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:12.0pt;
font-family:"Aptos",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.gmailsignatureprefix
{mso-style-name:gmail_signature_prefix;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:11.0pt;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1549298695;
mso-list-template-ids:519976560;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Glad to hear the new setup is up and running, and from our side we’ve had no more issues with this reported recently so it would all appear to be working !<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> vtigercrm-developers-bounces@lists.vtigercrm.com <vtigercrm-developers-bounces@lists.vtigercrm.com>
<b>On Behalf Of </b>Varma Namburi<br>
<b>Sent:</b> 24 April 2025 08:13<br>
<b>To:</b> vtigercrm-developers@lists.vtigercrm.com<br>
<b>Subject:</b> Re: [Vtigercrm-developers] Marketplace down - causing Slowdown<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Hi Martin, Alen,<br>
<br>
Apologies for the delayed follow-up — and Martin, thank you for highlighting the issue.<br>
<br>
As you mentioned, the delay was caused by the <b><i>getNews()</i></b> function in
<b><i>Modules/Users/views/Login.php</i></b>, which was waiting on an external response that didn’t arrive. The login page only continued once the curl->exec() call timed out, leading to a brief window of 503 errors for new logins. Users already logged in were
unaffected.<br>
<br>
Over the past few weeks, we investigated the issue, tested application-level adjustments, and finally implemented infrastructure-level caching for a more scalable and reliable solution. We also monitored the changes closely to ensure stability.<br>
<br>
<b>Here’s what we put in place:</b><o:p></o:p></p>
<ul type="disc">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<b>DNS-layer caching</b>, with tuned TTLs to speed up resolution<o:p></o:p></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<b>Edge caching</b> for the News API (including JSON), so external delays don’t block login<o:p></o:p></li></ul>
<p class="MsoNormal">Since making these updates, the system has been performing consistently well, with no recurrence of the 503 errors.<br>
<br>
<b>For example</b> <b>:</b> In the last 7 days alone, over 94% of requests — including 16M+ JSON responses from /api/app/news - were served directly from cache. This has greatly reduced backend news API server load and improved response times.<br>
<br>
We’d really appreciate it if you could review from your end and share any feedback. Your early report helped us narrow this down quickly — thanks again!<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Sat, Mar 8, 2025 at 10:12<span style="font-family:"Arial",sans-serif"> </span>PM Varma Namburi <<a href="mailto:varma@vtiger.com">varma@vtiger.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p class="MsoNormal">Hi Martin, Alen,<br>
<br>
This is Varma Namburi from the Vtiger Team.<o:p></o:p></p>
<div>
<p class="MsoNormal"><br>
Sorry for the delay, and thanks for your patience! We really appreciate you bringing this up. We've noted the impact of the getNews() function call and are reviewing it internally to explore ways to make the login page more resilient in cases like this.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><br>
We're actively looking into this and will follow up with any updates. Thanks again for your insights and for bringing this to our attention!<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Fri, Mar 7, 2025 at 8:09<span style="font-family:"Arial",sans-serif"> </span>PM Martin Allen <<a href="mailto:martin.allen@exe-squared.co.uk" target="_blank">martin.allen@exe-squared.co.uk</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">It was down for about 5 minutes, then went to responding with a 503 response, then came back up normally very shortly after.<br>
<br>
Worth looking at the code in Modules/Users/views/Login.php and specifically the getNews() function call.
<br>
Basically it was waiting for a response that didn’t come, so only continued when the curl->exec() call timed out.<br>
<br>
Anyone already logged in was unaffected - and as all our monitoring is just checking the login page, we had LOTS of alarm bells ringing for a short time!<br>
<br>
-----Original Message-----<br>
From: <a href="mailto:vtigercrm-developers-bounces@lists.vtigercrm.com" target="_blank">
vtigercrm-developers-bounces@lists.vtigercrm.com</a> <<a href="mailto:vtigercrm-developers-bounces@lists.vtigercrm.com" target="_blank">vtigercrm-developers-bounces@lists.vtigercrm.com</a>> On Behalf Of Alan Lord<br>
Sent: 07 March 2025 13:35<br>
To: <a href="mailto:vtigercrm-developers@lists.vtigercrm.com" target="_blank">vtigercrm-developers@lists.vtigercrm.com</a><br>
Subject: Re: [Vtigercrm-developers] Marketplace down - causing Slowdown<br>
<br>
All seems OK for me...<br>
<br>
Al<br>
<br>
On 07/03/2025 13:23, Martin Allen wrote:<br>
> Looks like <a href="http://marketplace.vtiger.com" target="_blank">marketplace.vtiger.com</a> is down – and this is causing the
<br>
> vTiger login view to be incredibly slow on every single installation <br>
> we look after!<br>
> <br>
> *Martin Allen*<br>
> <br>
> *Exe Squared Ltd*<br>
> <br>
> 01392 690659 (Direct)<br>
> 01392 248692 (Main Office)<br>
> <br>
> **<br>
> <br>
> The information in this email is confidential If you are not the <br>
> intended recipient, you must not read or use that information. This <br>
> email and any attachments are believed to be virus free however no <br>
> responsibility is accepted by Exe Squared Ltd for any loss or damage <br>
> arising in any way from receipt or use thereof. Exe Squared Ltd <br>
> Company Reg Number 12185709, is based at Silverdown Park, Fair Oak <br>
> Close, Exeter Airport Business Park EX5 2UX.<br>
> <br>
> <br>
> _______________________________________________<br>
> <a href="http://www.vtiger.com/" target="_blank">http://www.vtiger.com/</a><br>
_______________________________________________<br>
<a href="http://www.vtiger.com/" target="_blank">http://www.vtiger.com/</a><br>
<br>
_______________________________________________<br>
<a href="http://www.vtiger.com/" target="_blank">http://www.vtiger.com/</a><o:p></o:p></p>
</blockquote>
</div>
<div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><span class="gmailsignatureprefix">-- </span><o:p></o:p></p>
<div>
<p class="MsoNormal">Thanks & Regards,<br>
Varma.<br>
VtigerTeam.<br>
<br>
<b><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Connect with us on: </span></b><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><a href="http://twitter.com/#%21/vtigercrm" target="_blank"><span style="color:#0000CC">Twitter</span></a> <b>I</b> <a href="http://www.facebook.com/pages/vtiger/226866697333578?sk=wall" target="_blank"><span style="color:#0000CC">Facebook</span></a> <b>I</b> <a href="http://blog.vtiger.com/" target="_blank"><span style="color:#0000CC">Blog</span></a><b> I</b> <a href="http://wiki.vtiger.com/index.php/Main_Page" target="_blank"><span style="color:#0000CC">Wiki</span></a> <b>I </b><a href="http://forums.vtiger.com/" target="_blank"><span style="color:#0000CC">Forums </span></a><b>I</b> <a href="http://vtiger.com/" target="_blank"><span style="color:#0000CC">Website</span></a></span><o:p></o:p></p>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><span class="gmailsignatureprefix">-- </span><o:p></o:p></p>
<div>
<p class="MsoNormal">Thanks & Regards,<br>
Varma.<br>
VtigerTeam.<br>
<br>
<b><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Connect with us on: </span></b><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><a href="http://twitter.com/#%21/vtigercrm" target="_blank"><span style="color:#0000CC">Twitter</span></a> <b>I</b> <a href="http://www.facebook.com/pages/vtiger/226866697333578?sk=wall" target="_blank"><span style="color:#0000CC">Facebook</span></a> <b>I</b> <a href="http://blog.vtiger.com/" target="_blank"><span style="color:#0000CC">Blog</span></a><b> I</b> <a href="http://wiki.vtiger.com/index.php/Main_Page" target="_blank"><span style="color:#0000CC">Wiki</span></a> <b>I </b><a href="http://forums.vtiger.com/" target="_blank"><span style="color:#0000CC">Forums </span></a><b>I</b> <a href="http://vtiger.com/" target="_blank"><span style="color:#0000CC">Website</span></a></span><o:p></o:p></p>
</div>
</div>
</body>
</html>