Re: RoundCube Folder w/ 8500+ messages (Squirrelmail 1.5.x works fine)

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: RoundCube Folder w/ 8500+ messages (Squirrelmail 1.5.x works fine)

jjh-2
Nobody?

-------- Original Message --------
Subject: RoundCube Folder w/ 8500+ messages (Squirrelmail 1.5.x works fine)
Date: Sat, 28 Jul 2007 12:22:22 -0400
From: <[hidden email]>
To: [hidden email]



 I've searched around but haven't found any solutions to this.
Hopefully I haven't overlooked anything.

 I have a folder on my imap account (spam) with 8500+ messages in it.
 I can't for the life of me figure out how to get roundcube to quickly
open that folder.
 At first I thought I might need to replace my imap server.  My imap
server is courier (I love it), but I've also copied the same folder
into dovecot and dbmail without any differences.  From what I've read
both of the test servers I've tried should have benifits but as they
haven't I've ruled out that as a solution for now.
 I've also tried various imap clients.  RoundCube of course, which
takes (guessing) 30+ seconds to open the folder.  Navigating away and
back to the folder takes another 30+ seconds to open it again.  That
seems riddiculus but Squirrelmail 1.4.x seems to have the same issue.
Thunderbird seems to take 30+ seconds to load all the headers but once
they are loaded I can quickly move back and forth
between folders.
Finally, Squirrelmail 1.5.x works wonderfully - it loads in a second
and I can pick an set of messages (page 500 for example) and it will
immediately open to that set.  That would be great if I was still
using Squirrelmail, but RoundCube is miles ahead in the look and feel.


 Are there any current fixes or tweaks I can use for this?  I see two
ways to improve this, the first is caching of message headers like
Thunderbird apparently does. I could be wrong but I think all that is
currently cached in the database is messages, not headers.  (Database
and Imap and Apache are on the same machine btw) That would be
acceptible to me as once I have my client open it stays open for
hours/days.  The ideal fix though would be to impliment whatever is in
squirrelmail 1.5.x.  Now obviously that's a perl vs php but it should
translate just the same.  If they can do it, roundcube could do it,
no?

 Thanks,

 Jim

_______________________________________________
List info: http://lists.roundcube.net/users/
Reply | Threaded
Open this post in threaded view
|

Re: Re: RoundCube Folder w/ 8500+ messages (Squirrelmail 1.5.x works fine)

Brady J. Frey-2
Since Roundcube is syncing your mail with a database, I don't think
you're going to get too much speed out of it reading that much mail.
Ideas are: Jacking up php.ini to have a larger max resources (for
example in my /usr/local/php5/lib/php.ini file I can up my memory_limit
= whatever I choose), do the cache such as you recommended, use
something like Zend optimizer
http://www.zend.com/products/zend_optimizer/?engine=google&cmpg=optimizer&k_id=zend_optimizer 
to speed up your PHP, or do other tricks to get PHP rolling (found this
reference
http://phplens.com/lens/php-book/optimizing-debugging-php.php)... or
cleaning out your inbox, and I recommend Getting Things Done by David
Allen to help with that:)

In all seriousness, there are many variations I can think of, some maybe
the fact that Roundcube is going to a MySQL database, so MySQL on your
end could be slow, or PHP, or both. I don't think Squirrell uses MySQL
does it?

[hidden email] wrote:

> Nobody?
>
> -------- Original Message --------
> Subject: RoundCube Folder w/ 8500+ messages (Squirrelmail 1.5.x works fine)
> Date: Sat, 28 Jul 2007 12:22:22 -0400
> From: <[hidden email]>
> To: [hidden email]
>
>
>
>  I've searched around but haven't found any solutions to this.
> Hopefully I haven't overlooked anything.
>
>  I have a folder on my imap account (spam) with 8500+ messages in it.
>  I can't for the life of me figure out how to get roundcube to quickly
> open that folder.
>  At first I thought I might need to replace my imap server.  My imap
> server is courier (I love it), but I've also copied the same folder
> into dovecot and dbmail without any differences.  From what I've read
> both of the test servers I've tried should have benifits but as they
> haven't I've ruled out that as a solution for now.
>  I've also tried various imap clients.  RoundCube of course, which
> takes (guessing) 30+ seconds to open the folder.  Navigating away and
> back to the folder takes another 30+ seconds to open it again.  That
> seems riddiculus but Squirrelmail 1.4.x seems to have the same issue.
> Thunderbird seems to take 30+ seconds to load all the headers but once
> they are loaded I can quickly move back and forth
> between folders.
> Finally, Squirrelmail 1.5.x works wonderfully - it loads in a second
> and I can pick an set of messages (page 500 for example) and it will
> immediately open to that set.  That would be great if I was still
> using Squirrelmail, but RoundCube is miles ahead in the look and feel.
>
>
>  Are there any current fixes or tweaks I can use for this?  I see two
> ways to improve this, the first is caching of message headers like
> Thunderbird apparently does. I could be wrong but I think all that is
> currently cached in the database is messages, not headers.  (Database
> and Imap and Apache are on the same machine btw) That would be
> acceptible to me as once I have my client open it stays open for
> hours/days.  The ideal fix though would be to impliment whatever is in
> squirrelmail 1.5.x.  Now obviously that's a perl vs php but it should
> translate just the same.  If they can do it, roundcube could do it,
> no?
>
>  Thanks,
>
>  Jim
>
> _______________________________________________
> List info: http://lists.roundcube.net/users/
>  

_______________________________________________
List info: http://lists.roundcube.net/users/
Reply | Threaded
Open this post in threaded view
|

Re: Re: RoundCube Folder w/ 8500+ messages (Squirrelmail 1.5.xworks fine)

jjh-2
While your suggestions all are valid to some degree, I think you overlooked
one very important point.  I can swap out the latest version of RoundCube
with Squirrelmail 1.5.x and there is a huge increase in speed - almost
instant - when loading folders with thousands of messages. On one hand
that's apples and oranges, on the other it indicates to me that there must
be some issue in RoundCube if it cannot pull messages at the same speed as
another client can, on the same hardware, and same mail account.

On Mon, 20 Aug 2007 14:04:52 -0700, "Brady J. Frey" <[hidden email]>
wrote:
> Since Roundcube is syncing your mail with a database, I don't think
> you're going to get too much speed out of it reading that much mail.
> Ideas are: Jacking up php.ini to have a larger max resources (for
> example in my /usr/local/php5/lib/php.ini file I can up my memory_limit
> = whatever I choose), do the cache such as you recommended, use
> something like Zend optimizer
>
http://www.zend.com/products/zend_optimizer/?engine=google&cmpg=optimizer&k_id=zend_optimizer

> to speed up your PHP, or do other tricks to get PHP rolling (found this
> reference
> http://phplens.com/lens/php-book/optimizing-debugging-php.php)... or
> cleaning out your inbox, and I recommend Getting Things Done by David
> Allen to help with that:)
>
> In all seriousness, there are many variations I can think of, some maybe
> the fact that Roundcube is going to a MySQL database, so MySQL on your
> end could be slow, or PHP, or both. I don't think Squirrell uses MySQL
> does it?
>
> [hidden email] wrote:
>> Nobody?
>>
>> -------- Original Message --------
>> Subject: RoundCube Folder w/ 8500+ messages (Squirrelmail 1.5.x works
> fine)
>> Date: Sat, 28 Jul 2007 12:22:22 -0400
>> From: <[hidden email]>
>> To: [hidden email]
>>
>>
>>
>>  I've searched around but haven't found any solutions to this.
>> Hopefully I haven't overlooked anything.
>>
>>  I have a folder on my imap account (spam) with 8500+ messages in it.
>>  I can't for the life of me figure out how to get roundcube to quickly
>> open that folder.
>>  At first I thought I might need to replace my imap server.  My imap
>> server is courier (I love it), but I've also copied the same folder
>> into dovecot and dbmail without any differences.  From what I've read
>> both of the test servers I've tried should have benifits but as they
>> haven't I've ruled out that as a solution for now.
>>  I've also tried various imap clients.  RoundCube of course, which
>> takes (guessing) 30+ seconds to open the folder.  Navigating away and
>> back to the folder takes another 30+ seconds to open it again.  That
>> seems riddiculus but Squirrelmail 1.4.x seems to have the same issue.
>> Thunderbird seems to take 30+ seconds to load all the headers but once
>> they are loaded I can quickly move back and forth
>> between folders.
>> Finally, Squirrelmail 1.5.x works wonderfully - it loads in a second
>> and I can pick an set of messages (page 500 for example) and it will
>> immediately open to that set.  That would be great if I was still
>> using Squirrelmail, but RoundCube is miles ahead in the look and feel.
>>
>>
>>  Are there any current fixes or tweaks I can use for this?  I see two
>> ways to improve this, the first is caching of message headers like
>> Thunderbird apparently does. I could be wrong but I think all that is
>> currently cached in the database is messages, not headers.  (Database
>> and Imap and Apache are on the same machine btw) That would be
>> acceptible to me as once I have my client open it stays open for
>> hours/days.  The ideal fix though would be to impliment whatever is in
>> squirrelmail 1.5.x.  Now obviously that's a perl vs php but it should
>> translate just the same.  If they can do it, roundcube could do it,
>> no?
>>
>>  Thanks,
>>
>>  Jim
>>
>> _______________________________________________
>> List info: http://lists.roundcube.net/users/
>>

_______________________________________________
List info: http://lists.roundcube.net/users/
Reply | Threaded
Open this post in threaded view
|

Re: Re: RoundCube Folder w/ 8500+ messages (Squirrelmail 1.5.xworks fine)

Jason Fesler
> While your suggestions all are valid to some degree, I think you overlooked
> one very important point.  I can swap out the latest version of RoundCube
> with Squirrelmail 1.5.x and there is a huge increase in speed - almost

It sounds like Squirrelmail 1.5.x may finally be only pulling the headers
for a single "page" of index data?  If so, I need to investigate it.. :-)

Given the GUI nature of RC, I'm not sure how practical that is.  Unless
you allow scrolling right away, perhaps with dummy data, and dynamically
fetch the entries you need to show for the current page, as you scroll to
them; and used some kind of persistent connection from the client to the
server, to avoid bashing the imap server.

Few clients actually get this right (non-web); pine and mulberry (RIP) are
the only couple I know that only fetch the index headers for what needs to
be shown *now*; both fetch more on demand; and they usea persistent
connection (but that's cheating, since they aren't a web app).


_______________________________________________
List info: http://lists.roundcube.net/users/
Reply | Threaded
Open this post in threaded view
|

Re: Re: RoundCube Folder w/ 8500+ messages (Squirrelmail1.5.xworks fine)

jjh-2
> It sounds like Squirrelmail 1.5.x may finally be only pulling the headers
> for a single "page" of index data?  If so, I need to investigate it.. :-)
>
> Given the GUI nature of RC, I'm not sure how practical that is.  Unless
> you allow scrolling right away, perhaps with dummy data, and dynamically
> fetch the entries you need to show for the current page, as you scroll to
> them; and used some kind of persistent connection from the client to the
> server, to avoid bashing the imap server.

I don't know how either are polling the database, but shouldn't RoundCube
only be querying a maximum of the "rows per page" setting under Personal
Settings each time you navigate to or in a folder?  I hate to think it is
retrieving thousands of messages and then only showing me 40 each time I
touch something...

> ... and they use a persistent
> connection (but that's cheating, since they aren't a web app).

You can have a partially-persistent imap connection if imapproxy is used on
the backend, or are you talking Apache's not-persistence?

_______________________________________________
List info: http://lists.roundcube.net/users/
Reply | Threaded
Open this post in threaded view
|

Re: Re: RoundCube Folder w/ 8500+ messages (Squirrelmail1.5.xworks fine)

Chet Finnick
Just an FYI, my Inbox is 25,000+ messages and RoundCube loads it in just under 3 seconds.  My manager's Inbox is just uner 50,000 and it takes just over 3 seconds to load.  And this is from our data server that is almost a 5 hour drive away.  I'm not experiencing any problems with folders that have large amounts of messages.

[hidden email] wrote:
It sounds like Squirrelmail 1.5.x may finally be only pulling the headers
for a single "page" of index data?  If so, I need to investigate it.. :-)

Given the GUI nature of RC, I'm not sure how practical that is.  Unless
you allow scrolling right away, perhaps with dummy data, and dynamically
fetch the entries you need to show for the current page, as you scroll to
them; and used some kind of persistent connection from the client to the
server, to avoid bashing the imap server.
    

I don't know how either are polling the database, but shouldn't RoundCube
only be querying a maximum of the "rows per page" setting under Personal
Settings each time you navigate to or in a folder?  I hate to think it is
retrieving thousands of messages and then only showing me 40 each time I
touch something...

  
... and they use a persistent
connection (but that's cheating, since they aren't a web app).
    

You can have a partially-persistent imap connection if imapproxy is used on
the backend, or are you talking Apache's not-persistence? 

_______________________________________________
List info: http://lists.roundcube.net/users/

  


-- 
Chet Finnick

IntraMeta Corporation
  w.  972.231.5999
  c.  214-708-0121
  f.  972.231.7022
  e.  [hidden email]

_______________________________________________
List info: http://lists.roundcube.net/users/
Reply | Threaded
Open this post in threaded view
|

Re: Re: RoundCube Folder w/ 8500+ messages (Squirrelmail1.5.xworks fine)

Jason Fesler
In reply to this post by jjh-2
> You can have a partially-persistent imap connection if imapproxy is used on
> the backend, or are you talking Apache's not-persistence?

All the way through.  User to PHP instance to IMAP. Doing this via web
is... pretty, and simple for end users, but has a heck of a cost :-).

_______________________________________________
List info: http://lists.roundcube.net/users/
Reply | Threaded
Open this post in threaded view
|

Re: Re: RoundCube Folder w/ 8500+ messages (Squirrelmail1.5.xworksfine)

jjh-2
In reply to this post by Chet Finnick
Can you provide any details on your configuration?  

On Tue, 21 Aug 2007 14:26:32 -0500, Chet Finnick <[hidden email]>
wrote:

>            Just an FYI, my Inbox is 25,000+ messages and RoundCube
> loads it in just under 3 seconds.  My manager's Inbox is just uner
> 50,000 and it takes just over 3 seconds to load.  And this is from
> our data server that is almost a 5 hour drive away.  I'm not
> experiencing any problems with folders that have large amounts of
> messages.
>  [hidden email] [1] wrote:        
>
> It sounds like Squirrelmail 1.5.x may finally be only pulling the
> headers for a single "page" of index data?  If so, I need to
> investigate it.. :-) Given the GUI nature of RC, I'm not sure how
> practical that is.  Unless you allow scrolling right away, perhaps
> with dummy data, and dynamically fetch the entries you need to show
> for the current page, as you scroll to them; and used some kind of
> persistent connection from the client to the server, to avoid bashing
> the imap server.          
>
>  I don't know how either are polling the database, but shouldn't
> RoundCube only be querying a maximum of the "rows per page" setting
> under Personal Settings each time you navigate to or in a folder?  I
> hate to think it is retrieving thousands of messages and then only
> showing me 40 each time I touch something...          
>
> ... and they use a persistent connection (but that's cheating, since
> they aren't a web app).          
>
>  You can have a partially-persistent imap connection if imapproxy is
> used on the backend, or are you talking Apache's not-persistence?
> _______________________________________________ List info:
> http://lists.roundcube.net/users/ [2]    
> -- Chet Finnick IntraMeta Corporation   w.  972.231.5999   c.
> 214-708-0121   f.  972.231.7022   e.  [hidden email] [3]  
>
> Links:
> ------
> [1] mailto:[hidden email]
> [2] http://lists.roundcube.net/users/
> [3] mailto:[hidden email]
>

_______________________________________________
List info: http://lists.roundcube.net/users/
Reply | Threaded
Open this post in threaded view
|

Re: Re: RoundCube Folder w/ 8500+ messages (Squirrelmail1.5.xworksfine)

Chet Finnick
Yeah you bet...

Apache 2.0.54
PHP  5.0.4
mysql 5.0.19
Fedora 4
courier-imap 3.0.7

The mysql and courier-imap services are running on different servers then the apache/php.  Hope this helps.



[hidden email] wrote:
Can you provide any details on your configuration?  

On Tue, 21 Aug 2007 14:26:32 -0500, Chet Finnick [hidden email]
wrote:
  
           Just an FYI, my Inbox is 25,000+ messages and RoundCube
loads it in just under 3 seconds.  My manager's Inbox is just uner
50,000 and it takes just over 3 seconds to load.  And this is from
our data server that is almost a 5 hour drive away.  I'm not
experiencing any problems with folders that have large amounts of
messages.
 [hidden email] [1] wrote:         

It sounds like Squirrelmail 1.5.x may finally be only pulling the
headers for a single "page" of index data?  If so, I need to
investigate it.. :-) Given the GUI nature of RC, I'm not sure how
practical that is.  Unless you allow scrolling right away, perhaps
with dummy data, and dynamically fetch the entries you need to show
for the current page, as you scroll to them; and used some kind of
persistent connection from the client to the server, to avoid bashing
the imap server.           

 I don't know how either are polling the database, but shouldn't
RoundCube only be querying a maximum of the "rows per page" setting
under Personal Settings each time you navigate to or in a folder?  I
hate to think it is retrieving thousands of messages and then only
showing me 40 each time I touch something...           

... and they use a persistent connection (but that's cheating, since
they aren't a web app).           

 You can have a partially-persistent imap connection if imapproxy is
used on the backend, or are you talking Apache's not-persistence?
_______________________________________________ List info:
http://lists.roundcube.net/users/ [2]     
-- Chet Finnick IntraMeta Corporation   w.  972.231.5999   c. 
214-708-0121   f.  972.231.7022   e.  [hidden email] [3]   

Links:
------
[1] [hidden email]
[2] http://lists.roundcube.net/users/
[3] [hidden email]

    

_______________________________________________
List info: http://lists.roundcube.net/users/

  


-- 
Chet Finnick

IntraMeta Corporation
  w.  972.231.5999
  c.  214-708-0121
  f.  972.231.7022
  e.  [hidden email]

_______________________________________________
List info: http://lists.roundcube.net/users/
Reply | Threaded
Open this post in threaded view
|

Courier-IMAP storage (Was: Re: RoundCube Folder w/ 8500+ messages)

B. Johannessen
Chet Finnick wrote:
> [hidden email] wrote:
>> Chet Finnick wrote:
>>> Just an FYI, my Inbox is 25,000+ messages and RoundCube
>>> loads it in just under 3 seconds.
>>
>> Can you provide any details on your configuration?  
>
> courier-imap 3.0.7

That is very impressive for a plain Maildir backed IMAP server. Could
you provide some more information on the storage used for the Maildirs?


        Bob
_______________________________________________
List info: http://lists.roundcube.net/users/