Thunderbird:Testing:Memory Usage Problems

From MozillaWiki
Jump to: navigation, search

This article is for users with ALL types of performance issues.

Thunderbird performance (slowness) can be affected by CPU, memory and disk activity from many sources. You are likely frustrated, so the steps are simplified to be clear and easy to follow. Please step through the full list below to have the best possible chance of success.

Diagnosis

Before starting

  • Many steps are temporary *tests*, not a request for you to forever disable a feature. Disabling antivirus software for example.
  • Do not delete files. Instead, rename or move files because we may need "bad files" later for developers to examine.
  • As you go through the steps, collect the information you will need to later report a problem:
    • When referring to memory, cpu, amount of time, numbers of folders and messages, etc, provide numbers for quantities, not vague words like fast/slow, small/large.
    • List account types (imap, pop, rss, etc) from Help | More Troubleshooting Info.
    • Provide full Thunderbird version number from Help | About.
    • State precisely when the behavior occurs (startup, shutdown, or "normal operation"), and every step (action, menu click, ...) needed to reproduce the problem.
    • Cite the step numbers of this document that you tested, and any which helped or didn't help improved your issue. Also bug numbers, URLs of postings and references you used.
    • Whether your Thunderbird profile data is in a non-standard location.
    • Screen shots can be helpful.

Diagnosis Steps

  1. Are you using the newest Thunderbird? (Versions are listed at releases. The newest version there should match to your Help | About. If check for Updates doesn't get you to that version, then download it from thunderbird.net.)
  2. Start Thunderbird in Troubleshoot Mode. If Troubleshoot Mode helped, then start in normal mode. If the problem returned:
    • Visit Tools | Add-ons - disable two or more add-ons at a time (or themes or language packs) to determine which might be causing the problem. Are any Add-on items in the list of Problem Extensions?
    • Contact the author of suspected bad add-ons using their support website or email.
  3. Start OS in safe mode. Instructions: win10, win8, win7, MacOS. If performance is improved in safe mode, then
    • check for newer version of AV/firewall software, and check with vendor's support site for problem reports
    • test with antivirus disabled or removed - because we constantly see reports of antivirus causing slowness, and disabling isn't always reliable. See AV performance issues for examples and settings.
    • on Windows, especially Windows 10, try disabling MS-Windows Search (see next numbered item below)
    • test with firewall disabled or removed Ad-Aware example
    • any software loaded at boot time is a possible cause
    • run only one antivirus, one firewall, one malware (never multiples of each - more is not better)
    • on Mac, try with Time Machine disabled
    • uninstall Thunderbird, then DELETE the Thunderbird PROGRAM directory (not the data directory), then download, reinstall, and retest Thunderbird. If using beta get it from channels.
  4. Do you have Calendars? Try disabling two or more at a time. If you have many recurring, never ending calendar appointments try reducing the time period and number of them. Less likely starting with version 128's fix for bug 1844408. Examples: [1], [2]
  5. If you have gmail accounts, is problem gone after unsubscribing the "All Mail" folder?
  6. On Windows, did you move the Thunderbird profile directory or any Thunderbird account from the default location "%APPDATA%"? If yes, then MS-Windows Search will attempt to index all Thunderbird disk activity. Disable Windows indexing for your Thunderbird data.
  7. Folders should be well maintained - if not then the folder index takes more memory and time to load. Help | Troubleshooting lists large folders. Folders may also be broken. Signs of problems and common maintenance steps:
    • Do you see high memory or delay when opening a specific folder, deleting or archiving, moving message to a specific folder?
    • Do any folders lose their sort order or column settings?
    • Do you empty Trash and Spam (junk) folders on a regular basis? These should not be larger than a few thousand messages, and you should not use these folders for long term storage. Does performance improve if you empty them? An alternative to manual emptying is setting automatic retention value in Folder Properties. Also see [https://support.mozilla.org/en-US/kb/menu-reference#w_empty-trash File menu, bug 1192827
    • Do any Inbox or filtered folders have several thousand messages? If so, especially for Inbox, consider moving some messages to another folder. Perhaps use Archive.
    • Are compacts routinely performed? Compact must be automatic or done regularly (at least weekly) using manual File > Compact Folders?
  8. If Thunderbird memory usage is more than 1-2GB then perhaps folders are corrupted or not being closed. Please :
    • Get a diagnostic log of Thunderbird activity with MSGDB:5,timestamp using logging instructions.
    • File a bug, then edit the new bug and attach the zipped log file using "Add an attachment".
  9. Does setting Thunderbird to offline mode help? File | Offline | Work Offline - Work Offline should be "checked" - this stops checks for new mail, news and RSS feeds.
    • If Offline did not help, do File | Offline | Work Offline - Work Offline should now be "unchecked".
    • If Offline did help, then go back Online (disable Offline) and ...:
      • Does disabling interval checks for new mail help? (Account Settings | Server Settings | uncheck "Check for new messages every N minutes"). If it helps, then ... is a low value (1-3 minutes) specified for checking for new mail? Does increasing it to 5-10 minutes help?
      • Is a low value (1-3 minutes) specified for draft autosave at Options | Composition | General | Auto Save every N minutes. Does increasing it to 5-10 minutes help?
      • What type of mail account is this (imap or pop), and how many other accounts are the same type?
      • Are you absolutely certain that antivirus software is not to blame?
      • Are any non-Inbox IMAP folders configured to check for new messages via "always check this folder" in properties? Or is hidden preference mail.check_all_imap_folders_for_new set to true? These increase load on both Thunderbird and the mail server. bug 729504
  10. Does problem resolve by disabling Global Search indexing? Tools|Options|Advanced|General| uncheck Enable Global Search and Indexer (linux: Edit|Preferences|Advanced|... Mac: cmd , advanced tab) and restart Thunderbird. (note, disabling indexing also disables searching)
    • If problem did not go away, or to recheck indexing behavior, enable the checkbox again and restart Thunderbird.
    • If the problem did go away - did Thunderbird have extremely high disk IO, seem to be stuck on indexing per Tools|Activity Manager, or is global-messages-db.sqlite (file used for global search) in your profile folder too big? Rule of thumb - the file should not be more than 10-20% of the combined size of all messages in directories Mail and ImapMail.
      • Rename global-messages-db.sqlite (to keep it for later diagnosis), enable the Global Search checkbox, and restart Thunderbird. global-messages-db.sqlite will rebuild and your system load may be high until all your messages are reindexed, but your messages should not redownload from the server.
      • Please file a bug about your problem.
  11. Are any of these indexing methods enabled in Thunderbird:
  12. Do you have thousands of contacts in your address book? (possible related issues: google contacts add-on], [14]
  13. Do you see "Unresponsive Script" or "Not Responding"? If you do, is Thunderbird whitelisted in your antivirus software? If whitelisting didn't help, set javascript.options.mem.log to true and provide a couple log entries of the form CC(T+1186.6) collected: 131 (131 waiting for GC), suspected: 98, duration: 274 ms.
  14. Has a proxy ever been defined, and is it working correctly or was the proxy removed? (A proxy can cause "Unresponsive script")
  15. Does turning off the status bar in View > Toolbars help improve performance?
  16. If you see high CPU usage in Windows taskmgr, sort the CPU column in the details tab to see if mozilla updater.exe, Window's update, or some other process is hogging CPU.
  17. Is your Thunderbird profile or any part of Thunderbird not on a local hard drive? If not local, does it preform better when it is moved to the local disk?
    • Cloud storage is not supported.
    • USB drives are slower compared to a local disk. Some types or manufacturers are terrible. Make sure you've got a good one.
    • network drive can be slow due to network transmission or slow media (SMB, NFS, AFS, samba)
    • SSD? (sounds silly, but some SSD have been reported to be slow)
  18. If you suspect slow disk or high disk activity
    • check the previous item (NFS, etc)
    • in Windows you want at least 20% free disk space, and for spinning disks do periodic defrags
    • check whether any OS or application update process is in progress, for example Windows update
  19. How many virtual folders have you created?
  20. If you have newsgroup accounts, then in Account Settings|Server Settings try disabling "Check for new messages at startup". If problem is gone you may be seeing many newsgroups with many unread messages, bug 185634, bug 541001, or [15].
  21. If you have RSS news reader account:
    • Is "Automatically create tags from feed category names" enabled? [16]
    • In Account Settings|Server Settings disable "Check for new messages at startup". If problem is gone you may be seeing bug 551144 (fixed in version 13).
  22. Any Outlook or Hotmail accounts? bug 1355885
  23. Is filter logging enabled at Tools > Filters? (large logs cause high overhead)
  24. Do you use server side message filters to place messages in folders? bug 589310
  25. Check activity in Tools > Activity Manager to see if ISP (mail provider) is being lame. Examples: Yahoo and Gmail (and others) may cause message folders to be redownloaded. slow downloading email (Centurylink)
  26. Common resource requirements, hardware checks:
  27. Are you using Linux?
  28. If you have a reportable problem or question, please use the next section below.


Report a problem

Provide all the information you have collected plus the URL to a performance profile which you create in one of the following:

References

"Not Responding" - Examples: BTTray.exe, (Plusnet Protect/McAfee), McAfee, WSS, Windows Search Indexing and [20], McAfee, McAfee, McAfee (2013) Solutions: use XP3 compatibility mode in Windows 8 (Endicott), use XP3 compatibility mode (missychrissy), use XP3 compatibility mode (WaddyT/2013), use XP3 compatibility mode in Windows 8 (kgcode)

"Unresponsive Script" - Examples: Solutions:

Unresolved bugs and references:

  • McAfee antivirus is causing major performance and problems with TB24 (and was with TB17 and TB16). Symptom: slow delete and read, "Unresponsive Script", "(Not Responding)". Solution
  • bug 1342054 - Large number of tags causes very bad tags preference UI performance, from autotaging RSS feeds with "Automatically create tags from feed category names"
  • bug 1438317 - "Indexing messages. Determining which messages to index" stuck on local folder never ends. Modest CPU - 7-10%. Error console "Observed header that claims to be gloda indexed but that gloda has never heard of during compaction"
  • bug 1406653 - Gloda warning repeated every half-second causing high cpu and slow response. "Observed header that claims to be gloda indexed but that gloda has never heard of during compaction" in error console and "Determining which messages to index""
  • bug 1240722 - tidybird add-on causes files/folders to be opened and stay open twice - Thunderbird leaks file descriptors (mostly .msf files) like a sieve causing high memory. (no virtual folders)
  • bug 929281 - Long startup and "Unresponsive Script" with many pop accounts with version 24 (regression in TB18)
  • bug 764306 - When trying to scroll large folders or switch some folders the application beachballs with high CPU (xref: bug 872869)
  • High mem usage, increasing with every delete. No bug report. Solved by right+click, compact.
  • bug 632791 Gloda (Global Search) indexes one message per minute with high CPU. Excessively sized global-messages-db.sqlite caused by buggered, excessively sized offline store. [because offline folder storage has many duplicate messages. but it should not]
  • bug 563677 HTML Mail View slow (45++ sec) for pop and local folders - speed up FolderUriForPath (from bug 545126)
  • bug 487832 - Thunderbird startup performance issues [meta]
  • Template:Bug 357008 - (If you have a profile created between version 1.5 and 2.0) Remove obsolete directory address books entries from prefs
  • Lightning related:
    • bug 441710 - Thunderbird freezes/slow for 3-60 seconds caused by calendar/lightning [meta]
    • bug 753343 - Soft-hang and massive memory leaks with Lightning and ics enabled
    • bug 501689 - Investigate making storage asynchronous // bug 462277 - (calcache) [meta][GSoC 2014] Turn on offline cache by default for new calendars
    • bug 498968 - Create a new, faster storage "storage2" provider
  • larger, less selective list of bugs inclusive of those cited above

Bugs fixed but not released. Typically fixed in unreleased versions (eg. trunk regressions):

  • bug 1502923 - Thunderbird processing / "Not Responding" 5 to 30 minutes at startup. Also "Unresponsive Script" - Performance issues in unifinder.js

Fixed bugs/issues:

Performance bugs fixed in 60.0:

  • bug 1293245 message with malformed email address makes Thunderbird hang 100% CPU load (also some crashes)
  • bug 1428082 - Thunderbird Daily becomes unresponsive when Add-ons Manager opened from hamburger after bug 1427407 landed (TB59,fixed TB60)
  • bug 1415723 - Severe regression in at least Tb55 when repairing/rebuilding/compacting folder database (msf) file. (TB55, fixed TB59)
  • bug 1404003 Daily from 2017-09-28 re-downloads and indexes messages at every start - high CPU (TB58, fixed TB58)
  • bug 1116196 - Every startup, Warning: Unresponsive Script after approx 15 seconds, with system integration prompt (regression in TB18, stopped appearing in TB54beta)

Performance bugs fixed in 52.0.x:

  • bug 1290733 - UpdateSendLock() in compose window is very slow (TB23, fixed TB51)
  • bug 562965 - folderPane.js ftv_getRowProperties/ftv_getCellProperties disproportionately slow
  • bug 1290729 - Adding new recipients in compose is very slow
  • bug 1242050 - Default output *stream* buffer size of 16KB instead of 4KB
  • bug 558528 - Larger *disk* buffer size, 16KB instead of 4KB, in file I/O by Mail&News to improve performance on slower drives (for example network drives/shares).

Performance bugs fixed in 45.0.x:

  • bug 1223371 - remove windows search, aka integrated search, from Thunderbird System integration dialog startup prompt (not an actual perf improvement, just avoids tempting uneducated users from picking it)

Resolved at unknown date

  • bug 1185245 - Hardware acceleration (HWA) may cause message compose window to lag severely (WFM)

Performance bugs fixed in 38.0:

  • bug 872869 - nsMsgDatabase::ListAllKeys() very slow with high CPU downloading messages into large folder (xref: bug 764306) (regression in TB16)
  • bug 846123 - Thunderbird 100% CPU for minutes when copying a large number of messages (IMAP Online Copy, Copy between Offline-Use=Off folders) (landed in TB37)
  • bug 939462 - Feature to count and show number of unread e-mails in subfolders should be optional. (because enumeration is slow) (regression in TB15) Fixed by bug 1135310 Algorithm from bug 723248 for closing idle folder databases is broken because too many/all databases can be closed and bug 464973 "Expanded Columns" in the folder pane are no longer available to select columns for display for message total count and folder size
  • bug 870556 - O(n^2) performance freezes UI for several minutes fetching new mail from IMAP server for very large folder
  • bug 778907 - Highlighting/selecting 18000+ of emails inside a folder keeps giving the javascript is unresponsive script ("bulkoperations", i.e. not a general performance issue) (landed in TB36)
  • bug 1050840 - "Maximum number of imap server connections to cache" advanced account setting defaults to 1 for new accounts (regression in TB24)

Performance bugs fixed in 31.0:

  • bug 975795 - extremely slow for large selections, because folderDisplay.canArchiveSelectedMessages is too slow (landed in TB30) (helped by bug 943116 and bug 942638)
  • bug 1019969 - (resolved WFM, cause and fix unknown) Thunderbird 18-30b1 gets "Unresponsive script" 45 seconds after launch/startup. OK in 31.0
  • bug 926181 - Thunderbird 24 OSX freezes on startup with profile on network drive (AFP) - fixed via bug 918612 - Mac: Firefox 24 doesn't start, or very slow startup

Performance bugs fixed in 24.0:

  • bug 812923 - Dragging a huge number of messages onto a folder freezes TB even BEFORE dropping (TB24)
  • bug 750781 - Thunderbird 12 veryyyyy slow when opening Message. Sometimes shows "Not Responding" (TB21)
  • bug 810637 - HUGE memory consumption when downloading headers for large IMAP folder with mailnews.customDBHeaders (TB19)

Performance bugs fixed in 17.0:

  • bug 806760 - TB16 Redownloads messages / Constantly bringing folders "up to date" / horrible IMAP performance (TB16 regression)
  • bug 769346 - Poor performance of Move, Copy and Delete to Local Folders due to "one write request per a line of message data"

Performance bugs fixed in 16.0.2:

  • bug 803843 - IMAP mailfiles keep growing to gigabyte size

Performance bugs fixed in 16:

  • bug 793455 - DB cache is closing databases too frequently (regression in 15.0)
  • bug 770416 - (insignificant?) memory leak on some IMAP operations

Performance bugs fixed in 15.0.1:

  • bug 782899 - Daily hangs and leaks memory like crazy when trying to delete messages in the results of a filter search (regression in 15.0)

Performance bugs fixed in 15.0:

  • bug 784286 - Abnormal memory requirements viewing plain-text mail leading to crash
  • bug 723248 - add support for closing inactive databases (folders)
  • bug 762169 - Always asking to compact folders starting version 12

Performance bugs fixed in 14.0:

  • bug 538378 - Mass delete of messages resulted in massive memory usage // bug 296453 - Racing CPU, slow performance moving/deleting/dragging large number of messages

Performance bugs fixed in 13.0:

  • bug 551144 - checking rss feeds has problem freezing Thunderbird UI and high hard drive usage (not a v3 or v3.1 regression)

Performance bugs fixed in 10.0:

Performance bugs fixed in 9.0:

  • bug 689742 - Disable hardware acceleration (direct2d) in Thunderbird

Performance bugs fixed in 5.0

  • bug 574952 - Thunderbird 3.1 account list / folder pane slow to scroll on Mac (Mac-only)
  • bug 617839 - Switching to threads with a large number of emails is extremely slow

Performance bugs fixed in 3.1.5:

  • bug 542234 memory increases about 12MB on each check for new mail for pop accounts, 3.0 & 3.1
  • bug 599119 slow message display/slow navigation - plugin directory scans cause excessive overhead

Performance bugs fixed in 3.0.9:

  • bug 542234 memory increases about 12MB on each check for new mail for pop accounts, 3.0 & 3.1

Performance bugs fixed in 3.0.5:

  • bug 530098 gloda message deletion processing is expensive

Performance bugs fixed in 3.1:

  • bug 533776 gloda indexes while offline, if we startup without a network connection
  • bug 539389 Very slow file manipulation deleting or moving messages (profile stored on synchronized network folder). Tb3 requests write for each line of mail data. Should use buffering.
  • bug 545126HTML Mail View very slow (45++ sec) for pop and local folders (*not* fixed in v3.0.x) / bug 536873 slow load HTML messages with View Message Body as Original HTML - sits at "Loading Message ..." or presents blank screen
  • bug 530098 gloda message deletion processing is expensive (*not* fixed in v3.0.x)
  • bug 493848 gloda activity manager should not create events for event-driven indexing, never display in status bar (*not* fixed in v3.0.x)
  • bug 543737 gloda needs to avoid indexing ridiculously large message bodies (especially from offline store corruption) (indexing stops at 20k per message part)(*not* fixed in v3.0.x)
  • bug 560695 clicking (more) widget in headers with lots of recipients too slow (*not* fixed in v3.0.x)

Performance bugs fixed in 3.0.4:

  • bug 533776 gloda indexes while offline, if we startup without a network connection
  • bug 539389 Very slow file manipulation deleting or moving messages (profile stored on synchronized network folder). Tb3 requests write for each line of mail data. Should use buffering.

Memory related bugs fixed in 3.0.2/3.0.3:

  • bug 540214 imap STATUS not closing db - mail.check_all_imap_folders_for_new and "check this folder for new messages" causes high memory and error message Unable to open the summary file for xxxxx. Perhaps there was an error on disk, or the full path is too long.

Performance related bugs fixed in 3.0.1:

  • bug 534835 offline folder size constantly increases using Exchange IMAP (Exchange returns wrong rfc822 size) - messages redownload
  • bug 537820 quick search in smart folders/saved searches should prefer offline when it has a choice
  • bug 533865 Doing a quick search in a cross-folder saved search clears cached results

Memory related bugs fixed in 3.0

  • bug 525646 Mass delete of messages resulted in massive memory usage (not sustained usage, and was v3-only behavior I think, i.e. not from v2)

Fixed in version 2 and / or prior to releasing version 3.0:

  • RSS
  • google desktop search
  • bug 465353 general sluggish, lack of responsiveness during indexing of gloda (indexing speed kills CPU and performance)
  • bug 518918 Thunderbird 3pre high memory and consuming 100% CPU making mail nearly unusable, need to handle UID's > 0x7FFFFFFF (fixed in 3.0.0)
  • bug 465122 Gloda: Does not close msf files on initial indexing
  • bug 387403 Thunderbird eat memory until crash when opening e-mail with broken vcard (fixed in version 2.0.0.13)
  • bug 216535 large mailboxes seem to cause pathological memory usage, was duped to ...
  • bug 266679 Thunderbird consumes excessive file-handles and memory (fixed 1.5.<something>, and before release of 2.0)

Memory Overview

Related Articles

If you don't need background information about memory, please skip to diagnosis steps.

"How much memory is needed run Thunderbird?" is a frequently asked question. System Requirements defines minimum Thunderbird machine requirements for a "mythical" user. But, these are just minimums, and do not describe how much memory you might need in your "real" environment. A better answer of what is enough memory might be ...:

a) It depends on your Thunderbird configuration (number of large folders, number of accounts, etc)
b) It depends on what you are measuring or doing in Thunderbird at a specific time, because some operations require more memory than others. Like when you are doing mundane things like reading new messages versus when you are searching for messages.
c) Current versions require roughly at least 80 MB to 100 MB more memory than version 2. So (for example) if your version 2 used 50MB, then v3 might use 130 MB to 150 MB.

Which memory to measure ...

  • MS Windows - Working Set/Mem Usage of thunderbird.exe (start taskmgr from ctrl+alt+delete, click Processes tab), and both Total and Available physical memory from the Performance tab.
  • Mac - Real Private Memory of thunderbird (process manager, right click on column bar, select Real)
  • linux - RSS/RES (Reserved Segment Size) for thunderbird

Rules of thumb (these are not "laws") for an average user with NO addons ... after a day of using Thunderbird, if memory usage is :

  • less than 250MB, then you are OK.
  • not much above your version 2 memory usage plus 100MB, then you are probably OK.
  • is over 300MB, then you may have a problem that needs analysis.
  • increasing significantly over a period of day or more, and never drops back below 300MB, i.e. it increases cumulatively over time then you may have a problem. For example, if you click on a folder and memory increases by 10MB and stays at that level or higher forever, and next time you click the same folder and memory increases another 10MB and stays at that level or higher forever ..., then you have a problem.

Note: memory usage is affected by the number and types of extensions (add-ons) installed, number of folders and accounts, number of messages in folders, and bugs.

Other Software

Add-ons / Extensions reported to have caused high memory, performance, or other problems:

Programs (firewall, antivirus, etc) reported to have caused high memory, performance or other problems:

  • ogplanet Game Launcher/npOGPPlugin.dll bug 543082 slow opening an email on windows
  • Secunia PSI
  • THE SHIELD bug 542156
  • Eset NOD32 bug 541197 Repeated attempts to create huge temporary file during message synchronization due to Eset NOD32 causes slow gloda indexing
  • F-Prot Antivirus may cause very slow access to mailbox files. See: Antivirus Related Performance Issues

Miscellaneous

Terminology

  • memory usage - varies per OS: in Windows XP taskmgr the column is 'Mem Usage', in Vista it is 'Working Set'
  • Glossary

References

Retrieved from "https://wiki.mozilla.org/index.php?title=Thunderbird:Testing:Memory_Usage_Problems&oldid=1251929"