I love Firefox. Don’t get me wrong here. It allows me to customize my browsing environment to exactly how I like it, has a ton of great extensions, and generally lets me feel like I have more ownership of my browser.
What I don’t like is that it had gotten slow lately. I’m not talking about just plain takes a few more seconds to launch slow, but more like launching Firefox required a good 3 minutes to fully finish whatever it was doing and become usable.
So I did a little digging and here’s what I found.
Firefox 3 now uses a local sqlite database to store many of it’s settings like bookmarks, browsing history, favicons, etc. This is a good thing in general, as the previous storage format for those things was ugly. Now we have a full-fledged database to poke with a sharp stick should we choose to.
Greatness of using sqlite aside, databases can become bloated when you do a lot of adds/updates/deletes (especially deletes) if they don’t get the proper maintenance. Sqlite databases use VACUUM for this basic maintenance, so that’s the first thing to try if Firefox is feeling sluggish. It’s a fairly easy operation, and it’s well documented on the net with regard to Firefox. In general, you use your command line access (in my case, Terminal.app) and navigate to your Firefox profile directory. Once there, you can issue the following command to VACUUM your places.sqlite database:
sqlite3 places.sqlite VACUUM;
That was easy.
You can perform this same VACUUM procedure on the other sqlite databases Firefox uses, one by one (do a ls *.sqlite in your profile directory to find them all) or you can take my little inline shell script and do them all in one fell swoop:
for z in `/bin/ls *.sqlite`; do sqlite3 $z VACUUM; done
This assumes 3 things: you’re using a bash as a shell, /bin/ls is where your system ls lives, and sqlite3 is in your PATH. If you’re on Mac OS X, you can copy/paste that once you’ve changed to your profile directory and it’ll work fine. Other operating systems, well, you’re on your own, but it’s a simple thing to track down those bits.
What that little inline script says, in a nutshell, is “find all files ending in sqlite in the current directory, then for each of them, call sqlite $filename with the VACUUM; argument to perform the maintenance”.
That’s the easiest thing you can do and many people have mentioned that it does indeed help in many situations. That said, it didn’t help mine much.
My places.sqlite database had grown to over 60Mb. I do a lot of browsing in my day-to-day job, but 60Mb worth of history is A LOT of entries, no matter how you dice it. After continuing to hear Firefox thrash my disk for a good 2-3 minutes every time I started it, I decided to dig in and see what it was doing.
Thank God for Mac OS X and DTrace.
One of my favorite things about UNIX in general, is that there are so many neat tools for observing what is going on in your computer. One of those tools is called iosnoop, and it comes courtesy of the inclusion of DTrace in Mac OS X Leopard. iosnoop is actually just a bash script wrapper for a DTrace script, albeit with quite a few knobs and buttons.
[rts-xeon:~] ryan$ iosnoop -h
USAGE: iosnoop [-a|-A|-DeghiNostv] [-d device] [-f filename]
[-m mount_point] [-n name] [-p PID]
iosnoop # default output
-a # print all data (mostly)
-A # dump all data, space delimited
-D # print time delta, us (elapsed)
-e # print device name
-g # print command arguments
-i # print device instance
-N # print major and minor numbers
-o # print disk delta time, us
-s # print start time, us
-t # print completion time, us
-v # print completion time, string
-d device # instance name to snoop
-f filename # snoop this file only
-m mount_point # this FS only
-n name # this process name only
-p PID # this PID only
eg,
iosnoop -v # human readable timestamps
iosnoop -N # print major and minor numbers
iosnoop -m / # snoop events on filesystem / only
Well, firing up iosnoop as root (which is required to access the DTrace probes) is as easy as issuing
sudo iosnoop
in Terminal.app.
When I did that while Firefox was grinding through it’s startup, I was still seeing TONS of reads and writes to it’s places.sqlite database. Doing a bit of Google research told me I wasn’t the only one.
So I quit Firefox (it locks the sqlite database) and fired up my old friend sqlite3 to take a look at the database and found that moz_places had over 175,000 rows. 155,000 of those rows were marked as hidden (hidden = 1) which means that they were never visited in the browser, and “are commonly embedded pages, i-frames, RSS bookmarks and javascript calls” (via firefoxforensics).
Well, I’ve never once used a Live Bookmark and don’t need to keep the history of those other items, so I decided to just delete them all. You can do that using sqlite3 very easily with a quick
[rts-xeon:~/Library/Application Support/Firefox/Profiles/rts] ryan$ sqlite3 places.sqlite
SQLite version 3.4.0
Enter ".help" for instructions
sqlite> delete from moz_places where hidden=1 and url like 'http%';
sqlite> .quit
That SQL statement tells sqlite to delete all the hidden items that have a url beginning with http. It’s important not to just delete all the hidden items, as some of them are required for Firefox to operate. I found that out the hard way, and am glad I made a copy of my places.sqlite before deleting all the hidden items and breaking my Firefox history/bookmark manager entirely.
Once you’ve deleted those hidden items, your places.sqlite database should decrease significantly in size. Mine dropped from 61Mb or so down to around 9Mb. There are still 10,000 items in my history, but those are all places I’ve explicitly visited so I’m fine with that.
Launching Firefox is no longer a 3 minute endeavour – it comes up about as fast as Safari does and I get to have my cake and eat it too.
Hope this article helps someone, and feel free to leave your comments/experience below.
