What is Q4M?
Q4M (Queue for MySQL) is a message queue licensed under GPL that works as a pluggable storage engine of MySQL 5.1, designed to be robust, fast, flexible. It is already in production quality, and is used by several web services (see Users of Q4M).
To start using Q4M, download either a binary or source distribution from the install page, and follow the installation instructions. A small tutorial is also avialable. You may use SQL to access Q4M queues, or there is a wrapper module available for perl (Queue::Q4M).
For more information, please read the developer's weblog (Kazuho at Work) or subscribe to the mailing list.
What's New
- 0.8.9 release (Oct 26 2009)
- fix memory corruption when FLOAT or DOUBLE fields where used, fix deadlock under some situations calling queue_wait with multiple tables, fix compile error on GCC 4.3, etc.
- slides at MySQL Conference & Expo 2009 (Apr 22 2009)
- presentation slides used at MySQL Conference & Expo 2009 is available here.
- 0.8.5 release (Apr 16 2009)
- no bug fixes; bundle required boost header files
- 0.8.4 release (Feb 09 2009)
- fix crash on linux (i386) platform when a single tables becomes larger than 2GB
- 0.8.3 release (Sep 01 2008)
- fix race condition error that might lead to deadlock on shutdown, add support for FreeBSD (fix compilation problems and release of prebuilt binary, now also available as a FreeBSD port)
- 0.8.1 / 0.8.2 release (Aug 29 2008)
- fix file descriptor leak on DROP TABLE (regression in 0.8), adjustment to fix build error under certain environments. Version 0.8.2 is released only as prebuilt binaries for linux (to fix interoperability problems)
Design Goals of Q4M
- Robust
- Q4M logs all message transfers to disk. A power failure or OS crash would never loose a message.
- Fast
- Although Q4M syncs every operation to disk, it runs fast by using group commits, and checksums for data recovery. Even with HDD write cache disabled, it is able to transfer thousands of messages per second on an ordinal linux server.
- Flexible
- Since Q4M works as a storage engine of MySQL, ordinally SQL can be used for administration. It is also possible to JOIN Q4M queues against other MySQL tables. Or you may use triggers for sending data into queue.
Users of Q4M
- Pathtraq - one of Japan's largest web traffic statistics service
- Mixi Echo - microblogging service of Japan's largest SNS provider
Limitations and Known Issues
Limitations, known issues under version 0.5.
- row size is limited to 512MB
- table size is limited to 2**63 bytes
- no support for UPDATE and REPLACE
- no support for indexes
- removal of multiple rows from a single DELETE statement is not atomic
- may cause server crash under extremely low memory condition
- may lose data if OS crashes while Q4M is performing table compaction (since the directory is not fsynced)