DISKD refers to some features in Squid-2.4 to improve Disk I/O performance. The basic idea is that each cache_dir has its own diskd child process. The diskd process performs all disk I/O operations (open, close, read, write, unlink) for the cache_dir. Message queues are used to send requests and responses between the Squid and diskd processes. Shared memory is used for chunks of data to be read and written.
Yes. We benchmarked Squid-2.4 with DISKD at the Second IRCache Bake-Off. The results are also described here. At the bakeoff, we got 160 req/sec with diskd. Without diskd, we'd have gotten about 40 req/sec.
You need to run Squid version 2.4 or later. Your operating system must support message queues, and shared memory.
To configure Squid for DISKD, use the --enable-storeio option:
% ./configure --enable-storeio=diskd,ufs
You didn't put diskd in the list of storeio modules as described above. You need to run configure and and recompile Squid.
No. Diskd uses the same storage scheme as the standard "UFS" type. It only changes how I/O is performed.
Most Unix operating systems have message queue support by default. One way to check is to see if you have an ipcs command.
However, you will likely need to increase the message queue parameters for Squid. Message queue implementations normally have the following parameters:
Maximum number of bytes in a single queue.
Maximum number of message queue identifiers.
Maximum number of message segments.
Maximum size of a message segment.
Maximum number of messages in the whole system.
The messages between Squid and diskd are 32 bytes. Thus, MSGMAX should be 32 or greater. You may want to set it to a larger value, just to be safe.
We'll have two queues for each cache_dir -- one in each direction. So, MSGMNI needs to be at least two times the number of cache_dir's.
MSGMNB and MSGTQL affect how many messages can be in the queues at one time. I've found that 75 messages per queue is about the limit of decent performance. Thus, MSGMNB must be at least 75*MSGMAX, and MSGTQL must be at least 75 times the number of cache_dir's.
Your kernel must have
options SYSVMSG
You can set the parameters in the kernel as follows. This is just an example. Make sure the values are appropriate for your system:
options MSGMNB=16384 # max # of bytes in a queue options MSGMNI=41 # number of message queue identifiers options MSGSEG=2049 # number of message segments options MSGSSZ=64 # size of a message segment options MSGTQL=512 # max messages in system
Message queue support seems to be in the kernel by default. Setting the options is as follows:
options MSGMNB="8192" # max # bytes on queue options MSGMNI="31" # # of message queue identifiers options MSGMAX="2049" # max message size options MSGTQL="1024" # # of system message headers
If you have a newer version (DU64), then you can probably use sysconfig instead. To see what the current IPC settings are run
# sysconfig -q ipcTo change them make a file like this called ipc.stanza:
ipc: msg-max = 2049 msg-mni = 31 msg-tql = 1024 msg-mnb = 8192then run
# sysconfigdb -a -f ipc.stanzaYou have to reboot for the change to take effect.
In my limited browsing on Linux, I didn't see any way to change message queue parameters except to modify the include files and build a new kernel. On my system, the file is /usr/src/linux/include/linux/msg.h.
Refer to Demangling Message Queues in Sunworld Magazine.
I don't think the above article really tells you how to set the parameters. You do it in /etc/system with lines like this:
set msgsys:msginfo_msgmax=2049 set msgsys:msginfo_msgmnb=8192 set msgsys:msginfo_msgmni=31 set msgsys:msginfo_msgssz=64 set msgsys:msginfo_msgtql=1024
Of course, you must reboot whenever you modify /etc/system before changes take effect.
Shared memory uses a set of parameters similar to the ones for message queues. The Squid DISKD implementation uses one shared memory area for each cache_dir. Each shared memory area is about 800 kilobytes in size. You may need to modify your system's shared memory parameters:
Maximum number of shared memory segments per process.
Maximum number of shared memory segments for the whole system.
Largest shared memory segment size allowed.
Total amount of shared memory that can be used.
For Squid and DISKD, SHMMNI and SHMMNI must be greater than or equal to the number of cache_dir's that you have. SHMMAX must be at least 800 kilobytes. SHMALL must be at least SHMMAX 800 kilobytes multiplied by the number of cache_dir's.
Your kernel must have
options SYSVSHM
You can set the parameters in the kernel as follows. This is just an example. Make sure the values are appropriate for your system:
options SHMSEG=16 # max shared mem id's per process options SHMMNI=32 # max shared mem id's per system options SHMMAX=2097152 # max shared memory segment size (bytes) options SHMALL=4096 # max amount of shared memory (pages)
Message queue support seems to be in the kernel by default. Setting the options is as follows:
options SHMSEG="16" # max shared mem id's per process options SHMMNI="32" # max shared mem id's per system options SHMMAX="2097152" # max shared memory segment size (bytes) options SHMALL=4096 # max amount of shared memory (pages)
If you have a newer version (DU64), then you can probably use sysconfig instead. To see what the current IPC settings are run
# sysconfig -q ipcTo change them make a file like this called ipc.stanza:
ipc: shm-seg = 16 shm-mni = 32 shm-max = 2097152 shm-all = 4096then run
# sysconfigdb -a -f ipc.stanzaYou have to reboot for the change to take effect.
In my limited browsing on Linux, I didn't see any way to change shared memory parameters except to modify the include files and build a new kernel. On my system, the file is /usr/src/linux/include/asm-i386/shmparam.h
Oh, it looks like you can change SHMMAX by writing the file /proc/sys/kernel/shmmax.
Refer to Shared memory uncovered in Sunworld Magazine.
To set the values, you can put these lines in /etc/system:
set shmsys:shminfo_shmmax=2097152 set shmsys:shminfo_shmmni=32 set shmsys:shminfo_shmseg=16
Yes, this is a little problem sometimes. Seems like the operating system gets confused and doesn't always release shared memory and message queue resources when processes exit, especially if they exit abnormally. To fix it you can ``manually'' clear the resources with the ipcs command. Add this command into your RunCache or squid_start script:
ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s\n", $1, $2}' | /bin/sh
In the source code, these are called magic1 and magic2. These numbers refer to the number of oustanding requests on a message queue. They are specified on the cache_dir option line, after the L1 and L2 directories:
cache_dir diskd /cache1 1024 16 256 Q1=72 Q2=64
If there are more than Q1 messages outstanding, then Squid will intentionally fail to open disk files for reading and writing. This is a load-shedding mechanism. If your cache gets really really busy and the disks can not keep up, Squid bypasses the disks until the load goes down again.
If there are more than Q2 messages outstanding, then the main Squid process ``blocks'' for a little bit until the diskd process services some of the messages and sends back some replies.
Q1 should be larger than Q2. You want Squid to get to the ``blocking'' condition before it gets to the ``refuse to open files'' condition.
Reasonable values for Q1 and Q2 are 72 and 64, respectively.