Problem
Some stdio stream implementations suffer from a limitation or backward incompatibility. Most notably, Solaris 9 and previous versions using the LWP32 programming model have a FILE datastructure where the file descriptor field has a width of only 8 bits. Therefore stdio streams are only capable of handling file descriptors in the range from 0 to 255.
Some applications access the FILE datastructure components directly (e.g. the _file field), especially those written in the past few years. The structure has not been changed to allow for more file descriptors, which has proven to break newer applications (binaries).
Solaris 9 offers a maximum number of 65536 file descriptors, but only descriptors between 0 and 255 can be stored in the FILE datastructure. If a descriptor value greater than 255 is passed to e.g. fdopen(), the function simply fails with errno unset!
Solutions
Solaris 10
Solaris 10 offers several solutions to cope with this situation (based on source or binary applications, 64bit or 32bit environments or code generation respectively).
- Oracle offers a run-time solution for a 32bit environment (64bit environment do not have this restriction), see Solaris OS Solutions to 32-Bit stdio's 256 File-Descriptors Limitation:
Starting with the Solaris 10 release slated for July 2007, Sun will offer runtime and programming solutions in the form of an extended FILE facility to alleviate stdio's limitation of 256 file descriptors. Systems running the Solaris Express release or any OpenSolaris OS distribution after build 39 will have these solutions. This article's Patches and Bugs section contains instructions for installing the extended FILE facility on systems running any release from Solaris 10 3/05 to Solaris 10 11/06.
- Replacement library like SFIO, see Solaris Developers Note.
Solaris 9 (or previous versions)
They are several solutions:
- Do not use more then 256 descriptors - not recommended these days. Only very low volume mailservers seem to cope with this limitation. See References: Sol2 FAQ: 3.48) How can I increase the number of file descriptors per process?
- Use a STDIO replacement library, e.g. SFIO. Just get it work to get compiled - not tested yet.
- Filedescriptor pool workaround. This is builtin into Milter-greylist since version 4.0 (to be enabled separately).
Filedescriptor pool workaround
Milter-Greylist includes (since version 4.0) a work-around by means of the fd_pool module which is enabled by setting the configure option —enable-stdio-hack.
This module provides a replacement for the functions fdopen() and fclose(). The idea is to reserve a pool of open descriptors (associated with /dev/null) with values <256 which are aquired as needed by the fdopen() replacment.
On the other hand the fclose() replacement tries to return a descriptor lower than 256 to the reserved pool. The latter is not assured in all cases because the real fclose() call releases the associated descriptor which could not be reclaimed without a delay. A parallel running thread could grab the closed descriptor during the delay …
The whole system works only in situations where not more then 256 streams are used in parallel despite the maximum number of open descriptors could grow much greater.
This workaround must be enabled during compilation by defining USE_FD_POOL=1.
SFIO Library
Compilation:
mkdir sfio
cd sfio
gtar xvfz ../sfio.2005-02-01.tgz
set path= ( $path `pwd`/bin )
cd src/lib/vthread
make install CC=gcc CCMODE="-O2 -g -Wall"
cd ../sfio
make install_mt CC=gcc CCMODE="-O2 -g -Wall"
# or
./Runmake.solaris.gcc
