On Linux with the ext2 or ext3 file system you better not store more than 31998 directories in a directory. The same probably applies to files as well. Try this:

Dir.mkdir("dir")
33000.times {|i| Dir.mkdir("dir/#{i}") }

What I get on a Linux box is:

dir.rb:3:in `mkdir': Too many links - dir/31998 (Errno::EMLINK)

On FreeBSD using the default ufs2 there seems to be no limit. I tried 200_000 directories and it worked like a charm. Just don’t try to ls it :).

Why is this important? Because the crawler I am working on has crawled a little more than 33000 domains and now gets in trouble! I want a FreeBSD box!