Resolved as of 2013-08-16 06:30:00
cluster.eecs ran out of inodes
cluster.eecs ran out of inodes at 4:12am. We’ve increased the number of inodes by 20%. To prevent this problem in the future we will monitor the inode usage in addition to the space usage and adjust the inodes available before it runs out again.