On Tuesday, August 18th, Soda Hall building power will be shutdown from approximately 8am to 8pm for required facilities work. There will also be sporadic brief outages to various IRIS services between 9am and 2pm due to network maintenance.
Our primary datacenter and network core will receive backup generator support and we generally expect all services to remain online throughout the day. However, we are taking the opportunity to perform a number of network maintenance items which would have otherwise necessitated multiple outages later in the year.
As a result there will be multiple brief interruptions to various services including NetApp file servers, DHCP and DNS service, and the login.eecs and winterm terminal servers. We expect each individual interruption will last for less than five minutes, and this work will be performed between 9AM – 2PM. This post will be updated when all network maintenance is complete.
After network services are fully restored, sysadmins may need to reboot their systems, restart/remount nfs file systems, etc. to get fully back online.
As most of the building will be without power, computers and other electronic devices in offices and other locations in Soda Hall should be powered off.
Administrators of machines in 340 Soda should also ensure that all machines are properly shutdown prior to 8am on the 18th. If you would like us to power your machines back on after the outage, please email email@example.com with clear descriptions of which buttons we should press and where they are located.
[2020-08-18| larsrohr ]
We are investigating a fileserver issue that arose during this work. Currently, some users may not be able to access their files, either via NFS (unix) or CIFS (Windows/Mac). We expect to have this issue resolved later this afternoon.
[2020-08-18| larsrohr ]
We apologize greatly for the interruption in service. We had attempted to make a change to the NetApp fileserver in preparation for the upcoming retirement of NIS; this change was tested in our Disaster Recovery site, and was intended to be transparent to the end user, but it clearly had much greater repercussions than expected.
After fully reverting the change, and flushing credentials for any remaining fileshares reporting errors, fileservice has returned to normal.
As of about 6pm, building power was restored; by about 6:45, our machine rooms were transferred back to grid power from generator.
Resolved as of 2020-08-18 7pm