We have been alerted to possible PG&E power outages due to fire danger, and that campus power may be shutdown at some point over the next 2-3 days. If we receive notice of imminent shutdown, we will work to safely shutdown our NetApp fileserver and other critical infrastructure systems; this may occur on very short notice.
We will endeavor to update this News item as we receive any updates.
We recommend you proceed with shutting down any non-essential machines in advance, e.g. shutting down workstations at the end of the work day.
[2019-10-08 5:30pm | Lars Rohrbach]
As per campus announcement, we expect to be without power in Soda and Cory by about 8am, Wednesday, October 9.
At 6am Wed, Oct 9, we will be shutting down many IRIS services dependent on local machines in our buildings (Soda, Cory, Sutardja-Dai).
At 7am Wed, Oct 9, we will be shutting down NetApp fileservice.
Of special note: mail lists hosted on lists.eecs will be unavailable during this outage. You may wish to email individuals, rather than lists, as appropriate.
For machines at the Earl Warren Hall Data Center, staff have messaged that “… the Data Center will be automatically switching to emergency generator power. For our UPS-backed systems, there will be no expected outage during the transfer. Our Non-UPS powered systems will, however, experience a total loss of power for the duration of the outage. It is highly recommended that all non-UPS powered systems be gracefully shut down prior to this outage.”
See also https://technology.berkeley.edu/pge-outage/faqs for the campus IT FAQ on the PG&E outage.
[2019-10-09 7:50pm | Lars Rohrbach]
Based on communications from campus, we are expecting to resume normal service for the workday tomorrow, Thursday, October 10, provided that we do not experience any actual PG&E outage overnight.
Our current plan is to resume services as follows:
At approx. 5am Thu, Oct 10, restart EECS network infrastructure, followed closely by DHCP and DNS service.
Beginning at 5:30am, begin bringing up NetApp fileservice and associated systems.
Remaining infrastructure services and systems will be brought up beginning at 6:30am, and we hope to have all service restored by approx. 7am.
[2019-10-10 4:30am | Lars Rohrbach]
Campus power was shut down by PG&E, so this startup work is postponed.
[2019-10-10 12:00pm | Lars Rohrbach]
With limited power being supplied by campus co-generation, we have now restarted network service, DHCP, DNS, and LDAP. Fileservice and other dependent systems remain shutdown, to reduce electrical load.
Campus SSIDs such as AirBears2, eduroam, and CalVisitor are up in our locations, but EECS-Secure may not yet work.
[2019-10-11 4:30pm | Lars Rohrbach]
With notice of stable power from PG&E, we began restoring services this afternoon. By 4:15pm, fileservice and other dependent services were generally restored. We will be actively monitoring our systems over the coming days to see that all is well.
Please report any issues to help@eecs.
Resolved as of 2019-10-11 4:15pm