Home Directory Storage

Widespread service disruption due to UPS failure

November 21, 2009 by IRIS Staff

Last evening about 21:26 there was a power failure in the main EECS server room in Soda Hall. The UPS held up some systems until about 22:00 at which time staff shut the room down completely. Many EECS services were disrupted as a result. The problem was found to be in the UPS which fried the high voltage output terminal block. The terminal block was bypassed and the UPS has been removed from service until further notice. Most EECS computing services have been restored as of 03:15 this morning with the exception of the IRIS web server, which was restored around 10:15am Sunday.

Thanks to the staff who came in early Sunday morning to deal with this issue.

Resolved as of 2009-11-22 03:15:00

Follow-up SAN updates

June 7, 2009 by IRIS Staff

Some of the issues with yesterday’s upgrade could not be resolved yesterday, so we’ll be addressing them today. We expect the services to stay up, but there could be minor delays/slowdowns or short outages.
[Read more…] about Follow-up SAN updates

Major Upgrades on June 6, 2009

June 6, 2009 by IRIS Staff

On Saturday, June 6th, we will be doing major upgrades to some of our
backend systems. This work will start at 10 am. We expect that some of
services will start coming back up in about 3 hours, but due to the
complexity and extensiveness of the upgrades, the downtime may extend well
beyond the 3 hour mark. These upgrades are imperative for the continuing
reliability of the infrastructure services. We apologize for the
inconvenience.

The following services will be affected:

* Home directories
* Project space
* UNIX SWW
* Departmental/IDSG web servers
* IMAP (due to the additional work, IMAP will be down for an additional 2 hours)
* argus/login (UNIX login servers)
* Department websites www.eecs / www.cs and all hosted project sites
* Department mailing lists hosted on lists.eecs
* buffy and all ACG applications
* wikis hosted on wiki.eecs
* Department’s jabber server, jabber.eecs
[Read more…] about Major Upgrades on June 6, 2009

Storage Area Network Maintenance

May 27, 2009 by IRIS Staff

There will be some minor Storage Area Network (SAN) maintenance tonight at 5pm. Services are not expected to go down, but there could be occasional minor delays or very short outages.
[Read more…] about Storage Area Network Maintenance

Home Directory Downtime

March 7, 2009 by IRIS Staff

home.eecs will go down briefly (a minute or two) twice on Saturday (03/07). The first downtime will be around noon, and the second will probably happen within the hour. This is to replace some failing hardware.

Clients that access home.eecs via Network File System (NFS) should not be affected by this scheduled maintenance. Clients that use Common Internet File System (CIFS), such as Windows and MacOS machines, may notice a disconnection (and should re-connect automatically).

Resolved as of 2009-03-07 12:50:00

Windows file service outage on home/project today

September 15, 2008 by IRIS Staff

There will be a brief (few seconds) CIFS outage at 6pm today. This will affect Windows clients and Samba users.

Most people won’t notice the brief outage, but there’s a possibility that some Windows applications with open files may get a bit confused. We recommend all affected users save any open files before 6pm and wait until a few minutes after 6pm to resume work.

The purpose of this brief outage is to resolve a problem that has prevented some Mac users from accessing our file servers. Once we have confirmed that problem has been resolved, a separate announcement will be made.

Resolved as of 2008-09-15 18:13:00

File Server Slowness

September 9, 2008 by Lars Rohrbach

Access to EECS home directories on home.eecs was particularly slow for a while this morning. For users with mail or browser profiles in their home directories, this may have prevented their mail client or browser programs from functioning normally.

Access to project.eecs does not appear to have been affected.

Staff are investigating the cause. The likely culprit are slowdowns from a new backup combined with replication.

The symptoms disappeared at 9:30am on Tuesday, Sep 09. More should be known about the root cause by 9am Wednesday.
[Read more…] about File Server Slowness

Coeus/Project Downtime

August 20, 2008 by IRIS Staff

Attention System Administrators,

On Wednesday, August 20th from 6am-8am, IDSG will be migrating production file service of home directories and project space to the new NetApp filers. There will be downtime during this migration that will impact the majority of users in the department. Please evaluate the impact for your users and notify them of those details. A much shorter public announcement will be going out soon to the entire EECS community indicating that local System Administrators will be letting them know of any specific information as to how it will impact their work.

The following services will be affected:

– coeus
– project
– sww (not winsww)
– argus
– www.eecs and www.cs web servers
– the IRIS website

The following changes apply:

1. COEUS

– coeus.EECS.Berkeley.EDU will be replaced by home.EECS.Berkeley.EDU.

– NFS access will change from coeus:/vol/vol0/eecs/username to home:/eecs/username

– The amd and automaps will be appropriately changed.

– CIFS access will change from \\\\\\COEUS\HOME\username to
\\\\\\HOME\username

– The old hostname coeus will be a CNAME to home.EECS.Berkeley.EDU until September 20th.

2. PROJECT

– NFS access will change from project:/vol/vol[02]/projspace to project:/proj/projspace

**NOTE**: A few mount points will change in a different way. See the end of this announcement for details.

**NOTE2**: If your project spaces are merged, the details will be worked out with you directly outside of this announcement.

– The amd and automaps will be appropriately changed.

– CIFS share names will not be affected.

This announcement may be followed by additional updates if additional technical details need to be publicized. Certain requests may be delayed beyond the normal response time (e.g. amd/automap change requests, new user creation, project space creation, etc.) during the 2 weeks prior to the migration.

The downtimes for different services will be different. There will be one or more additional announcements as services get restored to allow early access to the appropriate systems. E.g. the actual downtime for coeus/home is expected to be only a few minutes.

This list will receive notice when access is restored.

Sysadmins should be ready to reboot systems that need to remount filesystems between 7:30 and 8 AM. CIFS access should resume during that time as well. The CIFS share names will not change and therefore should not require a reboot for those systems.

Be advised that there are a number of follow-up tasks after the outage, which could result in slow response from these systems.

If you experience problems during or after the outage, or have
additional questions – please contact Helpdesk at help@eecs or 2-7777 or 395 Cory.

Project Space Changes Exempt from the Default Rule
OLD	NEW
project:/vol/vol2/231images	project:/proj/deptweb/231images
project:/vol/vol0/bokor	project:/proj/bokor/bokor
project:/vol/vol0/bokorgroup	project:/proj/bokor/bokorgroup
project:/vol/vol2/brass	project:/proj/brass-a
project:/vol/vol2/jordan	project:/proj/jordan-a
project:/vol/vol2/landay	project:/proj/landay-a
project:/vol/vol0/sequin-a	project:/proj/sequin/sequin
project:/vol/vol2/sequin-a	project:/proj/sequin/sequin-a
project:/vol/vol2/sequin	project:/proj/sequin/sequin-b
project:/vol/vol2/smith	project:/proj/smith-b
project:/vol/vol0/yelick	project:/proj/yelick-old

Loss of Connections to Coeus and Project

February 27, 2008 by IRIS Staff

Unfortunately, the changes we made during the scheduled maintenance window to provide better redundancy caused an unforeseen loss of connectivity to our NIS servers on one network. This resulted in loss of mounts to Coeus and Project. We rolled back the changes and service has been restored. We will try again at a later date after we resolve the issue. Unfortunately, the issue was not discovered until after our scheduled downtime was over. We regret the inconvenience this caused.

Resolved as of 2008-02-27 08:45:00