Home Directory Storage

Urgent NetApp Fileserver Downtime, Saturday July 8, 2023

July 6, 2023 by Lars Rohrbach

We will be scheduling an urgent 4 hour Maintenance to update our NetApp fileserver from 8am to 12pm on Saturday, July 8, 2023. This update must be done prior to installing Microsoft Updates on AD Domain Controllers scheduled for release on Jul 11^th to secure the Netapp and maintain future connectivity with Microsoft AD Domain connection for our CIFS/SMB Shares.

All Home directories and Project shares (CIFS/SMB and NFS) may experience disconnection during the update that may cause data loss on active, open files. Please close your open files and save them before Saturday Morning. Services (e.g. webservice) that depend on this storage will also be down or unresponsive.

When maintenance is completed, NFS clients should automatically resume normal operation, but a reboot should help if they don’t. CIFS clients (such as Windows and MacOS) may need to remount filesystems, or simply be rebooted.

[Read more…]

CIFS Shares From home & project Unresponsive

March 15, 2023 by Lars Rohrbach

We are investigating reports that Windows/CIFS shares from home.eecs and project.eecs became unavailable around 3pm today.

[Read more…]

Firewall Migrations for IRIS NetApps and Instructional Servers, 3/21 6AM

March 17, 2022 by Derek Calderon

At 6AM on Monday, March 21, we will be migrating the following networks to our new border firewall:

169.229.3.96/27 (Instructional Support Group servers)
169.229.3.128/26 (IRIS NetApp file servers, including home and project shares)

There will be a brief interruption to these networks as they are migrated. This will affect all home, project, and other file storage provided by the IRIS NetApps. After the migration, we encourage system administrators to test that connections to the NetApps are behaving as expected. Some file service clients may need to be rebooted or have their nfs service restarted.

Please contact us at help@eecs.berkeley.edu with any questions or concerns.

[Read more…]

Change of Some Fileservice IP Addresses March 15

March 8, 2022 by Lars Rohrbach

Our current NetApp fileservice architecture has multiple hostnames pointing to the same IP address. As a performance measure, we will be introducing some new IP addresses for some of the hostnames, very early on Tuesday, March 15. If you have any custom firewall rules for the current IP addresses, you may need to modify them to include the new ones.

This change should not create any interruption in service. Any active mounts of the former IP addresses will continue to work.

[Read more…]

Dept NetApp Filer Maintenance, 3pm January 3, 2022

December 21, 2021 by Lars Rohrbach

We will be performing an OS upgrade on our departmental NetApp filers (upgrading from OnTAP version 9.7 to 9.8), which will require a service outage from 3pm to 9pm on Monday, January 3, 2022. All NFS and CIFS access to the filer (home and project storage) will be down during that time. Services (e.g. webservice) that depend on this storage will also be down or unresponsive.

While we are scheduling a 6 hour maintenance window, we will attempt to minimize the service outage if possible.

When maintenance is completed, NFS clients should automatically resume normal operation. CIFS clients (such as Windows and MacOS) may need to remount filesystems, or simply be rebooted.

[Read more…]

Soda Hall Power Shutdown Tuesday 8/18 8am-8pm

August 12, 2020 by Derek Calderon

On Tuesday, August 18th, Soda Hall building power will be shutdown from approximately 8am to 8pm for required facilities work. There will also be sporadic brief outages to various IRIS services between 9am and 2pm due to network maintenance.

Our primary datacenter and network core will receive backup generator support and we generally expect all services to remain online throughout the day. However, we are taking the opportunity to perform a number of network maintenance items which would have otherwise necessitated multiple outages later in the year.

As a result there will be multiple brief interruptions to various services including NetApp file servers, DHCP and DNS service, and the login.eecs and winterm terminal servers. We expect each individual interruption will last for less than five minutes, and this work will be performed between 9AM – 2PM. This post will be updated when all network maintenance is complete.

After network services are fully restored, sysadmins may need to reboot their systems, restart/remount nfs file systems, etc. to get fully back online.

[Read more…]

NetApp archive backups not working

October 17, 2019 by Lars Rohrbach

When fileservice was restored after the PG&E outage last Friday, the part of the service that handles backups became non-functional. Staff are working to restore backup service as soon as possible.

Snapshots remain functional and available, providing easy access to file versions from the last 7 days. Older archive backups, from before October 9, also remain available. The broken archiving process means that backups after Oct 9 will not be available.

We make nightly copies of current fileserver data to our Disaster Recovery (DR) site at the San Diego Supercomputing Center (SDSC). This mirror of current data remains operational, and is not affected by the archive process failure.

Potential Campus Power Outage

October 8, 2019 by Lars Rohrbach

We have been alerted to possible PG&E power outages due to fire danger, and that campus power may be shutdown at some point over the next 2-3 days. If we receive notice of imminent shutdown, we will work to safely shutdown our NetApp fileserver and other critical infrastructure systems; this may occur on very short notice.

We will endeavor to update this News item as we receive any updates.

We recommend you proceed with shutting down any non-essential machines in advance, e.g. shutting down workstations at the end of the work day.

[Read more…]

NetApp fileserver maintenance – Phase III: Mon, July 22, 6-8pm

July 6, 2019 by Lars Rohrbach

On Monday evening, July 22, we will be taking the next step in our fileserver migration: moving network interfaces from the old system to the new system. This will result in brief interruptions of access to home directories and project shares.

Our expectation is that each move of a network interface should only take a minute or so, and that most NFS and CIFS clients will likely recover just fine without a reboot. Any ongoing file copies or moves will be interrupted and terminated; any unsaved file editing should be closed before the maintenance period.

For any systems experiencing issues after maintenance is completed, a reboot would be recommended.

[Read more…]

NetApp fileserver maintenance – Phase II: Wed, June 19, 12-2pm

June 6, 2019 by Lars Rohrbach

As part of our NetApp fileserver refresh cycle, we have a NetApp technician scheduled to be on site June 19, to assist with installation of the new hardware, and transfer of storage racks from the old NetApp system to the new one. Work is planned to take place between noon and 2pm.

Our expectation is that this work will not be disruptive to existing fileservice, but it involves a good amount of work around the existing hardware, so there is potential for unexpected problems.