Header Search Widget
IRIS

Instructional & Research Information Systems

Department Webservice

Department Webservice

Unscheduled Outage of Several Departmental Services

by

A hardware failure and caused a temporary outage to the department’s Storage Area Network (SAN). The department’s web site, mailing list server, jabber serve, and the ACG application server (buffy) were all affected.

The virtual cluster that hosts these services is being restarted and services should resume by 3:30 pm.

Resolved as of 2010-11-11 15:05:00

Dept Website, Mailing Lists, ACG Applications down 4/1, 9pm

by

The department webserver (www.eecs/www.cs), mailing list server (lists.eecs), Jabber server (jabber.eecs) and ACG’s application server (buffy.eecs) will be down for maintenance 9pm-midnight on Thursday, April 1.

These services all run on a virtualized cluster. The cluster storage will be undergoing maintenance that requires all systems to be shut down. We expect the total downtime to be less than 2 hours, but are scheduling 3 in case of unexpected problems.

This is a reschedule of the maintenance of March 12th which was not successfully completed. I have installed some additional patches on the system which should avoid the problem which occurred on 3/12.

If successful, this will be the last time we ever have to do this very intrusive maintenance. Additionally, this will allow us to retire some buggy software on our SAN which has been the cause of previous department-wide storage outages.

Resolved as of 2010-04-01 23:11:00

Dept Website, Mailing Lists, ACG Applications down 8-9pm

by

The Dept Website, Mailing Lists server and, ACG’s Application server will be down 8-9pm tonight for a storage migration. The Storage Area Network (SAN) that these servers utilize is being reconfigured, requiring the server data to be copied to a new area.

Beginning at 7pm these services will be operating in a degraded mode while preparations are made. During this time less computing power will be available and the web servers might be slightly slower than normal. At 8pm all servers will all be shut down while the data is copied over. This should take approximately 20-30 minutes, but we are scheduling 1 hour in case of unexpected problems.

This maintenance will allow us to retire some buggy SAN management software which has been the cause of most of our recent storage outages.
[Read more…] about Dept Website, Mailing Lists, ACG Applications down 8-9pm

Brief Outage for Dept Website and Other Services, Thursday 8am

by

The Department website (www.eecs/www.cs), mailing list server (lists.eecs) and ACG application server (buffy.eecs) will be offline from around 8am-8:15am Thursday morning.

The Storage Area Network (SAN) that serves these machines will be reconfigured at 8am which will result in a brief outage. Ideally all services will remain available but service might be disrupted for ten to fifteen minutes.
[Read more…] about Brief Outage for Dept Website and Other Services, Thursday 8am

Unplanned outage of some IRIS Services

by

The Storage Area Network (SAN) that hosts the virtual machine images for www.eecs, buffy.eecs and lists.eecs briefly stopped serving the storage around 1:45am. The virtual machines stopped at that time and were restarting this morning around 9:45am.

The SAN software that is causing this problem is due to be retired in the next few weeks.

Resolved as of 2009-12-29 09:45:00

December 28th Shutdown

by

IMAP/mail, home/project, Windows services (AD/WINS/printing), LDAP/NIS/DNS, DHCP and most other IDSG and IMG services will be affected. We will try to keep some basic network services running (DHCP, DNS/NIS), possibly in a reduced capacity.

While EECS-Secure, EECS-Guest and EECS-Open wireless LANS will be out of service during this time, the EECS and AirBears wireless LANS will remain operational during the outage.

The reason for this extended downtime is multi fold. The first is to repair the UPS that failed a couple of weeks back. The repaired unit will be placed back on line and extensive testing must be done to ensure it’s operation before we can depend on the UPS again. The second is to move data around which will allow us to remove from service a piece of software used to manage storage. This software is no longer under development and we need to remove our dependency on it. Lastly, some routine maintenance will be done which can only be done when systems are down. These are the kind of things that can only be done between semesters and we ask for you understanding during this disruption in service.

While the stated outage is from 10:00AM to 9:00PM, it may not take that long. This is a worst case scenario. If possible, services will be brought up sooner than 9:00 PM.
[Read more…] about December 28th Shutdown

EECS Website, Mailing Lists and IM services down since 10:30PM

by

Starting around 10:30PM tonight, the EECS website, Mailing List and IM services (aka www.eecs, lists.eecs and Jabber) went down due to a problem with EECS Storage.

While EECS Storage is back up and most affected services — including access to home directories, project storage, Software Warehouse, and IMAP — recovered, Maillists and IM stayed down due to a backend database server that had not yet recovered from the storage outage. The Xen cluster that provides these services lost all contact with the SAN and and to be restarted.

Staff are working now to bring these remaining services back up.

In the meanwhile, if you are still having problems accessing your home directory, project storage, Software Warehouse, or IMAP, please contact the EECS Helpdesk.

Resolved as of 2009-12-22 00:33:00

Yearly SSL Renewal for Department Servers

by

New SSL certificates will be installed at 10pm tonight (Monday) on www.eecs, lists.eecs, jabber.eecs and iris.eecs.

Each of these services will be restarted between 10pm and 10:15pm tonight, Monday, December 14th, 2009. Brief outages may be noticeable, but no server should be unavailable for more than a minute or two.

Please note that as recommended by our Certificate Authority we are moving to 2048 bit keys. Please report any compatibility issues to IRIS and your software vendor.
[Read more…] about Yearly SSL Renewal for Department Servers