As always, we apologize for all inconveniences caused by the network’s performance.
[2015-01-13 18:58:31 | Lars Rohrbach]
Our network engineers are continuing to investigate degraded wireless and wired networking. The problems relate to the recent upgrade of our core network infrastructure. Stabilizing the network is our top priority.
Symptoms of the network degradation include dropped connections to wireless networks (including AirBears and other campus networks, as they are routed on our equipment), slow/laggy communication (web browsing, printing, accessing home/project/cluster), slow ping response times and dropped packets.
[2015-01-16 16:07:37 | Derek Calderon]
We believe that the majority of the wireless issues have been resolved as of 1:30 PM today. If you are still experiencing performance issues (worse than those which preexisted in December) please email email@example.com.
The network issues over the past two weeks appear to have been caused by preexisting issues in our routing and switching topologies which were revealed and exacerbated after the routing upgrade on January 3rd. Without going into painful detail, the primary issue was overextension of several layer 2 VLANs (causing switching issues which led to dropped packets and high latencies), and of our network management VLAN in particular. This was mitigated by removing several devices from the management network (in favor of L3 loopback interfaces) and by shrinking the footprint of the VLAN. We also relocated our wireless controllers from the management subnet onto a new wireless management net, thereby removing the management network as a factor in the performance of wireless clients.
We also experienced issues with our new OSPF topology, which were cleared up after enabling passive mode on non-internal subnets and by removing redundant neighbor adjacencies. OSPF was also made independent of the aforementioned management network.
We greatly apologize for all of the issues you’ve had to put up with over the past weeks, and hope that the worst is now behind is. If you’re experiencing ongoing issues with the network, please email firstname.lastname@example.org.
Resolved as of 2015-01-16 13:30:00