[Complete] Multiple Services affected by emergency electrical work 3/21/09

[Update 3/21/09 10:13 AM] The emergency power repairs have been completed, and all systems have been restored to normal.

Due to emergency electrical work in the server room, the following services will be unavailable from 8:00 AM to 1:00 PM Saturday, 3/21/09:

• Moodle
• itech.ups.edu Web site
• ASUPS Web server and file server
• Sophos PureMessage quarantine Web site (mail delivery will be unaffected)
• Kronos server (electronic time cards)

Web Site Navigation Not Displaying

[Update 3/6/09 10:41 AM] This error has been resolved; however,

The right navigation on academic departments and programs pages linked from http://www.ups.edu/academicdepartments.xml is not displaying at this time. The missing navigation on these pages includes links to departmental and program Web sites. The problem has been referred to the vendor of our content management system, Ingeniux, and they promise a solution today.

We will add temporary links to the department and program Web sites until this issue is resolved.

CMS maintenance required

I am shutting down the CMS server in order to perform needed maintenance on the system. This is required because some pages are missing their navigation. Folks who use the CMS will only be affected for a few minutes; however, publishing will be slow for the next 90 minutes.

The next scheduled CMS will be the afternoon of March 26.

[Resolved] Alexandria Shares Offline Saturday 2/28/09

On Saturday, February 28th from 6-8am, Alexandria will be offline for storage expansion and maintenance.  During the outage, student home directories (H: drive) and department shares will be inaccessible.

This change is required to expand the amount of disk space available on Alexandria and to improve backup performance.

[Update 2/28/2009 @ 8:00am]  Normal service on Alexandria was restored by 7:15am this morning.

Whitney Retired

The Windows 2000 server known as Whitney was retired today at 2:13pm.  It is not known to be used anymore, but writting this entry just in case something is dependant on it and breaks after today.  A special one-time commvault backup was done, written to tape 104L3 with 365 days retention.

[Resolved] CX400 Battery Replacement Tuesday February 17th, 5:30AM

On February 17th from 5:30-6AM batteries in our EMC CX400 Disk Array will be replaced.  This is a fully redundant system and no outage is expected.  However, as with any change, there is always a risk of outage.  If a problem arises with the change, the following services may be impacted:

email
CRM
Database Services (FAMIS, Banner, Millennium, etc.)

Please note that Cascade will not be affected.

ARP problem with 192.124.98.129 2/7/2009

At about 2:15 PM today, the ARP entry for 192.124.98.129 (ezproxy.ups.edu) was incorrect on the network core router. This of course caused the Ezproxy server to go offline. The ARP cache was cleared, and the correct entry was reestablished. The cause for this has not been discovered yet. A duplicate IP address is suspected.

[RESOLVED] Mail to and from the Internet is not flowing 2/7/2009

Since 5:20 AM this morning most email has not been delivering to and from the Internet. We are addressing the problem right now, and hope to have this resolved within the hour.

This problem was due to a routing misconfiguration during this morning’s scheduled maintenance. Mail deliver was restored at 10:30 AM. Mail sent to or from the Internet will be delayed 1-2 hours while the mail servers process the backlog.