Outgoing Email Outage on 11/6 from 5:30-7am (Update- Completed at 6:10am)

On the morning of November 6th at 5:30am messages sent from inside the University to external receipients will be delayed until 7am.  The outage does not affect messages from University users to other University users.

Update:  This outage is compete as of 6:10am on 11/6.  All outgoing mail should now be back up to speed.  Please contact the helpdesk if you see any irregular behavior.

New Storage Array and SAN Installation Complete 10/27/2008

The Installation of the new storage array and SAN to support upgrades of the database and email systems was completed with the reboots of all database servers from 5:30 to 6:30 AM this morning. CRM was restored at 7:45 AM. The extended timeframe was necessary to ensure that all existing and new LUNs had redundant paths to their hosts in the SAN.

Aruba Wireless

Some users were not getting IP addresses when connecting to Aruba Wireless.  This was do to a shortage of IP addresses in DHCP.  I expanded the scope on Saturday to make more IP addresses available.  This problem should not reoccurr.

Database/Email Systems Unavailable 10/25

Saturday, October 25, Technology Services will be performing important updates on our database systems and storage area network. This will impact our database and email systems. In order to minimize service disruption, there will be two groups of outages:

  • 7 AM until 5 PM  – Cascade Web, Banner, Famis, Basis, and the CRM will be unavailable.
  • 7:30 AM until 9:00 AM – Email will be unavailable.

Network files and shares, and other network services will not be impacted.

Problems with CMS Publishing

The Content Management System stopped publishing content to the main web server at about 9:10am this morning. I believe that this problem has been resolved; however, new content will be unlikely to appear on the main web server until after 3:30pm.

If your changes haven’t shown up on the main web server by 4:00 pm, please publish the pages again. Thank you.

Outgoing email Outage – Thursday October 16th from 6:30-7:30am

From 6:30-7:30 am on October 16th upgrades to the Sophos Pure Message anti-spam and anti-virus servers will cause delays in outgoing messages (from @ups.edu accounts to Internet users).  Service is expected to be fully restored by 7:30am.


Outgoing mail was down from about 6:45-7:20am, mail is now flowing regularly.  Please contact the helpdesk if you see additional delays.

Sophos Anti-Spam Service Outage – Tuesday, October 14th from 5:30-7:30am

All Sophos pure message anti-spam and anti-virus email servers will be down for maintenance from 5:30-7:30am to upgrade the software to the latest version. This upgrade is required to improve spam blocking for UPS email users.

During the upgrade mail should continue to flow between the University and the Internet, but may be slightly delayed at times until the upgrade is complete at 7:30am. The spam quarantine will be unavailable during parts of this outage as well. If the quarantine is not available please check back later in the day.

Update — 10/14/2008 @ 6:54am

All upgrades are complete and working as expected.  Due to the upgrade process some email may have been delayed by a half hour or more, but should now be back to normal.

Spam Quarantine Outage – Friday, October 10th 6:30-8:30am

Starting at 6:30am on Friday, October 10th, the Sophos Spam Quarantine will be down for maintenance.  During this time users will not be able to access their spam quarantines, but mail will continue to be delivered as normal.  The Quarantine is expected to be available by 8:30am.

Update — 10/10/2008 @ 8:30am

Migration is taking longer than expected waiting on replication of small files.  We currently estimate that the Spam Quarantine will be up approximately 9:30am.

Update — 10/10/2008 @ 9:45am

Migration was completed at approximately 9:30am.

CRM application tier bounced at 10:20 am on 10/1/2008

CRM fulfillment experienced an error during the processing of an email (rollback segment problem). Restarting the email caused a runaway cpu-intensive session even though the email was subsequently cancelled.

Cause of runaway session: user created content using the wrong content type (admission content type instead of campus email), which relied on admission data being in the system.

Resolution: We bounced the application server and killed the runaway session. User recreated email with correct content type.