As of noon today all mail list services are back online. Please contact TechHelp at x8585 if you continue to experience problems with lists.
Early this morning our mail list server had a problem and is not working properly. We are working on resolving the issue, but it is not likely to be resolved before 5pm today. We will update this post as we have more information.
The ResNet internet connection was down from 12:00 midnight to 3:00 AM this morning. Service has been restored.
Today between 5pm and 6pm wiki.ups.edu was moved over to userweb.pugetsound.edu. No problems were encountered, but please let us know if you encounter issues.
Nick Webb x3185
Both Crm11ip instance and Crm App had to be restarted due to the exceeded processes on the databasee. The problem was caused by the returning mails sent last week.
Installed Oracle Critical Patch, CpuOct2009, on Cascade, Banner, Famis, Millennium, and Rman databases.
Orcas needs a reboot to properly configure VMWare tools, the misconfiguration is causing performance issues on Orcas. All services will be available by 6am.
BJP was migrated off of Camano, onto Orcas
We discovered that the CRM email bounce-backs were not being processed, resulting in a very full mailbox. Nick renamed the inbox, essentially cleaning out the mailbox, thinking that maybe the box was too full to process the messages properly. This appears to have fixed the problem. We will look into whether these are being cleaned out properly when the bounceback is processed (or, being marked for deletion but then never deleted).
We also discovered that messages were being routed to the Junk email mailbox. CRM only processes bounce-backs found in the Inbox. We suspect that our use of Outlook to access the bounces email account has caused this to occur. We will look into using a tool other than Outlook for accessing this IMAP account.
All production databases were patched successfully. If you experience any issues with Puget Sound systems, please contact TechHelp at (253) 879-8585.
The next production maintenance window is Sunday, November 29th, from 8:00 AM to noon. From 8:00 AM to noon, all database services may be intermittently or entirely unavailable, including: Cascade, FAMIS, Millennium, and Banner.
Updated 10/25/2009 @ noon — All production servers were patched successfully. If you experience any issues with Puget Sound systems, please contact TechHelp at (253) 879-8585.
The next production maintenance window is Sunday, October 25th, from 8:00 AM to noon.
From 8:00 AM to Noon, all services may be intermittently or entirely unavailable, including:
The following systems were patched today between 8am and noon:
This month, on 10/15/2009 from 7am to 9pm we performed our monthly patch maintenance for PreProduction servers. We patched the following systems:
As of 9:00pm 10/13/09 all users have complete access to their email once again. Some messages into and out of these users accounts may be delayed as the system catches up.
Update as of 4:45PM 10/13/09
The outage of email for users EJ-LB continues as we run a database recovery process on the affected volume/users. The process could take up to 18 hours, completing by mid-morning Wednesday (10am).
This post will be updated as more information is available.
Exchange Email is unavailable for users with usernames starting with EJ-LB. Currently there is not a time estimate of when email for these users will be available again, but this post will be updated when the issue is corrected.
10/10/2009 The Aruba wirless controller was upgraded and additional new licensing applied starting at 4:30am. The controller was rebooted at 5:20am. The process is complete at 5:30am. Monitoring until 6:00am shows all access points are functioning properly.
Update at 11:06 AM
All users now have access to their email and Calendars. Please contact TechHelp if you continue to see problems.
Updated at 9:29 AM
Email is still unavailable for Students and Technology Services users starting with the letters L through S and Technology Services continues to work the problem. We now expect the problem to be resolved by 12 noon, while doing everything we can to resolve the issue more quickly. Watch this post for more information as it becomes available.
At this time some users are unable to access their email and calendars via Outlook, Webmail, and by other means. Technology Services has identified the problem and expects resolution by 9:30am.