Banner, Budget Administration, and Your Employment information will be unavailable from 5:00 p.m. on Friday, February 20, until 7:00 a.m. on Monday, February 23 due to the Banner migration to new hardware.
[Update 2/22/09 @ 12:00 noon] All services are back on-line. The upgrade went as planned and ahead of schedule.
At 11:30 am, one of the servers began experiencing performance problems. Banner, CRM, FAMIS, and Cascade batch processing are impacted by this.
Update: 2:05 pm, the performance problem on Camano has been resolved. CRM is still down.
Update: 2:45 pm, service has been restored on CRM
On February 17th from 5:30-6AM batteries in our EMC CX400 Disk Array will be replaced. This is a fully redundant system and no outage is expected. However, as with any change, there is always a risk of outage. If a problem arises with the change, the following services may be impacted:
email
CRM
Database Services (FAMIS, Banner, Millennium, etc.)
Please note that Cascade will not be affected.
Testing
At about 2:15 PM today, the ARP entry for 192.124.98.129 (ezproxy.ups.edu) was incorrect on the network core router. This of course caused the Ezproxy server to go offline. The ARP cache was cleared, and the correct entry was reestablished. The cause for this has not been discovered yet. A duplicate IP address is suspected.
The Ezproxy server was unavailable starting at approximately 2:15 PM today. This caused much of the Library’s Research Gateway section of the website to be unavailable. Service was restored at approximately 3:30 PM.
As of 6 AM this morning, Puget Sound has been connected to the Washington K-20 Educational Network, and network traffic between the academic/administrative buildings and the Internet has been rerouted over this new network. Residential Internet traffic is routed over the pre-existing Internet connection.
We experienced a wireless interuption from 5:45-7:15. Our servers were unavailable from the internet from 5:45-6:15am. All connectivity has been restored. All services are back to normal. If you experience any additional issues please call the Technology Services HelpDesk at 879-8585.
Since 5:20 AM this morning most email has not been delivering to and from the Internet. We are addressing the problem right now, and hope to have this resolved within the hour.
This problem was due to a routing misconfiguration during this morning’s scheduled maintenance. Mail deliver was restored at 10:30 AM. Mail sent to or from the Internet will be delayed 1-2 hours while the mail servers process the backlog.
On Saturday morning, February 7, from 5:30-7:00 AM, staff will be working on the installation of the new K-20 network. The goal of this work is to implement the new network service. We expect intermittent service disruptions during this time.
Technology Services will be performing network maintenance during the normal maintenance window of 5:30-6:00 AM. There will be a short disruption in the college’s Internet connection during this time.
Today we upgraded sophos puremessage on all servers, and in doing so we had several issues with the end user web ui. First it switched over to IPv6, to solve that:
1. Add this to /etc/modprobe.conf
# added 2/5/09 to disable ipv6 from starting -NJW
alias net-pf-10 off
alias ipv6 off
2. Update “NETWORKING_IPV6=no” in /etc/sysconfig/network
Then in /opt/pmx/etc/manager/httpd2/ssl Update default.conf to this:
# HTTPS Port — see old/ directory for previous/default config -NJW 2/5/2009
Listen *:28080
<VirtualHost *:28080>
SSLCertificateFile etc/manager/httpd2/sophos2007.pem
SSLEngine On
Include etc/manager/httpd2/rpc.conf
</VirtualHost>
Basically just changing to SSL for the port 28080.
Finally restart the server (required for IPv6 kernel?). If you see yourself needing to update the httpd configs, just restart it as the pmx user each time:
pmx-httpd restart
On the morning of Thursday, February 5th from 5:30 to 7:30am, TS will be performing upgrades to improve the effectiveness of our spam filtering system. During the upgrade, delivery of email into and out of the college may be delayed. E-mail sent may not arrive until after 7:30am. E-mail sent within the University is not affected by this outage (i.e. e-mail from one @ups.edu account to another).
[Update: 2/5 at 9:30am] Mail delivery was restored by 7:10 am; however, the update for the end user quarantine page took longer than expected. All services were restored by 9:30am.
Hotmail, MSN and Live is accepting delivery of Puget Sound e-mail messages. TS staff will continue to work with Microsoft to identify any spam coming from our e-mail accounts. As long as there are no additional compromised e-mail accounts, e-mail flow will continue as normal.
We discovered that the external connection to Internet sites off campus is unavailable. TS is investigating and will update this message when more information is available.
[Update 2/2/09 @ 11:30 am] External Internet service has been restored.