At about 2:15 PM today, the ARP entry for 192.124.98.129 (ezproxy.ups.edu) was incorrect on the network core router. This of course caused the Ezproxy server to go offline. The ARP cache was cleared, and the correct entry was reestablished. The cause for this has not been discovered yet. A duplicate IP address is suspected.
[RESOLVED] Ezproxy server down 2/7/2009
The Ezproxy server was unavailable starting at approximately 2:15 PM today. This caused much of the Library’s Research Gateway section of the website to be unavailable. Service was restored at approximately 3:30 PM.
[Completed] K-20 Network Installed 2/7/2009
As of 6 AM this morning, Puget Sound has been connected to the Washington K-20 Educational Network, and network traffic between the academic/administrative buildings and the Internet has been rerouted over this new network. Residential Internet traffic is routed over the pre-existing Internet connection.
[RESOLVED]Short Service Interruptions 2/7/2009
We experienced a wireless interuption from 5:45-7:15. Our servers were unavailable from the internet from 5:45-6:15am. All connectivity has been restored. All services are back to normal. If you experience any additional issues please call the Technology Services HelpDesk at 879-8585.
[RESOLVED] Mail to and from the Internet is not flowing 2/7/2009
Since 5:20 AM this morning most email has not been delivering to and from the Internet. We are addressing the problem right now, and hope to have this resolved within the hour.
This problem was due to a routing misconfiguration during this morning’s scheduled maintenance. Mail deliver was restored at 10:30 AM. Mail sent to or from the Internet will be delayed 1-2 hours while the mail servers process the backlog.
K-20 Network Installation
On Saturday morning, February 7, from 5:30-7:00 AM, staff will be working on the installation of the new K-20 network. The goal of this work is to implement the new network service. We expect intermittent service disruptions during this time.
[Resolved] Network Maintenance 2/6/2009
Technology Services will be performing network maintenance during the normal maintenance window of 5:30-6:00 AM. There will be a short disruption in the college’s Internet connection during this time.
Sophos Upgrade, SSL goes away on CSM/End User Web UI
Today we upgraded sophos puremessage on all servers, and in doing so we had several issues with the end user web ui. First it switched over to IPv6, to solve that:
1. Add this to /etc/modprobe.conf
# added 2/5/09 to disable ipv6 from starting -NJW
alias net-pf-10 off
alias ipv6 off
2. Update “NETWORKING_IPV6=no” in /etc/sysconfig/network
Then in /opt/pmx/etc/manager/httpd2/ssl Update default.conf to this:
# HTTPS Port — see old/ directory for previous/default config -NJW 2/5/2009
Listen *:28080
<VirtualHost *:28080>
SSLCertificateFile etc/manager/httpd2/sophos2007.pem
SSLEngine On
Include etc/manager/httpd2/rpc.conf
</VirtualHost>
Basically just changing to SSL for the port 28080.
Finally restart the server (required for IPv6 kernel?). If you see yourself needing to update the httpd configs, just restart it as the pmx user each time:
pmx-httpd restart
[Resolved] E-mail Delivery Delay Thursday 2/5/2009 5:30-7:30am – Part 2 of 2
On the morning of Thursday, February 5th from 5:30 to 7:30am, TS will be performing upgrades to improve the effectiveness of our spam filtering system. During the upgrade, delivery of email into and out of the college may be delayed. E-mail sent may not arrive until after 7:30am. E-mail sent within the University is not affected by this outage (i.e. e-mail from one @ups.edu account to another).
[Update: 2/5 at 9:30am] Mail delivery was restored by 7:10 am; however, the update for the end user quarantine page took longer than expected. All services were restored by 9:30am.
[Resolved] Hotmail, MSN and Live E-mail Delivery Restored
Hotmail, MSN and Live is accepting delivery of Puget Sound e-mail messages. TS staff will continue to work with Microsoft to identify any spam coming from our e-mail accounts. As long as there are no additional compromised e-mail accounts, e-mail flow will continue as normal.
[Resolved] External Internet Connection was Unavailable
We discovered that the external connection to Internet sites off campus is unavailable. TS is investigating and will update this message when more information is available.
[Update 2/2/09 @ 11:30 am] External Internet service has been restored.
[Resolved] Spam Quarantine Unavailable Tuesday 2/3/2009 5:30-7:30am – Part 1 of 2
On the morning of Tuesday, February 3rd, from 5:30-7:30am, TS will be performing upgrades to improve reliability of our spam filtering system. During the upgrade, some users will be unable to access their quarantined email messages in the Sophos Pure Message system. Should you experience problems accessing quarantined messages, please try again after 7:30am.
[Updated 2/3/2009 @ 7:45AM]
The quarantine is mostly available at this time, but a few users may find that certain messages are not available. We expect this to be resolved within 15-30 minutes.
K-20 Network Installation Delayed
Due to a flaw in the K-20 network’s routing, we were unable to complete the K-20 Internet connection upgrade today, Saturday, January 31. Internet connectivity for the academic and administrative buildings was interrupted for approximately 15 minutes this morning around 9:30 AM. We have restored internet connectivity to the administrative and academic buildings. We will continue to work with K-20 to find a resolution, and schedule the implementation during the coming week.
[Resolved] Cascade batch processing unavailable, 1/30/2009
Cascade’s batch processing system went down unexpectedly at 11:15 am. We are working to restore this service.
Batch processing service was restored at 11:51 am.
Washington K-20 Network Install Scheduled 1/31/09
On Saturday, January 31, at 9 AM routing will be adjusted to direct Internet-bound traffic for the academic and administrative buildings to the Washington K-20 network. Traffic for the residential buildings (residence halls, Greek Row houses, and on-campus houses) will continue to flow over the existing Internet circuit provided by Integra Telecom. From 9:00 -10:15 AM, Internet connection may be interrupted more than once for a few minutes at a time.