{PUGETSOUND NETWORK}: Planned Routing Changes, Tuesday, Feb 20, 5:00 a.m. – 6:00 a.m. [COMPLETE]

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

SYSTEMS IMPACTED: 

  • None

DATE OF IMPACT: 

  • Tuesday, February 20 | 5 a.m. –  6 a.m. [COMPLETE]

REASON FOR MAINTENANCE:

  • To ensure a secure, reliable computing environment, TS will be making routing changes to our internal network.

 

WHAT YOU SHOULD KNOW:

  • No network outages or downtime are expected from the routing changes being made.

  • Security Services will remain reachable at 253-879-3311 during this time.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help:  https://www.pugetsound.edu/osd (Logon Required)

Email Support: servicedesk@pugetsound.edu

{PUGETSOUND NETWORK}: Planned Routing Changes, Tuesday, Feb 20, 5:00 a.m. – 6:00 a.m.

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

SYSTEMS IMPACTED: 

  • None

DATE OF IMPACT: 

  • Tuesday, February 20 | 5 a.m. –  6 a.m.

REASON FOR MAINTENANCE:

  • To ensure a secure, reliable computing environment, TS will be making routing changes to our internal network.

 

WHAT YOU SHOULD KNOW:

  • No network outages or downtime are expected from the routing changes being made.

  • Security Services will remain reachable at 253-879-3311 during this time.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help:  https://www.pugetsound.edu/osd (Logon Required)

Email Support: servicedesk@pugetsound.edu

{PUGETSOUND NETWORK}: Planned Firewall Maintenance, Thursday, Feb 15, 5:00 a.m. – 6:00 a.m. – [COMPLETE]


This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

SYSTEMS IMPACTED: 

  • Inbound and outbound internet access 

  • GlobalProtect VPN 

  • Blue emergency towers

  • Systems requiring Internet connectivity on the Puget Sound campus, such as Zoom phones

DATE OF IMPACT: 

  • Thursday, February 15 | 5 a.m. –  6 a.m. [COMPLETE]

REASON FOR MAINTENANCE:

  • To ensure a secure, reliable computing environment, TS will be completing routine firmware updates on our network security appliances (firewalls).

 

WHAT YOU SHOULD KNOW:

  • Security Services will remain reachable at 253-879-3311 during this time.

  • Impacted systems will be intermittently unavailable for short durations of time (~1-5 minutes) during the maintenance window.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help:  https://www.pugetsound.edu/osd (Logon Required)

Email Support: servicedesk@pugetsound.edu

{PUGETSOUND NETWORK}: Planned Firewall Maintenance, Thursday, Feb 15, 5:00 a.m. – 6:00 a.m.

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

SYSTEMS IMPACTED: 

  • Inbound and outbound internet access 

  • GlobalProtect VPN 

  • Blue emergency towers

  • Systems requiring Internet connectivity on the Puget Sound campus, such as Zoom phones

DATE OF IMPACT: 

  • Thursday, February 15 | 5 a.m. –  6 a.m.

REASON FOR MAINTENANCE:

  • To ensure a secure, reliable computing environment, TS will be completing routine firmware updates on our network security appliances (firewalls).

 

WHAT YOU SHOULD KNOW:

  • Security Services will remain reachable at 253-879-3311 during this time.

  • Impacted systems will be intermittently unavailable for short durations of time (~1-5 minutes) during the maintenance window.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help:  https://www.pugetsound.edu/osd (Logon Required)

Email Support: servicedesk@pugetsound.edu

{FIREWALL, NETWORK, WIFI, PHONES} Planned Campus Maintenance – Wed 2/14 5:30 a.m. – 7 a.m. [COMPLETE]

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound campus community,

Technology Services (TS) will perform critical maintenance to our campus core network as we continue to transition the university to a more modern network. During this time, network connectivity on and off campus will be intermittently unavailable as we make routing changes to our firewall and internal network.

Firewall, Network, WiFi, Phones, Internet: On Campus Network Maintenance

Wednesday, February 14 | 5:30 a.m. –  7 a.m. [COMPLETE]

WHAT YOU SHOULD KNOW: 

  • Internet access will be intermittently unavailable as routing changes are implemented.

  • Systems requiring Internet connectivity on the Puget Sound campus, such as phones, will be intermittently unavailable.

  • Blue emergency towers will be intermittently unavailable during the maintenance window.

  • Outages are expected to last no more than a few minutes.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help:  https://www.pugetsound.edu/osd (Logon Required)

Email Support: servicedesk@pugetsound.edu

CORRECTION: {FIREWALL, NETWORK, WIFI, PHONES} Planned Campus Maintenance – Wed 2/14 5:30 a.m. – 7 a.m.

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound campus community,

Technology Services (TS) will perform critical maintenance to our campus core network as we continue to transition the university to a more modern network. During this time, network connectivity on and off campus will be intermittently unavailable as we make routing changes to our firewall and internal network.

Firewall, Network, WiFi, Phones, Internet: On Campus Network Maintenance

Wednesday, February 14 | 5:30 a.m. –  7 a.m.

WHAT YOU SHOULD KNOW: 

  • Internet access will be intermittently unavailable as routing changes are implemented.

  • Systems requiring Internet connectivity on the Puget Sound campus, such as phones, will be intermittently unavailable.

  • Blue emergency towers will be intermittently unavailable during the maintenance window.

  • Outages are expected to last no more than a few minutes.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help:  https://www.pugetsound.edu/osd (Logon Required)

Email Support: servicedesk@pugetsound.edu

{FIREWALL, NETWORK, WIFI, PHONES} Planned Campus Maintenance – Wed 2/14 5:30 a.m. – 7 a.m.

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound campus community,

Technology Services (TS) will perform critical maintenance to our campus core network as we continue to transition the university to a more modern network. During this time, network connectivity on and off campus will be intermittently unavailable as we make routing changes to our firewall and internal network.

Firewall, Network, WiFi, Phones, Internet: On Campus Network Maintenance

Tuesday, February 14 | 5:30 a.m. –  7 a.m.

WHAT YOU SHOULD KNOW: 

  • Internet access will be intermittently unavailable as routing changes are implemented.

  • Systems requiring Internet connectivity on the Puget Sound campus, such as phones, will be intermittently unavailable.

  • Blue emergency towers will be intermittently unavailable during the maintenance window.

  • Outages are expected to last no more than a few minutes.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help:  https://www.pugetsound.edu/osd (Logon Required)

Email Support: servicedesk@pugetsound.edu

{THOMPSON NETWORK} Battery Backup Replacement – Tuesday 2/13/24 7:15 a.m. – 7:45 a.m. [COMPLETE]

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

Planned Maintenance: Thompson Battery Backup Replacement

Tuesday, February 13 | 7:15 a.m. – 7:45 a.m. [COMPLETE]

REASON FOR MAINTENANCE: 

  • Technology Services will replace the battery backup system that provides power to networking equipment that services the first floor of Thompson.

 

WHAT YOU SHOULD KNOW:

  • Network, WiFi, and phone connectivity will be temporarily unavailable on the first floor of Thompson during the maintenance window.

  • Actual downtime is expected to last no longer than 15 minutes.

  • The remaining floors of Thompson will not be affected by the maintenance.

  • Oppenheimer Cafe will not be affected by the maintenance.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help:  https://www.pugetsound.edu/osd (Logon Required)

Email Support: servicedesk@pugetsound.edu

{THOMPSON NETWORK} Battery Backup Replacement – Tuesday 2/13/24 7:15 a.m. – 7:45 a.m.

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

Planned Maintenance: Thompson Battery Backup Replacement

Tuesday, February 13 | 7:15 a.m. – 7:45 a.m.

REASON FOR MAINTENANCE: 

  • Technology Services will replace the battery backup system that provides power to networking equipment that services the first floor of Thompson.

 

WHAT YOU SHOULD KNOW:

  • Network, WiFi, and phone connectivity will be temporarily unavailable on the first floor of Thompson during the maintenance window.

  • Actual downtime is expected to last no longer than 15 minutes.

  • The remaining floors of Thompson will not be affected by the maintenance.

  • Oppenheimer Cafe will not be affected by the maintenance.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help:  https://www.pugetsound.edu/osd (Logon Required)

Email Support: servicedesk@pugetsound.edu

{PUGETSOUND SYSTEMS}: Patch Maintenance, Sun Feb 11, 6 a.m.-6 p.m. [COMPLETE]

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

Puget Sound Systems and Applications: Planned Maintenance

Sunday, February 11 @ 6 a.m. – 6 p.m. [COMPLETE]

SYSTEMS IMPACTED: 

  • Campus file servers

  • myPugetSound.edu

  • PeopleSoft Human Resources

  • PeopleSoft Financials

  • PeopleSoft Campus Solutions

  • vDesk

  • Various Puget Sound systems and applications

REASON FOR IMPACT:

  • To ensure a secure, reliable computing environment, TS will be completing routine patch maintenance on our Puget Sound production servers.

 

ADDITIONAL INFORMATION:

  • The university network, internet access, and wireless will NOT be affected.

  • Technology Services schedules regular maintenance windows, generally on the second Sunday of every other month, to ensure a secure, reliable computing environment.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help:  https://www.pugetsound.edu/osd (Logon Required)

Email Support: servicedesk@pugetsound.edu

{FIREWALL, NETWORK, WIFI, PHONES} Planned Campus Maintenance – Thu 2/8 5 a.m. – 7 a.m. {RESCHEDULED – DATE PENDING}

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound campus community,

Technology Services (TS) will perform critical maintenance to our campus core network as we continue to transition the university to a more modern network. During this time, network connectivity on and off campus will be intermittently unavailable as we make routing changes to our firewall and internal network.

Firewall, Network, WiFi, Phones, Internet: On Campus Network Maintenance

Thursday, February 8 | 5 a.m. –  7 a.m. {RESCHEDULED – DATE PENDING}

WHAT YOU SHOULD KNOW: 

  • Internet access will be intermittently unavailable as routing changes are implemented.

  • Systems requiring Internet connectivity on the Puget Sound campus, such as phones, will be intermittently unavailable.

  • Blue emergency towers will be intermittently unavailable during the maintenance window.

  • Outages are expected to last no more than a few minutes.

  • Due to unforeseen issues this will be rescheduled.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help:  https://www.pugetsound.edu/osd (Logon Required)

Email Support: servicedesk@pugetsound.edu

{PUGETSOUND SYSTEMS}: Patch Maintenance, Sun Feb 11, 6 a.m.-6 p.m.

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

Puget Sound Systems and Applications: Planned Maintenance

Sunday, February 11 @ 6 a.m. – 6 p.m.

SYSTEMS IMPACTED: 

  • Campus file servers

  • myPugetSound.edu

  • PeopleSoft Human Resources

  • PeopleSoft Financials

  • PeopleSoft Campus Solutions

  • vDesk

  • Various Puget Sound systems and applications

REASON FOR IMPACT:

  • To ensure a secure, reliable computing environment, TS will be completing routine patch maintenance on our Puget Sound production servers.

 

ADDITIONAL INFORMATION:

  • The university network, internet access, and wireless will NOT be affected.

  • Technology Services schedules regular maintenance windows, generally on the second Sunday of every other month, to ensure a secure, reliable computing environment.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help:  https://www.pugetsound.edu/osd (Logon Required)

Email Support: servicedesk@pugetsound.edu

{FIREWALL, NETWORK, WIFI, PHONES} Planned Campus Maintenance – Thu 2/8 5 a.m. – 7 a.m.

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound campus community,

Technology Services (TS) will perform critical maintenance to our campus core network as we continue to transition the university to a more modern network. During this time, network connectivity on and off campus will be intermittently unavailable as we make routing changes to our firewall and internal network.

Firewall, Network, WiFi, Phones, Internet: On Campus Network Maintenance

Thursday, February 8 | 5 a.m. –  7 a.m.

WHAT YOU SHOULD KNOW: 

  • Internet access will be intermittently unavailable as routing changes are implemented.

  • Systems requiring Internet connectivity on the Puget Sound campus, such as phones, will be intermittently unavailable.

  • Blue emergency towers will be intermittently unavailable during the maintenance window.

  • Outages are expected to last no more than a few minutes.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 


TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help:  https://www.pugetsound.edu/osd (Logon Required)

Email Support: servicedesk@pugetsound.edu

Follow up to Network Outage on January 25, 2024 – Root Cause Analysis

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,


Technology Services has prepared a Root Cause Analysis for the network incident that occurred on January 25th. The summary below provides additional technical details about the event as well as the course of action taken to restore services.


ISSUE AND IMPACT:

On January 25th at 2:30 p.m. the campus network began experiencing a level of instability that prevented users from accessing systems and services connected to the internet. During the duration of the network outage, services like Zoom Phone, Canvas, Google Workspace, and Okta were unreachable from within the campus network.


ROOT CAUSE:

After careful analysis of our event logs, we identified that the initial trigger of the event was caused by a change pushed by our Juniper Mist controller to a series of network devices. This change triggered a network routing convergence across a series of networks which were legacy artifacts of a security policy implementation that was never fully completed by Technology Services. Additionally, these changes triggered a series of events on our Palo Alto Firewalls ultimately causing the devices to no longer accept routing updates from the Juniper Core. At this point, both the Juniper Core and the Palo Alto firewall units were no longer able to successfully pass internet-bound traffic.


SOLUTION:

Technology Services reviewed all available logging information with our engineering team and determined that the legacy networks highlighted above as well as the associated firewall configuration should be removed from our network. Also, after careful review of the routing protocol used between the Juniper Core and the Palo Alto firewalls, it was decided that we should utilize a static routing protocol between the firewalls and the campus core network. The above stated solution has been scheduled for implementation on Thursday, February 8th at 5:00 a.m. A separate communication will be shared through the Service Announcements page detailing the events that will occur during the outage window.


For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help:  https://www.pugetsound.edu/osd (Logon Required)

Email Support: servicedesk@pugetsound.edu

Network Connectivity Issues

The Puget Sound Network was down starting around 2:50 p.m. and has been restored as of 7:11 p.m. Thank you for your patience as we worked through this issue. Our Infrastructure and Operations team identified a routing issue between our data centers and the internet causing severe network instability. The outage we experienced this afternoon ties back to a legacy network design that our teams are actively working to remediate.


Technology Services will be providing a Root Cause Analysis (RCA) for this incident in the next few days which will be available in our Service Announcements page.


For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.


Thank you again for your patience. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk.