{PUGETSOUND SYSTEMS}: Patch Maintenance, Sun, Feb 9, 6 a.m.-6 p.m.

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

Puget Sound Systems and Applications: Planned Maintenance

Sunday, February 9 @ 6 a.m. – 6 p.m.

SYSTEMS IMPACTED: 

  • Campus file servers

  • myPugetSound.edu

  • PeopleSoft Human Resources

  • PeopleSoft Financials

  • PeopleSoft Campus Solutions

  • vDesk

  • Various Puget Sound systems and applications

  • Network connections to campus buildings (limited to 6 a.m. – 8 a.m.)

REASON FOR IMPACT:

  • To ensure a secure, reliable computing environment, TS will be completing routine patch maintenance on our Puget Sound production servers.

 

ADDITIONAL INFORMATION:

  • Technology Services schedules regular maintenance windows, generally on the second Sunday of every other month, to ensure a secure, reliable computing environment.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help: support.pugetsound.edu 

Email Support: servicedesk@pugetsound.edu

Subject: {PUGETSOUND NETWORK}: Planned Maintenance, Wed, Jan 29, 6:00 a.m. – 7:00 a.m. [COMPLETE]

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

Puget Sound Network: Planned Maintenance

Wednesday, January 29 | 6:00 a.m. – 7:00 a.m. [COMPLETE]

SYSTEMS IMPACTED: 

  • Puget Sound Network

REASON FOR MAINTENANCE:

  • Technology Services will be reconfiguring a failover protocol on the network.

 

WHAT YOU SHOULD KNOW:

  • During the maintenance window, the network may be intermittently unavailable.

To learn more about Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help: support.pugetsound.edu 

Email Support: servicedesk@pugetsound.edu

{PUGETSOUND NETWORK}: Planned Maintenance, Wed, Jan 29, 6:00 a.m. – 7:00 a.m.

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

Puget Sound Network: Planned Maintenance

Wednesday, January 29 | 6:00 a.m. – 7:00 a.m.

SYSTEMS IMPACTED: 

  • Puget Sound Network

REASON FOR MAINTENANCE:

  • Technology Services will be reconfiguring a failover protocol on the network.

 

WHAT YOU SHOULD KNOW:

  • During the maintenance window, the network may be intermittently unavailable.

To learn more about Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help: support.pugetsound.edu 

Email Support: servicedesk@pugetsound.edu

Campus Network Outage Summary – Friday, January 24

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has resolved the following incident. Details on the issue are available below:

Puget Sound Network: Campus Network Outage

Friday, January 24 | 3:13 p.m. – 10:30 p.m.

Outage Summary:
On Friday, January 24th, at 3:13 p.m., Technology Services observed a campus-wide network outage. Due to the scale of the outage and the campus community's inability to communicate via Zoom or email, our Emergency Response Group elected to issue an Everbridge alert to notify the campus. The team undertook several remedial actions to restore network connectivity, and a follow-up Everbridge alert was issued at approximately 5:15 p.m., indicating the outage had been resolved. Unfortunately, at around 5:50 p.m., the team observed a recurrence of the same network behavior seen during the initial outage. At this point, the campus network became unavailable again.

Following a thorough review of event logs and identifying a viable remediation method, our network engineers, with the support of Juniper, implemented a configuration change to the core and distribution layers of the network. This action stabilized and restored connectivity at approximately 9:30 p.m. We apologize for the disruption caused by this outage and appreciate your understanding as we worked to restore services.

Technical Details:
During the initial onset of the outage, our engineers reviewed network traffic patterns and identified what appeared to be a network loop causing the outage. A network loop, in its simplest form, occurs when data travels in a continuous circle without any mechanism to stop it. This can happen if a data port is connected to another port within the same network, creating an infinite data loop, or if a configuration change inadvertently causes traffic to route in a circular path. While protocols like Spanning Tree Protocol (STP) exist to prevent loops from disrupting an entire network, loops can sometimes originate in areas where STP is not applicable.

The team conducted a comprehensive review of all changes made in the two weeks prior to the incident to determine if any recent network modifications had caused the outage. We are confident that no changes performed by our team were responsible. Additionally, we found no evidence of a cyber event contributing to the outage.

Unfortunately, a series of factors led the team to a false positive regarding the root cause of the outage. During the event, our logging system was overwhelmed with excessive and unnecessary details, which obscured the underlying issue and made it difficult to diagnose the cause. The campus network's routing fabric uses Border Gateway Protocol (BGP) to determine the best paths for data to travel. BGP relies on a supplementary protocol, Bidirectional Forwarding Detection (BFD), to detect routing failures. Upon further analysis of the logs, we discovered an unknown issue causing BFD to continuously reset routing across the campus network, rendering it virtually impossible for devices to communicate.

After consulting with Juniper support, our team confirmed it was safe to disable BFD temporarily to restore network connectivity. Once this configuration change was made, network traffic stabilized immediately, and services were restored. Additionally, packet captures of the network have been obtained to further analyze the outage and identify the root cause of the BFD resets. Once a fix has been identified, we will implement the necessary adjustments and re-enable BFD during the next available routine change window.

Our teams continued to work through the weekend to address residual disruptions caused by the outage and ensure the stability of our data center operations.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help: support.pugetsound.edu 

Email Support: servicedesk@pugetsound.edu

{myPugetsound, PeopleSoft HR, PeopleSoft Financials} unplanned outage, Jan 22 – RESOLVED

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) is working to resolve the following incident. Details on the outage are available below:

myPugetSound and PeopleSoft Outage – Resolved

January 22 starting at 12:20 p.m. until 1:45 p.m.

SYSTEMS IMPACTED: 

  • myPugetSound, PeopleSoft HR, and PeopleSoft Financials. 

CURRENT STATUS: 

  • The TS team has resolved the issue affecting our PeopleSoft systems.

 

WHAT YOU SHOULD KNOW:

  • None at this time

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we work to resolve this issue as soon as possible. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help: support.pugetsound.edu 

Email Support: servicedesk@pugetsound.edu

{myPugetsound, PeopleSoft HR, PeopleSoft Financials} unplanned outage

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) is working to resolve the following incident. Details on the outage are available below:

myPugetSound and PeopleSoft Outage

January 22 starting at 12:20 p.m.

SYSTEMS IMPACTED: 

  • myPugetSound, PeopleSoft HR, and PeopleSoft Financials. 

CURRENT STATUS: 

  • The TS team is investigating an outage affecting our PeopleSoft systems and working to resolve the issue as quickly as possible. TS will update as new information is available. 

 

WHAT YOU SHOULD KNOW::

  • None at this time

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we work to resolve this issue as soon as possible. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help: support.pugetsound.edu 

Email Support: servicedesk@pugetsound.edu

{WHEELOCK NETWORK}: Planned Maintenance, Thurs, Jan 16, 6:00 a.m. – 7:30 a.m.

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

Wheelock Network: Planned Maintenance

Thursday, January 16 | 6:00 a.m. – 7:30 a.m.

SYSTEMS IMPACTED: 

  • Wheelock Network (Wi-Fi, Phones, Ethernet)

REASON FOR MAINTENANCE:

  • Technology Services will be replacing a faulty network switch.

 

WHAT YOU SHOULD KNOW:

  • During the maintenance window, some wired and Wi-Fi network connections in Wheelock will go offline.

To learn more about Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help: support.pugetsound.edu 

Email Support: servicedesk@pugetsound.edu

{CAMPUS NETWORK}: Planned Maintenance, Mon, Dec 30, 9:00 a.m. – 3:00 p.m.

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

Campus Network: Planned Maintenance

Monday, December 30 | 9:00 a.m. – 3:00 p.m.

SYSTEMS IMPACTED: 

  • Various Puget Sound systems and applications (9 a.m. – 11 a.m.)

  • Wired and wireless network in all campus buildings 

  • Physical phones in all campus buildings

  • Blue emergency towers

REASON FOR MAINTENANCE:

  • Firmware upgrades will be installed on network switches to ensure a secure, reliable computing environment 

  • Network switches will be rebooted as part of the firmware upgrade process

 

WHAT YOU SHOULD KNOW:

  • During the full maintenance window, short network outages at different times will occur on a rolling basis building by building throughout campus 

  • Between 9 a.m. – 11 a.m., various Puget Sound systems and applications will be intermittently unavailable from off-campus networks

  • Card swipe access to the buildings have a local cache and will continue to function

  • Security Services will remain reachable at (253) 879-3311

To learn more about Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. For urgent issues during the holiday closure from Dec. 23 – Jan. 1, please contact Security Services at (253) 879-3311.

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help: support.pugetsound.edu 

Email Support: servicedesk@pugetsound.edu

{PUGETSOUND SYSTEMS}: Patch Maintenance, Sun, Dec 15, 6 a.m.-6 p.m. [COMPLETE]

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

Puget Sound Systems and Applications: Planned Maintenance

Sunday, December 15 @ 6 a.m. – 6 p.m. [COMPLETE]

SYSTEMS IMPACTED: 

  • Campus file servers

  • myPugetSound.edu

  • PeopleSoft Human Resources

  • PeopleSoft Financials

  • PeopleSoft Campus Solutions

  • vDesk

  • Various Puget Sound systems and applications

REASON FOR IMPACT:

  • To ensure a secure, reliable computing environment, TS will be completing routine patch maintenance on our Puget Sound production servers.

 

ADDITIONAL INFORMATION:

  • The university network, internet access, and wireless will NOT be affected.

  • Technology Services schedules regular maintenance windows, generally on the second Sunday of every other month, to ensure a secure, reliable computing environment.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help: support.pugetsound.edu 

Email Support: servicedesk@pugetsound.edu


Network Outage: 12/12 Approximately 3:15PM – 4:15PM

This logo identifies the Technology Services department within the University of Puget Sound


Dear members of the Puget Sound community,


On Thursday, December 12 at approximately 3:15 p.m. Technology Services (TS) was made aware of a network outage affecting both Collins Memorial Library and the Facilities Services Complex. While we are still working to identify the root cause of the issue, we believe the network instability may be related to the staging of software for our network infrastructure ahead of the holiday break.


During this time, users would have experienced an inability to access any of the following services:

  • Internet Access
  • Wired and Wireless Network Connectivity (WiFi)
  • Desk Phones Restarting or Dropping Calls

If you experienced any of the outages above, please contact the Service Desk at 253.879.8585 so we can identify those who were affected.

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help:  https://www.pugetsound.edu/osd (Logon Required)

Email Support: servicedesk@pugetsound.edu

Technology Services Early Closure Notice – Today Dec 12 @ Noon

This logo identifies the Technology Services department within the University of Puget Sound


Dear members of the Puget Sound community,


On Today, December 12 at 12 p.m. Technology Services (TS) and Print and Copy Services (PCS), will be closing early for an on-campus, team holiday gathering.


During this time, limited services will be available from the following areas:

  • Service Desk Support
  • Print and Copy Services (PCS)
  • Media Services

In case of a technical emergency, please contact Security Services at 253.879.3311 and they will reach the appropriate resource within TS. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help:  https://www.pugetsound.edu/osd (Logon Required)

Email Support: servicedesk@pugetsound.edu

{PUGETSOUND SYSTEMS}: Patch Maintenance, Sun, Dec 15, 6 a.m.-6 p.m.

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

Puget Sound Systems and Applications: Planned Maintenance

Sunday, December 15 @ 6 a.m. – 6 p.m.

SYSTEMS IMPACTED: 

  • Campus file servers

  • myPugetSound.edu

  • PeopleSoft Human Resources

  • PeopleSoft Financials

  • PeopleSoft Campus Solutions

  • vDesk

  • Various Puget Sound systems and applications

REASON FOR IMPACT:

  • To ensure a secure, reliable computing environment, TS will be completing routine patch maintenance on our Puget Sound production servers.

 

ADDITIONAL INFORMATION:

  • The university network, internet access, and wireless will NOT be affected.

  • Technology Services schedules regular maintenance windows, generally on the second Sunday of every other month, to ensure a secure, reliable computing environment.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help: support.pugetsound.edu 

Email Support: servicedesk@pugetsound.edu

{PUGET SOUND NETWORK}: Planned Maintenance, Mon, Nov 25, 6:00 a.m. – 7:00 a.m.

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

Puget Sound Network: Planned Maintenance

Monday, November 25 | 6:00 a.m. – 7:00 a.m.

SYSTEMS IMPACTED: 

  • Anderson-Langdon Network

  • Greek Row Network

  • Harned-Thompson Network

  • Harrington Network

  • Jones Network

  • Music Network

  • North Quad Emergency Tower

  • Oppenheimer Network

  • Schiff Network

  • Smith Network

  • Todd-Phibbs Network

  • Union Ave Network

REASON FOR MAINTENANCE:

  • Technology Services will be restarting network equipment in Harrington, Jones, and Music to correct occasional outages.

 

WHAT YOU SHOULD KNOW:

  • During the maintenance window, all wired and Wi-Fi network connections listed in systems impacted will go offline for approximately 15 minutes.

  • Card swipe access to the buildings have a local cache and will continue to function.

  • The North Quad Emergency Tower will go offline during this maintenance.

  • Security Services will remain reachable at (253) 879-3311.

To learn more about Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help: support.pugetsound.edu 

Email Support: servicedesk@pugetsound.edu

{HARNED, THOMPSON NETWORK}: Planned Maintenance, Thu, Nov 21, 6:00 a.m. – 7:00 a.m.

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

Harned, Thompson Network: Planned Maintenance

Thursday, November 21 | 6:00 a.m. – 7:00 a.m.

SYSTEMS IMPACTED: 

  • Harned Network

  • Thompson Network

REASON FOR MAINTENANCE:

  • Technology Services will be restarting all network equipment in Harned and Thompson to correct occasional outages.

 

WHAT YOU SHOULD KNOW:

  • During the maintenance window, all wired and Wi-Fi network connections in Harned and Thompson will go offline for approximately 15 minutes.

To learn more about Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help: support.pugetsound.edu 

Email Support: servicedesk@pugetsound.edu

{PUGETSOUND NETWORK}: Emergency Firewall Maintenance, Wednesday, November 20, 5 a.m. – 6 a.m.

This logo identifies the Technology Services department within the University of Puget Sound

Dear members of the Puget Sound community,

Technology Services (TS) has planned the following maintenance activities. Details on the maintenance being performed are available below:

Network Security Devices (Firewalls): Emergency Maintenance

Wednesday, November 20 | 5 a.m. – 6 a.m.

SYSTEMS IMPACTED: 

  • Inbound and outbound internet access 

  • GlobalProtect VPN 

  • Blue emergency towers 

  • Systems requiring Internet connectivity on the Puget Sound campus, such as Zoom phones

REASON FOR MAINTENANCE:

  • Technology Services will be installing software updates on our network security devices (firewalls) to address a critical severity vulnerability.

 

WHAT YOU SHOULD KNOW:

  • During the maintenance window, impacted systems will be intermittently unavailable for short durations of time (~1-5 minutes).

  • Security Services will remain reachable at (253) 879-3311 during this time.

For more information on Technology Services' Incident Communication Plan or to view a list of our Critical Systems and definitions, please visit our Service Announcements page on the Puget Sound website.

Thank you for your patience as we conduct this important work. If you have any questions or concerns, please don't hesitate to contact the TS Service Desk. 

TS Service Desk

Walk-In Support: Tech Center in Collins Library

Phone Support: (253) 879-8585

Online Help: support.pugetsound.edu 

Email Support: servicedesk@pugetsound.edu