UTHPC - Newly Disclosed Linux kernel Security Incident – Temporary Service Interruptions – Incident details

Experiencing partial outage

Newly Disclosed Linux kernel Security Incident – Temporary Service Interruptions

Identified
Major outage
Started about 16 hours ago

Affected

rocket.hpc.ut.ee

Major outage from 7:24 AM to 7:24 AM, Under maintenance from 7:24 AM to 11:13 AM, Major outage from 11:13 AM to 12:00 AM

UT HPC webservices

Degraded performance from 7:24 AM to 7:24 AM, Major outage from 7:24 AM to 11:13 AM, Operational from 11:13 AM to 12:00 AM

hpc.ut.ee

Degraded performance from 7:24 AM to 7:24 AM, Major outage from 7:24 AM to 11:13 AM, Operational from 11:13 AM to 12:00 AM

docs.hpc.ut.ee

Degraded performance from 7:24 AM to 7:24 AM, Major outage from 7:24 AM to 11:13 AM, Operational from 11:13 AM to 12:00 AM

support.hpc.ut.ee

Degraded performance from 7:24 AM to 7:24 AM, Major outage from 7:24 AM to 11:13 AM, Operational from 11:13 AM to 12:00 AM

registry.hpc.ut.ee

Degraded performance from 7:24 AM to 7:24 AM, Major outage from 7:24 AM to 11:13 AM, Operational from 11:13 AM to 12:00 AM

Updates
  • Update
    UTC
    Update

    We have made progress in addressing the recently disclosed Linux kernel security vulnerability.

    The following services have now been restored and are available:

    • Cloud Service

    • Web Server (Website Hosting Service)

    Mitigation work is still ongoing for the following services:

    • Rocket Cluster and related services such as Galaxy, Open OnDemand

    • SAPU

    These services will remain unavailable while we complete the required patching and validation to ensure they can be restored safely.

    We will continue to provide updates as additional services become available.

  • Identified
    UTC
    Identified
    We are continuing to work on a fix for this incident.
  • Investigating
    UTC
    Investigating

    We are currently responding to a newly disclosed Linux kernel security vulnerability affecting the XFS filesystem. As a precaution, we are implementing mitigation measures across affected infrastructure.

    Affected services:

    • Web Servers

    • Rocket Cluster, along with OpenOndemand

    • SAPU

    • Cloud

    • Galaxy

    To reduce risk while mitigation work is underway, Cluster access has been temporarily disabled. You may experience temporary service interruptions across the affected services during this maintenance.

    Our teams are actively applying mitigations and validating service integrity. We will provide further updates as work progresses and restore normal access as soon as it is safe to do so.

    Thank you for your patience and understanding.