UTHPC - GPU nodes on our Rocket cluster are currently unavailable – Incident details

GPU nodes on our Rocket cluster are currently unavailable

ResolvedMajor outage1 hour 14 minutes

Affected

Cluster

Major outage from 12:09 PM to 12:44 PM, Partial outage from 12:09 PM to 1:24 PM

rocket.hpc.ut.ee

Major outage from 12:09 PM to 12:44 PM, Partial outage from 12:44 PM to 1:24 PM

ondemand.hpc.ut.ee

Partial outage from 12:09 PM to 1:24 PM

Services

Partial outage from 12:09 PM to 1:24 PM

galaxy.hpc.ut.ee

Partial outage from 12:09 PM to 1:24 PM

  • ResolvedUTC

    The emergency maintenance has been completed, and the GPU nodes on the Rocket cluster are available again.

    Affected jobs have been requeued and will run again as resources become available.

    Thank you for your patience and understanding.

  • IdentifiedUTC

    GPU nodes on our Rocket cluster are currently unavailable due to a recently disclosed vulnerability. All ongoing jobs on these nodes will be stopped as the nodes undergo emergency maintenance. Once GPU node availability is restored, the affected jobs will be returned to the job queue.

    We apologize for any inconvenience and will inform you when the nodes are available again.

  • InvestigatingUTC

    GPU nodes on our Rocket cluster are currently unavailable due to a recently disclosed vulnerability. All ongoing jobs on them will be killed as the nodes will undergo emergency maintenance. We apologize for any inconvenience and will inform you when the nodes are available for submission.