MGHPCC Shutdown¶
Before getting started¶
- Double check with Norhteastern and BU if they are planning any network upgrades around the shutdown period.
Turn off Kumo¶
- Turn off the VMs on dell blades. We have Ironic controllers and some ceph nodes.
- Turn off all dell blades from the chassis management controller (easy).
- Next step, turn off Kumo Storage (it’s running FreeNAS)
Turn off Kaizen¶
- Power off all elastic hosts (send it ipmitool chassis power soft)
- Shutdown HIL, BMIm and vms on the kznbmihost1,2.
- Power off all the openstack VMs. This includes k-openshift.
- Then power off the computes, and then the controllers.
For shutting down OpenStack (and bringing it back up), see this Red Hat knowledge base article: https://access.redhat.com/solutions/1977013
A handy command line for stopping all running Nova instances:
openstack server list --status active --all-projects -f value -c ID |
xargs -P10 -n1 openstack server stop
Turn off the power9 cluster.¶
- power it off,
Turn off CNV and OCP cluster¶
- Take an
etcd
backup - Shut down the the worker nodes
- Shut down the controller nodes
For more information on shutting down and restarting OpenShift, see:
- https://docs.openshift.com/container-platform/4.5/backup_and_restore/graceful-cluster-shutdown.html
- https://docs.openshift.com/container-platform/4.5/backup_and_restore/graceful-cluster-restart.html#graceful-restart-cluster
Turn off Engage1¶
- Power off openstack VMs, then computes and then controllers.
- Power off Ceph.
- Power off VMs on engage1-services and emergency.
Turn off MaaS VMs and Hosts¶
- Turn off the VMs that are running the hil ipmi gateways.
- Kumo Ceph and Kumo openstack are colocated on the 3 lenovo nodes, those nodes are provisioned under maas.
The researd ceph cluster¶
- Turn it off like how we turn off the production ceph.
oVirt VMs¶
- Simply power off all the VMs except for the hosted engine. Might want to turn off the ipmi-gw and dns server last.
- Do not turn off the ovirt hosts. They will come online when power is restored.
SSO hosts¶
https://galeracluster.com/library/training/tutorials/safe-to-bootstrap-feature.html
- Turn off one SSO host at a time so we don’t mess up galera/mariadb.
Power off the 2 SSD NFS servers¶
- some oVirt VMs are using storage here, so we want to turn these off after the oVIrt VMs.