Day-2 OpenShift is the whole job
The migration is the headline, but after cutover every workload lives here. The hiring manager runs this platform on-prem — he wants someone who upgrades clusters without drama, backs up etcd, understands operators, and knows what RHCOS being immutable means for hardening.
OpenShift vs vanilla Kubernetes
OpenShift is Kubernetes plus opinion. Name the deltas and you signal you've actually run it:
What Red Hat adds
- Operators everywhere — the cluster upgrades itself via the Cluster Version Operator (CVO); add-ons via OLM
- Integrated Router (HAProxy) + integrated registry + ImageStreams
- SCCs — OpenShift's extra security layer on top of RBAC;
restrictedby default - RHCOS — immutable RHEL CoreOS hosts, configured via MachineConfigs (Ignition), not manual edits
- oc — kubectl with platform verbs:
oc adm upgrade,oc debug node/,oc adm must-gather
Why that matters on-prem
- No cloud LB, no cloud storage — you bring MetalLB, LSO/ODF, and your own load balancers for the API
- Upgrades are CVO-driven: pick a channel, watch MachineConfigPools roll node by node
- Everything is Red Hat-supported as a unit — one vendor to call at 3 AM (their RHEL licensing point)
- RHEL workers are an option: user-provisioned RHEL compute nodes for workloads that need traditional agents
Control plane, workers, and the operator trio
Control plane
API server, controllers, scheduler — and etcd, the state of the whole cluster. 3+ control-plane nodes; etcd quorum loss = cluster down. Backup etcd, restore to a new cluster, and you can rebuild anything else.
Workers
Run the workloads. Grouped into MachineConfigPools (master/worker) that control how config changes roll. Draining + MCO reboots are normal, boring, scheduled events.
The operator trio
CVO upgrades the platform · MCO manages host config (Ignition) · OLM installs/updates add-on operators like MetalLB or MTV. "Operators" is the answer to half the admin questions.
Networking: OVN-Kubernetes is the default CNI
Since 4.12, OVN-Kubernetes replaced OpenShiftSDN. It gives you Kubernetes NetworkPolicies plus egress IPs, and it pairs with Multus for multiple network interfaces (NADs) — which matters the moment migrated VMs need VLAN-tagged or macvlan interfaces that look like their old network. Ingress is the HAProxy Router; services get MetalLB VIPs. Traffic map on the L4/L7 page.
Storage: LVM, NFS, iSCSI — the on-prem trio
The posting names these three explicitly, which tells you VMs and pods live on shared SAN/NAS storage you maintain yourself. Know each well enough to troubleshoot under pressure:
LVM
Volume groups, logical volumes, thin pools, snapshots. The rescue toolkit: vgextend/pvmove to swap a failing disk, lvextend + grow filesystem, boot from snapshot for rollback. Default everywhere on RHEL — including the hypervisor hosts themselves.
NFS
Shared storage for anything that needs RWX: pod volumes, VM import/export staging, shared config. Know NFSv4 mount options (hard/soft, intr, timeouts) and the difference between sec=sys and Kerberized mounts — a silent hang vs. a hard failure often comes down to mount flags.
iSCSI
Block storage off a SAN. Workflow: iscsiadm discovery → login → LUN appears as a block device → multipath for redundancy. In OpenShift, iSCSI usually lands via a CSI driver into a StorageClass; the mapping layer is what you debug when volumes don't attach.
On OpenShift: StorageClasses abstract all of this — RWO (block/VM disks) vs RWX (shared) access modes, CSI drivers per backend, and the Local Storage Operator (local disks → PVs) or ODF (Ceph) when there's no SAN. Migration angle: the Windows VMs you're moving probably sit on VMware datastores that are NFS/iSCSI-backed today — the MTV storage map is how they land on OpenShift StorageClasses instead.
The four rituals he'll assume you own
1 · Upgrades
- Choose a channel (stable/fast/EUS — Extended Update Support for slower cadence, 2-year support on 4.x even-numbered)
oc adm upgrade --to=4.14.9, watch CVO progress; MCO rolls pools with maxUnavailable- Canary: upgrade a pool (custom MCP) or infra first; respect PodDisruptionBudgets or drains stall forever
- Pre-upgrade: etcd backup, cert-expiry check, operators pinned?
oc get clusteroperatorshealthy first
2 · etcd
- Backup:
oc get secrets -n openshift-etcd→ run cluster-backup.sh against a healthy member - Restore is a full-cluster operation: restore quorum, then static pod redeploy
- Watch disk latency and defrag schedule — etcd is the most common cause of "everything is slow"
3 · Certificates
- Cluster certs expire (~24h-1y); expired API cert = login failures. Check + renew before they bite
- Ingress/API certs via cert-manager or manual; watch SANs for internal/external names
4 · Node maintenance
- Drain → patch RHCOS → reboot via MCO; use NodeMaintenance where available
- Capacity: HPA + ClusterAutoscaler; watch requests vs limits (scheduling = the #1 incident class)
- Image hygiene: registry pruning, pull-through caches for air-gapped/disconnected (common in locked-down DCs)
SCCs restrict what pods may do (restricted default: no root, no host paths); RBAC gates access; NetworkPolicies segment east-west; the Compliance Operator runs CIS/STIG scans against the cluster; hosts are SELinux-enforcing RHCOS configured only through MachineConfigs. For a PCI-DSS-shaped shop, that chain — SCC → NetworkPolicy → Compliance Operator → host CIS — is exactly the story to tell. More on the hardening page.
The verbs that make you look senior
oc get clusteroperators— first thing on any "cluster is broken" call; find the Degraded oneoc adm must-gather— capture everything for Red Hat support / post-mortemsoc debug node/<node>— jump into a node's host context without SSHoc adm policy,oc adm top,oc rsh— the daily drivers- Pod landmines:
CrashLoopBackOff(read the exit + logs),ImagePullBackOff(registry/creds),Pending(scheduling: resources, taints, quotas, nodeSelector)
Questions to answer out loud
Walk me through upgrading an on-prem OpenShift cluster.
Pre-flight: etcd backup, cert expiry check, all clusteroperators healthy, PDBs understood, operators compatible with the target. Pick the version + channel (EUS if you can't upgrade quarterly). Run oc adm upgrade --to=<ver>; CVO replaces operator manifests, then MCO rolls MachineConfigPools with maxUnavailable — nodes drain, reboot into the new RHCOS, come back. Verify clusteroperators, run smoke tests, then canary workloads before declaring victory.
What's the difference between a Deployment and an operator? Why do operators matter here?
A Deployment runs copies of a pod; an operator encodes a human's operational knowledge — install, upgrade, backup, failover, config drift — as a controller with CRDs. OpenShift's whole lifecycle (CVO, MCO, OLM) is operators. When you migrate Windows VMs to OpenShift Virtualization and expose them through MetalLB, operators are what make day-2 operations repeatable instead of tribal knowledge.
etcd quorum is lost. What now?
The cluster becomes read-only/unavailable — API server can't elect or serve. First, don't panic-reboot everything; find which members are healthy, restore quorum on the healthiest, or restore from the latest etcd snapshot to a fresh cluster if the dataset is intact. This is why scheduled, tested etcd backups are non-negotiable, and why I test restores, not just backups.