BGP Deep Dive — the core skillset this job is hiring for

Interview reality: the hiring manager is screening for "deep BGP." Expect the classic ladder: definitions -> path selection -> design (leaf-spine) -> MetalLB integration -> failure/troubleshooting. Speak in the vocabulary below and you'll sound like the person he needs.

1. What BGP is

• Path-vector exterior gateway protocol, RFC 4271. Runs over TCP port 179 (reliable transport — no retransmission logic needed, unlike OSPF).
• Exchanges reachability info (NLRI = prefix + attributes) between Autonomous Systems (or inside one).
• IGP (OSPF/IS-IS) = interior, fast convergence, topology-based, builds paths within an AS.
• BGP = exterior/inter-AS, policy-based, slow convergence by design, scales to the whole Internet table.
• ASN: 2-byte public 1–64511, private 64512–65534 (RFC 6996); 4-byte private range 4200000000–4294967294. Most DC fabrics use private ASNs.

2. eBGP vs iBGP — say this table cold

Consequences: iBGP full mesh explodes at N^2 (full mesh = N(N-1)/2 sessions) → route reflectors. RR rules: clients + non-clients; routes from a client are reflected to clients AND non-clients; RRs add cluster-id/originator-id attributes to prevent loops. Confederation = sub-AS splitting, less common in DCs now.

3. Session lifecycle (FSM) — the "stuck in Active" question

Idle → Connect (TCP SYN) → Active (retry, TCP couldn't complete) → OpenSent → OpenConfirm → Established.

• OPEN: version, ASN, hold timer, router ID, capabilities (MP-BGP AFI/SAFI, 4-byte ASN, route refresh, graceful restart).
• KEEPALIVE: every hold/3 (e.g. hold 90s → keepalive 30s). Hold timer mismatch: both use the lower.
• UPDATE: withdrawn routes + path attributes + NLRI.
• NOTIFICATION: error code → session torn down.

"Stuck in Active/Idle" causes, in order of likelihood:

1. TCP/179 blocked by firewall/ACL (the classic — filters only permit 80/443).
2. No route to the peer (eBGP single hop and the peer IP isn't on a connected interface).
3. Wrong remote-as or wrong peer IP (mismatched ASN = NOTIFICATION at OpenSent).
4. MD5/TCP-AO password mismatch → TCP resets, never reaches OpenSent.
5. TTL issue with multihop (forgot ebgp-multihop).
6. CoPP/control-plane policing dropping BGP packets under load.
7. Hold timer expiry mid-session = flapping (CPU starvation, MTU blackholing large UPDATEs, saturated link).

4. Path attributes + best-path selection (memorize the order)

Attributes:

• Well-known mandatory: ORIGIN (i = network/aggregate, e = EGP, ? = redistribute), AS_PATH (sequence of ASNs; AS_SET from aggregation), NEXT_HOP.
• Well-known discretionary: LOCAL_PREF (higher wins, iBGP-only, default 100), ATOMIC_AGGREGATE.
• Optional transitive: COMMUNITY (32-bit tag; well-known: NO_EXPORT 0xFFFFFF01, NO_ADVERTISE 0xFFFFFF02), large communities.
• Optional non-transitive: MED (multi-exit discriminator, lower wins — sent TO a neighbor AS to influence ITS inbound choice).

Best-path selection (Cisco/FRR order — know the first 6 cold):

1. Highest weight (Cisco local knob)
2. Highest LOCAL_PREF
3. Locally originated (network/aggregate) beats learned
4. Shortest AS_PATH
5. Lowest ORIGIN type (IGP < EGP < incomplete)
6. Lowest MED (only if from the same neighbor AS, unless always-compare-med)
7. eBGP-learned over iBGP-learned
8. Lowest IGP metric to NEXT_HOP
9. (multipath/ECMP if configured)
10. Oldest eBGP route
11. Lowest router ID, then lowest peer address

Memory trick: "We Love Oranges AS Oranges Mean Pure Refreshment" — Weight, LocalPref, Originated, AS path, Origin, MED, eBGP/iBGP, IGP cost, Router ID.

5. Traffic engineering — the "how do you steer traffic" question

• Inbound (how the world reaches you): AS_PATH prepend (make one link less attractive) and MED (lower = preferred for the neighbor AS). You don't control inbound directly — only via hints.
• Outbound (how you reach the world): LOCAL_PREF (your decision, iBGP-wide), weight (per-router).
• Communities are the automation glue: tag routes at ingress, act at egress. NO_EXPORT prevents leaking routes beyond the peer's AS.

6. On-prem datacenter design — this is the money question

Classic 3-tier (access/distribution/core) with L2 + STP: blocked links, single active path, STP risk. Modern DC answer:

Leaf-spine with eBGP underlay (RFC 7938):

• Every leaf and spine is its own private ASN. eBGP over point-to-point /31 (or unnumbered) L3 links.
• Every link active (no STP — pure L3), ECMP across spines (maximum-paths 4), all paths available.
• BGP unnumbered (RFC 5549): no IPv4 addresses on fabric links; IPv6 link-local peering + extended next-hop encoding carries IPv4 routes. Huge operational win: plug in a leaf, no IPAM.
• BFD (sub-second) on sessions because BGP timers alone converge too slowly.
• Horizontally scales: add spines for bandwidth, add leaves for ports.
• iBGP with route reflectors = acceptable for a small fabric (2-4 leaves); eBGP is the standard answer.

For THIS job the fabric connects to OpenShift: MetalLB/FRR speakers on every worker node peer with the ToR switches and advertise the LoadBalancer VIPs as /32s → ECMP distributes flows across nodes. That is "BGP within the OpenShift environment" done right. (Details in 03.)

Small-DC reality check: if their fabric is small, it may be iBGP + RR or even two collapsed-core routers with HSRP/VRRP + static. Worth asking "what's the fabric topology today?" — good question, and be ready to discuss migrating to eBGP leaf-spine as part of the OCP rollout.

7. BGP to the Internet edge (secondary but askable)

• Multihoming: 2 ISPs, default route or partial/full tables, LOCAL_PREF per ISP, AS_PATH prepend for inbound balance, prefix-lists OUT (never leak — filter everything you don't originate), RPKI ROV (RFC 6811) to drop invalid origin routes, BCP 38 ingress filtering, RTBH (RFC 5635) for DDoS.
• Full table vs default: full = better path selection + resilience; default = cheap RAM. For a mid-size enterprise: partial + defaults is typical.

8. Convergence & stability knobs

• BFD for sub-second link failure detection; BGP PIC (prefix-independent convergence) for fast failover without full recompute.
• Graceful restart + NSR: keep forwarding during control-plane restart.
• Route dampening: rarely used now; flap detection matters more (watch log storms, then find the root — usually a link, MTU, or CPU).
• Add-Path: send multiple paths per prefix (RR HA, fast convergence).
• max-prefix on every session (safety valve), timers tuning, update-source/loopback peering for redundancy (peer to loopback via IGP, not physical IPs).

9. Troubleshooting ladder (interview gold — walk through real flows)

"Route received but not installed" — check in order:

1. Is the session Established? (show bgp summary)
2. Did inbound policy filter it? (prefix-list/route-map IN)
3. Is NEXT_HOP reachable? iBGP route with unreachable next hop = RIB failure. Fix: next-hop-self or ensure IGP has the route.
4. Max-prefix tripped → session shutdown.
5. Same prefix from multiple peers — best-path lost due to higher local-pref/weight elsewhere.

"Session flaps": MTU on the path (large UPDATEs), hold timer starvation (CPU/CoPP), unstable physical link, duplicate router IDs in iBGP mesh, MD5 mismatch mid-session (rare).

"Asymmetric routing / ECMP hashing": ECMP hashes the 5-tuple (src/dst IP, src/dst port, proto) — same flow sticks to one path. Hash polarization happens when spine+leaf use the same hash algorithm → uneven distribution.

Commands (FRR/vtysh, since MetalLB uses FRR): show bgp summary # sessions + state + uptime + PfxRcd show bgp ipv4 unicast <prefix> # path details, attributes, best-path show ip route <prefix> # installed or not, next hop show bgp ipv4 unicast neighbor <ip> advertised-routes / received-routes

10. BGP + Kubernetes (the 30-second version you'll say repeatedly)

BGP's job in a K8s DC: announce LoadBalancer service VIPs from the nodes into the physical fabric so ToR routers can ECMP traffic to it. Calico does the same trick for pod CIDRs (full L3 pod networking, no overlay). In OpenShift's default stack, the overlay (OVN-K, Geneve) handles pod-to-pod; BGP only exists at the MetalLB edge. Be crisp about that boundary — the hiring manager will test it.