Flooding Reduction in CLOS Networks

In a CLOS topology, OSPF or ISIS routers might receive multiple identical copies of an LSA or LSP from different neighbors, two neighbors may even exchange the same LSA or LSP simultaneously. The unnecessary flooding of link-state information waste valuable resources of OSPF or ISIS routers. To tackle this issue, this document proposes some extensions to OSPF or ISIS so as to reduce the unnecessary flooding within CLOS networks and therefore improve the scalability of CLOS networks. Due to the flooding issue as mentioned above, some data center network operators have opted for BGP as the underlay routing protocol for their CLOS networks. However, with the advent of high-performance Ethernet networks for AI and HPC, it has become necessary to have visibility of the whole network topology, including link capacity and load information, to enable end-to-end path load-balancing, also known as global load-balancing or adaptive routing. This implies that link-state routing protocols like OSPF or ISIS may need to be reviewed as the routing protocol for large-scale AI and HPC Ethernet networks, provided the scaling issue associated with link-state routing protocols is well addressed. To address the scaling issue, this document proposes a pragmatic approach. The idea is to configure partial routers as non-reflectors for a given OSPF area or a given ISIS level. These routers cannot reflect the LSAs or LSPs they receive from neighbors in that OSPF area or ISIS level to their other neighbors in the same OSPF area or ISIS level.

(Note that the diagram above does not include the connections between nodes. However, it can be assumed that leaf nodes are connected to every spine node in their CLOS topology.) In a three-stage CLOS network as shown in Figure 1, also known as a leaf-spine network, all nodes should be in OSPF area zero or ISIS Level-2. All leaf nodes SHOULD be configured as non-reflectors. To further reduce the flooding of link-state, all spine nodes, except for two, COULD also be configured as non-reflectors.

(Note that the diagram above does not include the connections between nodes. However, it can be assumed that the leaf nodes in a given PoD are connected to every spine node in that PoD. Similarly, each spine node (e.g., S1) is connected to all super-spine nodes in the corresponding PoD-interconnect plane (e.g., Plane-1).) For a five-stage CLOS network as illustrated in Figure 2, each Pod consisting of leaf and spine nodes is configured as an OSPF non-zero area or an ISIS Level-1 area. The Pod-interconnect plane consisting of spine and super-spine nodes is configured as an OSPF area zero or an ISIS Level-2 area. Therefore, spine nodes play the role of OSPF area border routers or ISIS Level-1-2 routers. All leaf nodes SHOULD be configured as non-reflectors, and all spine nodes SHOULD be configured as non-reflectors for OSPF area zero or ISIS Level-2. To reduce the link-state flooding further, all spine nodes in each Pod, except for at least two of them, COULD be configured as non-reflectors for the associated non-zero OSPF area or ISIS Level-1 area. Similarly, all super-spine nodes in each Pod-interconnect plane, except for two of them, COULD be configured as non-reflectors.

This memo makes use of the terms defined in and .

Routers that are configured as non-reflectors for a given OSPF area or ISIS level SHOULD NOT reflect the LSAs or LSPs they receive from neighbors in that OSPF area or ISIS level to their other neighbors in the same OSPF area or ISIS level.

TBD.