7. Graph Library and Inbuilt Nodes¶

7.4.1. Anatomy of Node:¶

The Fig. 7.1 diagram depicts the anatomy of a node.

The node is the basic building block of the graph framework.

A node consists of:

7.4.2. Node creation and registration¶

• Node implementer creates the node by implementing ops and attributes of struct cne_node_register.

• The library registers the node by invoking CNE_NODE_REGISTER on library load using the constructor scheme. The constructor scheme used here to support multi-process.

7.4.3. Link the Nodes to create the graph topology¶

The Fig. 7.2 diagram shows a graph topology after linking the N nodes.

Once nodes are available to the program, Application or node public API functions can links them together to create a complex packet processing graph.

There are multiple different types of strategies to link the nodes.

7.4.4. Create the graph object¶

Now that the nodes are linked, it’s time to create a graph by including the required nodes. The application can provide a set of node patterns to form a graph object. The fini() API used underneath for the pattern matching to include the required nodes. After the graph create any changes to nodes or graph is not allowed.

The cne_graph_create() API shall be used to create the graph.

Example of a graph object creation:

{"pktdev_rx-0-0", ip4*, pktdev_tx-*"}

In the above example, A graph object will be created with pktdev Rx node of port 0 and queue 0, all ipv4* nodes in the system, and pktdev tx node of all ports.

7.4.5. Multicore graph processing¶

In the current graph library implementation, specifically, cne_graph_walk() and cne_node_enqueue* fast path API functions are designed to work on single-core to have better performance. The fast path API works on graph object, So the multi-core graph processing strategy would be to create graph object PER WORKER.

7.4.6. In fast path¶

Typical fast-path code looks like below, where the application gets the fast-path graph object using cne_graph_lookup() on the worker thread and run the cne_graph_walk() in a tight loop.

struct cne_graph *graph = cne_graph_lookup("worker0");

while (!done) {
    cne_graph_walk(graph);
}

7.4.7. Context update when graph walk in action¶

The fast-path object for the node is struct cne_node.

It may be possible that in slow-path or after the graph walk-in action, the user needs to update the context of the node hence access to struct cne_node * memory.

cne_graph_foreach_node(), cne_graph_node_get(), cne_graph_node_get_by_name() APIs can be used to to get the struct cne_node*. cne_graph_foreach_node() iterator function works on struct cne_graph * fast-path graph object while others works on graph ID or name.

7.4.8. Get the node statistics using graph cluster¶

The user may need to know the aggregate stats of the node across multiple graph objects. Especially the situation where each graph object bound to a worker thread.

Introduced a graph cluster object for statistics. cne_graph_cluster_stats_create() API shall be used for creating a graph cluster with multiple graph objects and cne_graph_cluster_stats_get() to get the aggregate node statistics.

An example statistics output from cne_graph_cluster_stats_get()

+---------+-----------+-------------+---------------+-----------+---------------+-----------+
|Node     |calls      |objs         |realloc_count  |objs/call  |objs/sec(10E6) |cycles/call|
+---------------------+-------------+---------------+-----------+---------------+-----------+
|node0    |12977424   |3322220544   |5              |256.000    |3047.151872    |20.0000    |
|node1    |12977653   |3322279168   |0              |256.000    |3047.210496    |17.0000    |
|node2    |12977696   |3322290176   |0              |256.000    |3047.221504    |17.0000    |
|node3    |12977734   |3322299904   |0              |256.000    |3047.231232    |17.0000    |
|node4    |12977784   |3322312704   |1              |256.000    |3047.243776    |17.0000    |
|node5    |12977825   |3322323200   |0              |256.000    |3047.254528    |17.0000    |
+---------+-----------+-------------+---------------+-----------+---------------+-----------+

7.4.9. Node writing guidelines¶

The process() function of a node is the fast-path function and that needs to be written carefully to achieve max performance.

Broadly speaking, there are two different types of nodes.

7.4.10. Static nodes¶

The first kind of nodes are those that have a fixed next_nodes[] for the complete burst (like pktdev_rx, pktdev_tx) and it is simple to write. process() function can move the obj burst to the next node either using cne_node_next_stream_move() or using cne_node_next_stream_get() and cne_node_next_stream_put().

7.4.11. Intermediate nodes¶

The second kind of such node is intermediate nodes that decide what is the next_node[] to send to on a per-packet basis. In these nodes,

• Firstly, there has to be the best possible packet processing logic.

• Secondly, each packet needs to be queued to its next node.

This can be done using cne_node_enqueue_[x1|x2|x4]() APIs if they are to single next or cne_node_enqueue_next() that takes array of nexts.

In scenario where multiple intermediate nodes are present but most of the time each node using the same next node for all its packets, the cost of moving every pointer from current node’s stream to next node’s stream could be avoided. This is called home run and cne_node_next_stream_move() could be used to just move stream from the current node to the next node with least number of cycles. Since this can be avoided only in the case where all the packets are destined to the same next node, node implementation should be also having worst-case handling where every packet could be going to different next node.

7.6.1. pktdev_rx¶

This node does cne_eth_rx_burst() into stream buffer passed to it (src node stream) and does cne_node_next_stream_move() only when there are packets received. Each cne_node works only on one Rx port and queue that it gets from node->ctx. For each (port X, rx_queue Y), a cne_node is cloned from pktdev_rx_base_node as pktdev_rx-X-Y in cne_node_eth_config() along with updating node->ctx. Each graph needs to be associated with a unique cne_node for a (port, rx_queue).

7.6.2. pktdev_tx¶

This node does cne_eth_tx_burst() for a burst of objs received by it. It sends the burst to a fixed Tx Port and Queue information from node->ctx. For each (port X), this cne_node is cloned from pktdev_tx_node_base as “pktdev_tx-X” in cne_node_eth_config() along with updating node->context.

Since each graph doesn’t need more than one Txq, per port, a Txq is assigned based on graph id to each cne_node instance. Each graph needs to be associated with a cne_node for each (port).

7.6.3. pkt_drop¶

This node frees all the objects passed to it considering them as cne_mbufs that need to be freed.

7.6.4. ip4_lookup¶

This node is an intermediate node that does LPM lookup for the received ipv4 packets and the result determines each packets next node.

On successful LPM lookup, the result contains the next_node id and next-hop id with which the packet needs to be further processed.

On LPM lookup failure, objects are redirected to pkt_drop node. cne_node_ip4_route_add() is control path API to add ipv4 routes. To achieve home run, node use cne_node_stream_move() as mentioned in above sections.

7.6.5. ip4_rewrite¶

This node gets packets from ip4_lookup node with next-hop id for each packet is embedded in node_mbuf_priv1(mbuf)->nh. This id is used to determine the L2 header to be written to the packet before sending the packet out to a particular pktdev_tx node. cne_node_ip4_rewrite_add() is control path API to add next-hop info.

7.6.6. null¶

This node ignores the set of objects passed to it and reports that all are processed.