The YumaPro performance advantages in real world deployment scenarios are described in this article and compared to performance of open source servers Yuma123 and Netopeer2.


Typical provisioning applications work by creating a lot of the same type of list entries and use non-trivial YANG models. 

Powerful YANG constraints that provide referential integrity and model-based system validation can significantly affect performance of the server since all YANG statements must be validated and applied to each related node.


Server performance is tested with the following common scenarios:

  1. Provisioning Test: N Transactions Each Adding 1 Entry; typical use case, used most often in normal operation
  2. Typical YANG Test, such as Openconfig: Create N Entries That Reference Other Parts Of The Configuration In One Transaction; important because most real world YANG models are complex and require high memory and CPU usage from the underlying system
  3. Restore Test: Create N Entries in 1 Transaction; not used often, used for exception handling and not for everyday operation


NOTE: details such as the YANG model used in the tests are listed at the end of this article.


1: Provisioning Test:  N Transactions Each Adding 1 Entry




Times to perform N transactions each adding one entry. Less time is better.


 N:

100

1,000

5,000

10,000

25,000

50,000

Slower than YumaPro

YumaPro-Basic

0.55

1.82

19.17

72.48

487.12

2,024.83
(34 mins)


Netopeer2

0.96

 8.23

 125.6

 452.3

 2,952.00

(49 mins)

 12,831.00
(3.6 hours)

up to 6 times slower

Yuma123

0.61

5.06

578.3

4,750.00
(1.3 hours)

80,461.00
(22 hours)

 -

up to 165 or more times slower

All times in seconds.

 

2: Complex YANG Test, such as Openconfig:  Create N Entries That Reference Other Parts Of The Configuration In One Transaction




Times to create N entries that reference other parts of the configuration in one transaction. Less time is better.

 


 N:

100

1,000

5,000

10,000

25,000

50,000

Slower than YumaPro

YumaPro-Basic

0.53

0.58

0.61

0.8

1.25

2.3


Netopeer2

0.52

 ¹

 ¹

 ¹

 ¹

 ¹

crashes with 1,000 or more entries

Yuma123

0.59

15.47

2,295.00

(38 mins)

17,876.00

(5 hours)

297,137.00
(82 hours)

 -

up to 237,000 or more times slower


¹ Tests with values over 100 cause the server to crash.

All times in seconds.


3: Restore Test:  Create N Entries in 1 Transaction




Times to create N entries in one transaction. Less time is better.

 

 N:

100

1,000

5,000

10,000

25,000

50,000

Slower than YumaPro

YumaPro-Basic

0.58

0.54

0.62

0.74

1.13

1.71


Netopeer2

0.56

0.65

0.73

0.95

1.68

2.76

up to 1.6 times slower

Yuma123

0.59

0.66

1.7

5.14

22.94

 -

up to 20 or more times slower

All times in seconds.



Technical descriptions of the scenarios:

  1. Provisioning Test: Create N number of "/top-container/local-list" list entries one after another - every list entry will be created in a separate transaction
  2. Complex YANG Test, such as Openconfig: Create N number of "local-leafref-members" list entries that reference the "/top-container/local-list/name" key value with ONE <edit-config>
  3. Restore Test: Create N number of "/top-container/local-list" list entries with ONE <edit-config> - all the list entries and parent container will be created in one transaction


NOTE: the default target is set to "running" for simplicity.


The YANG data model listed below is used in the tests . It contains a few data nodes and YANG constraints:


module perf-test {
  namespace "http://netconfcentral.org/ns/perf-test";
  prefix "perft";

  revision 2017-10-23 {
    description "Initial revision.";
  }

  typedef local-ref {
    type leafref {
      path "/top-container/local-list/name";
    }
  }

  list local-leafref-members {
    key "name";

    max-elements 100000;

    leaf name {
      type local-ref;

      must "/perft:top-container/perft:local-list[perft:name = current()]"
        +  "/perft:type = 'l2vlan'" {
        error-message
          "Only VLAN interfaces can be added!";
      }

    }
  }

  container top-container {
    list local-list {
      key "name";

      leaf name {
        type string;
      }

      leaf type {
        type string;
        mandatory true;
      }
    }
  }

  leaf local-leafref-leaf {
    type local-ref;
  }

}



For more details please contact us at: support@yumaworks.com