Parsing command response is a challenge when automating network elements. It was a big challenge when interfaces were cli due to complexity and variation in cli implementation among vendors. NETCONF together with Python xml libraries made life easier. I wrote two lab-7 & lab-8 on how to parse command response in Python. They are written for NETCONF xml response using minidom and etree, xpath.
PyEZ table and view abstract xml parsing from user. Under the hood it does xml parsing in xpath but as a user you don’t need to worry about it. A user simply provides the command and the response fields they are interested in, PyEZ does the parsing and adds it in Python dictionary. User data is formatted in yaml.
In this lab I will demonstrate below functions of PyEZ
1) How to use existing PyEZ tables. PyEZ comes with lots of pre-built tables check references links for latest tables
2) How to quickly create your own tables & views on the fly in the Python script
3) how to build external table and use it in Python script
Below construction of table & view
table name - user defined table name rpc - rpc command, for example get-interface-information. you can get rpc command for a cli like this 'show interfaces | display xml rpc' args - default optional argument for the rpc command args_key - optional key for the table. if this key is not set you need to specify key name for get() like this portStatsTable(dev).get(interface_name'ge-0/0/1'). If key is set you can run get() like this portStatsTable(dev).get('ge-0/0/1') item - This is the top level xpath expression in <rpc-reply>. PyEZ searches for this string when parsing command response view - view table name fields - key value pair of dictionary. Here you specify response fields you are interested in, you also need to create key for that field
Using pre-built tables
Let’s see how to use pre-built PyEZ table. I am interested in interface error stats so I will try PhyPortErrorTable. This table contains input,output received counts and input,output error counts.
Below is the snippet of yaml file for the table
PhyPortErrorTable:
rpc: get-interface-information
args:
extensive: True
interface_name: '[fgx]e*'
args_key: interface_name
item: physical-interface
view: PhyPortErrorView
PhyPortErrorView:
groups:
ts: traffic-statistics
rxerrs: input-error-list
txerrs: output-error-list
fields_ts:
rx_bytes: { input-bytes: int }
rx_packets: { input-packets: int }
<truncated ....>
fields_rxerrs:
rx_err_input: { input-errors: int }
rx_err_drops: { input-drops: int }
<truncated .....>
fields_txerrs:
tx_err_carrier-transitions: { carrier-transitions: int }
tx_err_output: { output-errors: int }
<truncated ....>
Step-1: Start Python and import required packages
sjakhwal@rtxl3rld05:~$ python Python 2.7.6 (default, Mar 22 2014, 22:59:56) [GCC 4.8.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from jnpr.junos import Device >>> from jnpr.junos.utils.config import Config >>> from lxml import etree >>> dev = Device(host='192.168.122.35',user='root',passwd='juniper1').open() >>> dev.facts['model'] 'FIREFLY-PERIMETER' >>>
Step-2: Import PyEZ table and get command response. PyEZ keeps command response in a Python dictionary format, it makes name field as key automatically. You can see that interface_names are keys in the dictionary
>>>from jnpr.junos.op.phyport import PhyPortErrorTable
>>> from pprint import pprint as pp
>>> phyErrors = PhyPortErrorTable(dev).get()
>>> phyErrors.keys()
[‘ge-0/0/0’, ‘ge-0/0/1’, ‘ge-0/0/2’]
>>>
>>> pp (phyErrors[‘ge-0/0/1’].items())
[(‘tx_err_resource’, 0),
(‘rx_err_drops’, 0),
(‘rx_packets’, 562159),
(‘rx_err_l2-channel’, 0),
(‘tx_err_collisions’, 0),
(‘tx_err_drops’, 0),
(‘tx_err_fifo’, 0),
(‘rx_err_frame’, 0),
(‘rx_err_runts’, 0),
(‘tx_bytes’, 84),
(‘tx_err_output’, 0),
(‘tx_err_aged’, 0),
(‘tx_err_hs-crc’, 0),
(‘rx_err_l2-mismatch’, 0),
(‘tx_err_carrier-transitions’, 1),
(‘rx_bytes’, 29229986),
(‘rx_err_fifo’, 0),
(‘rx_err_resource’, 0),
(‘rx_err_l3-incompletes’, 0),
(‘tx_err_mtu’, 0),
(‘rx_err_discards’, 0),
(‘rx_err_input’, 0),
(‘tx_packets’, 2)]
>>>
>>> pp (phyErrors[‘ge-0/0/1’].rx_bytes)
29229986
>>>
>>> for key in phyErrors.keys():
… for phyError in phyErrors[key].items():
… pp (phyError)
…
(‘tx_err_resource’, 0)
(‘rx_err_drops’, 0)
(‘rx_packets’, 783248)
(‘rx_err_l2-channel’, 0)
(‘tx_err_collisions’, 0)
(‘tx_err_drops’, 0)
(‘tx_err_fifo’, 0)
(‘rx_err_frame’, 0)
(‘rx_err_runts’, 0)
….
<truncated>
creating table and views on the fly
Now lets create our own table and views in the Python script. I am using same table (phyPortErrorStatsTable) but updated it to parse only ‘traffic-statistics’ & ‘input-error-list’. I have also reduced the number of error counts.
Note:Be extra cautious about yaml format, make sure all fields are aligned properly otherwise loading will fail.
Step-1: You need to load two new packages for this. Import packages and create YAML string
>>>from jnpr.junos.factory.factory_loader import FactoryLoader
>>>import yaml
>>> PortErrorStatsTableString = """
... ---
... customPortErrorStatsTable:
... rpc: get-interface-information
... args:
... extensive: True
... interface_name: '[fgx]e*'
... args_key: interface_name
... item: physical-interface
... view: customPhyPortErrorView
...
... customPhyPortErrorView:
... groups:
... ts: traffic-statistics
... rxerrs: input-error-list
... fields_ts:
... rx_packets: { input-packets: int }
... tx_packets: { output-packets: int }
... fields_rxerrs:
... rx_err_drops: { input-drops: int }
... rx_err_discards: { input-discards: int }
... """
Step-2: Load YAML string and get command response. Remaining steps are same as using pre-built tables
>> globals().update(FactoryLoader().load(yaml.load(PortErrorStatsTableString)))
>>>
>>> customErrorsTable = customPortErrorStatsTable(dev).get()
>>> print customErrorsTable
customPortErrorStatsTable:192.168.122.35: 3 items
>>> print customErrorsTable.keys()
['ge-0/0/0', 'ge-0/0/1', 'ge-0/0/2']
>>> pp (customErrorsTable.items())
[('ge-0/0/0',
[('rx_err_drops', 0),
('rx_packets', 792394),
('tx_packets', 19432),
('rx_err_discards', 0)]),
('ge-0/0/1',
[('rx_err_drops', 0),
('rx_packets', 571240),
('tx_packets', 2),
('rx_err_discards', 0)]),
('ge-0/0/2',
[('rx_err_drops', 0),
('rx_packets', 0),
('tx_packets', 0),
('rx_err_discards', 0)])]
>>>
>>> print customErrorsTable['ge-0/0/2'].tx_packets
0
>>>
Using table and views from external file
okay so the last part is to create external table file.
create a sub-directory myPyezTable and create .yml and .py file using below steps
Step-1: create portStats.py and put these 4 statements in the file
from jnpr.junos.factory import loadyaml from os.path import splitext _YAML_ = splitext(__file__)[0] + '.yml globals().update(loadyaml(_YAML_))
Step-2: create portStats.yml file and add your yaml table in it. Note both yml and py file names are same
---
customPortErrorStatsTable:
rpc: get-interface-information
args:
extensive: True
interface_name: '[fgx]e*'
args_key: interface_name
item: physical-interface
view: customPhyPortErrorView
customPhyPortErrorView:
groups:
ts: traffic-statistics
rxerrs: input-error-list
fields_ts:
rx_packets: { input-packets: int }
tx_packets: { output-packets: int }
fields_rxerrs:
rx_err_drops: { input-drops: int }
rx_err_discards: { input-discards: int }
Step-3: add _init.py file in the sub-directory. This is an empty file
Step-4: import your table. Remaining steps are same as using pre-build PyEZ tables
>>> from myPyezTable.portStats import customPortErrorStatsTable
>>> portErrors = customPortErrorStatsTable(dev).get()
>>> pp (portErrors.items())
[('ge-0/0/0',
[('rx_err_drops', 0),
('rx_packets', 793354),
('tx_packets', 19525),
('rx_err_discards', 0)]),
('ge-0/0/1',
[('rx_err_drops', 0),
('rx_packets', 572104),
('tx_packets', 2),
('rx_err_discards', 0)]),
('ge-0/0/2',
[('rx_err_drops', 0),
('rx_packets', 0),
('tx_packets', 0),
('rx_err_discards', 0)])]
>>> pp (portErrors.keys())
['ge-0/0/0', 'ge-0/0/1', 'ge-0/0/2']
>>>
View of my sub-directory & external files
sjakhwal@rtxl3rld05:~/myPyezTable$ pwd
/home/sjakhwal/myPyezTable
sjakhwal@rtxl3rld05:~/myPyezTable$ ls
__init__.py __init__.pyc portStats.py portStats.pyc portStats.yml
sjakhwal@rtxl3rld05:~/myPyezTable$ cat __init__.py
sjakhwal@rtxl3rld05:~/myPyezTable$ cat portStats.py
from jnpr.junos.factory import loadyaml
from os.path import splitext
_YAML_ = splitext(__file__)[0] + '.yml'
globals().update(loadyaml(_YAML_))
sjakhwal@rtxl3rld05:~/myPyezTable$ cat portStats.yml
---
customPortErrorStatsTable:
rpc: get-interface-information
args:
extensive: True
interface_name: '[fgx]e*'
args_key: interface_name
item: physical-interface
view: customPhyPortErrorView
customPhyPortErrorView:
groups:
ts: traffic-statistics
rxerrs: input-error-list
fields_ts:
rx_packets: { input-packets: int }
tx_packets: { output-packets: int }
fields_rxerrs:
rx_err_drops: { input-drops: int }
rx_err_discards: { input-discards: int }
References:
Welcome to cloudNet 