{"id":359,"date":"2018-10-18T10:34:09","date_gmt":"2018-10-18T09:34:09","guid":{"rendered":"http:\/\/www.dbonline.se\/?p=359"},"modified":"2018-11-06T07:53:40","modified_gmt":"2018-11-06T06:53:40","slug":"getting-xml-data-from-site-to-zabbix","status":"publish","type":"post","link":"https:\/\/www.dbonline.se\/index.php\/2018\/10\/18\/getting-xml-data-from-site-to-zabbix\/","title":{"rendered":"Getting XML data from site to Zabbix"},"content":{"rendered":"<p>This is a horrible problem that i solved in a very dirty way.<\/p>\n<p>And here is how!<\/p>\n<p>First i get the required information from the website and store it in a file.<br \/>\nThis is automatically done witch crontab on linux with the command sudo crontab -e<br \/>\nI will deconstruct bash line i wrote<br \/>\n*\/1 * * * *\u00a0 &#8211; <strong>Crontab information that the bash command should run every minute<\/strong><\/p>\n<p>curl -s &#8221;http:\/\/10.10.20.2\/index.html&#8221; emerson | tac | tac &#8211; <strong>Get the data from the site and pipe it through tac. We do this so the grep command have time to regex the XML table. when this is done the data looks like this:<\/strong><br \/>\n<span style=\"color: #ff0000;\">&lt;?xml version=&#8221;1.0&#8243; encoding=&#8221;UTF-8&#8243; ?&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;?xml-stylesheet type=&#8221;text\/xsl&#8221; href=&#8221;\/simon.xsl&#8221; ?&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;icom:SimpleMonitoring schemaVersion=&#8221;1.01&#8243; appVersion=&#8221;PA 104037&#8243; fileVersion=&#8221;(null)&#8221; xmlns:icom=&#8221;http:\/\/www.emersonnetworkpower.com\/icom&#8221; xmlns:xsi=&#8221;http:\/\/www.w3.org\/2001\/XMLSchema-instance&#8221; xsi:schemaLocation=&#8221;http:\/\/www.emersonnetworkpower.com\/icom icom-1-02.xsd&#8221;&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;Item id=&#8221;354&#8243; name=&#8221;SinglState&#8221;&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;Label&gt;Unit Status&lt;\/Label&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;Value valueType=&#8221;4&#8243; offset=&#8221;0.000&#8243; gain=&#8221;1.000&#8243; precision=&#8221;0&#8243;&gt;Unit On&lt;\/Value&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;Unit&gt;&lt;\/Unit&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;\/Item&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;Item id=&#8221;361&#8243; name=&#8221;LocTemp&#8221;&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;Label&gt;Return Air Temperature&lt;\/Label&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;Value valueType=&#8221;6&#8243; offset=&#8221;0.000&#8243; gain=&#8221;1.000&#8243; precision=&#8221;1&#8243;&gt;26.8&lt;\/Value&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;Unit&gt;&amp;#176;C&lt;\/Unit&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;\/Item&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;Item id=&#8221;379&#8243; name=&#8221;Std. Sensor Humidity&#8221;&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;Label&gt;Return Air Humidity&lt;\/Label&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;Value valueType=&#8221;6&#8243; offset=&#8221;0.000&#8243; gain=&#8221;1.000&#8243; precision=&#8221;1&#8243;&gt;40.1&lt;\/Value&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;Unit&gt;%rH&lt;\/Unit&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;\/Item&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;Item id=&#8221;380&#8243; name=&#8221;Supply Air Temperature&#8221;&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;Label&gt;Supply Air Temperature&lt;\/Label&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;Value valueType=&#8221;6&#8243; offset=&#8221;0.000&#8243; gain=&#8221;1.000&#8243; precision=&#8221;1&#8243;&gt;&#8212;&lt;\/Value&gt;<\/span><br \/>\n<span style=\"color: #ff0000;\">&lt;Unit&gt;&lt;\/Unit&gt;<\/span><\/p>\n<p>grep -oP &#8217;(?&lt;=\\&#8221;&gt;).*?(?![a-zA-Z]|\\d|[.]|\\s)&#8217;\u00a0<strong> &#8211; Regex the output of the curl output. after we have done this the data looks like this:<\/strong><br \/>\n<span style=\"color: #ff0000;\">Unit On<\/span><br \/>\n<span style=\"color: #ff0000;\">26.8<\/span><br \/>\n<span style=\"color: #ff0000;\">40.1<\/span><br \/>\n<span style=\"color: #ff0000;\">23.0<\/span><br \/>\n<span style=\"color: #ff0000;\">24<\/span><br \/>\n<span style=\"color: #ff0000;\">18.0<\/span><br \/>\n<span style=\"color: #ff0000;\">On<\/span><br \/>\n<span style=\"color: #ff0000;\">On<\/span><br \/>\n<span style=\"color: #ff0000;\">Off<\/span><br \/>\n<span style=\"color: #ff0000;\">Off<\/span><br \/>\n<span style=\"color: #ff0000;\">Off<\/span><br \/>\n<span style=\"color: #ff0000;\">Off<\/span><br \/>\n<span style=\"color: #ff0000;\">Off<\/span><br \/>\n<span style=\"color: #ff0000;\">Off<\/span><\/p>\n<p><strong>We then store this data in a file somewhere where zabbix_client can easily get it:<\/strong><br \/>\n&gt; \/home\/zabbix\/zabbixdata\/emerson.txt<\/p>\n<p><strong>The complete constructed bash line looks like this:<\/strong><br \/>\n*\/1 * * * * curl -s &#8221;http:\/\/10.10.20.2\/index.html&#8221; emerson | tac | tac | grep -oP &#8217;(?&lt;=\\&#8221;&gt;).*?(?![a-zA-Z]|\\d|[.]|\\s)&#8217; &gt; \/home\/zabbix\/zabbixdata\/emerson.txt<\/p>\n<p>Now we need to configure the zabbix agents user defined script. Its located in the\u00a0\/etc\/zabbix\/zabbix_agentd.d directory.<br \/>\nThere i created a file with nano,<br \/>\n<strong>command:<\/strong> nano\u00a0etc\/zabbix\/zabbix_agentd.d\/userparameter_emerson.conf<br \/>\nHere i wrote the following user parameters. i Use head and tail to get the specified line and the\u00a0correct value.<br \/>\n<span style=\"color: #ff0000;\">UserParameter=emerson.unit.status,head -1 \/home\/zabbix\/zabbixdata\/emerson.txt<\/span><br \/>\n<span style=\"color: #ff0000;\">UserParameter=emerson.unit.returntemp,head -2 \/home\/zabbix\/zabbixdata\/emerson.txt | tail -1<\/span><br \/>\n<span style=\"color: #ff0000;\">UserParameter=emerson.unit.returhum,head -3 \/home\/zabbix\/zabbixdata\/emerson.txt | tail -1<\/span><br \/>\n<span style=\"color: #ff0000;\">UserParameter=emerson.unit.supplytemp,head -4 \/home\/zabbix\/zabbixdata\/emerson.txt | tail -1<\/span><br \/>\n<span style=\"color: #ff0000;\">UserParameter=emerson.unit.returntempsetpoint,head -5 \/home\/zabbix\/zabbixdata\/emerson.txt | tail -1<\/span><br \/>\n<span style=\"color: #ff0000;\">UserParameter=emerson.unit.returnhumsetpoint,head -6 \/home\/zabbix\/zabbixdata\/emerson.txt | tail -1<\/span><br \/>\n<span style=\"color: #ff0000;\">UserParameter=emerson.unit.supplytempsetpoint,head -7 \/home\/zabbix\/zabbixdata\/emerson.txt | tail -1<\/span><br \/>\n<span style=\"color: #ff0000;\">UserParameter=emerson.unit.fanstatus,head -8 \/home\/zabbix\/zabbixdata\/emerson.txt | tail -1<\/span><br \/>\n<span style=\"color: #ff0000;\">UserParameter=emerson.unit.coolingstatus,head -9 \/home\/zabbix\/zabbixdata\/emerson.txt | tail -1<\/span><br \/>\n<span style=\"color: #ff0000;\">UserParameter=emerson.unit.freecoolingstatus,head -10 \/home\/zabbix\/zabbixdata\/emerson.txt | tail -1<\/span><br \/>\n<span style=\"color: #ff0000;\">UserParameter=emerson.unit.electricalheatingstatus,head -11 \/home\/zabbix\/zabbixdata\/emerson.txt | tail -1<\/span><br \/>\n<span style=\"color: #ff0000;\">UserParameter=emerson.unit.hotwaterstatus,head -12 \/home\/zabbix\/zabbixdata\/emerson.txt | tail -1<\/span><br \/>\n<span style=\"color: #ff0000;\">UserParameter=emerson.unit.dehumstatus,head -13 \/home\/zabbix\/zabbixdata\/emerson.txt | tail -1<\/span><br \/>\n<span style=\"color: #ff0000;\">UserParameter=emerson.unit.humstatus,head -14 \/home\/zabbix\/zabbixdata\/emerson.txt | tail -1<\/span><br \/>\n<span style=\"color: #ff0000;\">UserParameter=emerson.unit.maintstatus,head -15 \/home\/zabbix\/zabbixdata\/emerson.txt | tail -1<\/span><\/p>\n<p>The syntax is: UserParameter=key,value<br \/>\nThe key needs to correspond to the zabbix item i&#8217;m going to create and the value is the bash script to get the correct value from the file we are creating every minute.<\/p>\n<p>Now we need to restart the agent so it detects the userparameters: sudo service zabbix-agent restart<\/p>\n<p>In Zabbix we then on the zabbix server add an <strong>items<\/strong> with corresponding keys<img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" data-attachment-id=\"372\" data-permalink=\"https:\/\/www.dbonline.se\/index.php\/2018\/10\/18\/getting-xml-data-from-site-to-zabbix\/image2\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.dbonline.se\/wp-content\/uploads\/2018\/10\/image2.png?fit=816%2C855&amp;ssl=1\" data-orig-size=\"816,855\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"image2\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/www.dbonline.se\/wp-content\/uploads\/2018\/10\/image2.png?fit=286%2C300&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/www.dbonline.se\/wp-content\/uploads\/2018\/10\/image2.png?fit=720%2C754&amp;ssl=1\" class=\"aligncenter size-full wp-image-372\" src=\"https:\/\/i0.wp.com\/www.dbonline.se\/wp-content\/uploads\/2018\/10\/image2.png?resize=720%2C754\" alt=\"\" width=\"720\" height=\"754\" srcset=\"https:\/\/i0.wp.com\/www.dbonline.se\/wp-content\/uploads\/2018\/10\/image2.png?w=816&amp;ssl=1 816w, https:\/\/i0.wp.com\/www.dbonline.se\/wp-content\/uploads\/2018\/10\/image2.png?resize=286%2C300&amp;ssl=1 286w, https:\/\/i0.wp.com\/www.dbonline.se\/wp-content\/uploads\/2018\/10\/image2.png?resize=768%2C805&amp;ssl=1 768w\" sizes=\"auto, (max-width: 720px) 100vw, 720px\" \/><\/p>\n<p><strong>DONE!<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"<p>This is a horrible problem that i solved in a very dirty way. And here is how! First i get the required information from the website and store it in a file. This is automatically done witch crontab on linux<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"enabled":false},"version":2}},"categories":[1],"tags":[],"class_list":["post-359","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p8vDvB-5N","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/www.dbonline.se\/index.php\/wp-json\/wp\/v2\/posts\/359","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dbonline.se\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dbonline.se\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dbonline.se\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dbonline.se\/index.php\/wp-json\/wp\/v2\/comments?post=359"}],"version-history":[{"count":4,"href":"https:\/\/www.dbonline.se\/index.php\/wp-json\/wp\/v2\/posts\/359\/revisions"}],"predecessor-version":[{"id":373,"href":"https:\/\/www.dbonline.se\/index.php\/wp-json\/wp\/v2\/posts\/359\/revisions\/373"}],"wp:attachment":[{"href":"https:\/\/www.dbonline.se\/index.php\/wp-json\/wp\/v2\/media?parent=359"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dbonline.se\/index.php\/wp-json\/wp\/v2\/categories?post=359"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dbonline.se\/index.php\/wp-json\/wp\/v2\/tags?post=359"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}