User Tools

Site Tools


maintenance:releases:7.1.1_20190924

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

maintenance:releases:7.1.1_20190924 [2019/09/24 08:07] (current)
yspeerte created
Line 1: Line 1:
 +===== NetYCE 7.1.1 Build_20190924 =====
 +===== Release notes =====
 +Date: 2019-09-24
 +
 +
 +\\ 
 +<WRAP widths 60% box safety>
 +==== Enhancement ====
 +</​WRAP>​
 +
 +=== Custom reports scheduling ===
 +<WRAP indent>
 +Custom reports can be scheduled to be executed periodically. The scheduling allowed
 +the time-of-day,​ date and day-of-the-week to be specified. Options for multiple ​
 +times per day, dates or weekdays were not available.
 +
 +Since the report scheduler deploys the standard unix '​crontab'​ functions, these 
 +advanced scheduling functions could be made available for the reports too. The
 +crontab options are powerful but require some familiarity.
 +
 +The Custom reports tool now includes a text box with the resulting crontab time
 +specification after the (basic) choices are made. This text box can then be modified
 +to include the advanced options as desired. This information is then used to
 +schedule the report. Incorrect settings are rejected.
 +
 +The crontab format uses five fields:
 +<​code>​
 + ​.---------------- minute (0 - 59)
 + ​| ​ .------------- hour (0 - 23)
 + ​| ​ |  .---------- day of month (1 - 31)
 + ​| ​ |  |  .------- month (1 - 12) OR jan,​feb,​mar,​apr ...
 + ​| ​ |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,​mon,​tue,​wed,​thu,​fri,​sat
 + ​| ​ |  |  |  |
 + ​* ​ *  *  *  * 
 +
 +Examples:
 + 00 12 * * *                   = daily at noon
 + 05 04 * * mon                 = mondays at 04:05 in the morning
 +</​code>​
 +
 +The advanced options allow values to be divided, enumerated and ranged:
 +<​code>​
 +Examples:
 + */10 * * * *                   = every 10 minutes
 + 00 12 * * mon,​tue,​wed,​thu,​fri ​ = noon on weekdays
 + */15 7-18 * * mon-fri ​         = every quarter hour during office hours
 +</​code>​
 +</​WRAP>​
 +
 +=== Ansible support ===
 +<WRAP indent>
 +Scenario'​s now have the ability to support a (local) **Ansible** installation. ​
 +
 +The new "​Ansible_exec"​ command for scenarios allow the launching of either an 
 +existing Playbook, or the launching of a NetYCE Scenariobook by Ansible. This
 +effective extends the use of NetYCE with the vendors supported by Ansible, while
 +at the same time extending Ansible with the scenarios, parameterisation and
 +modelling of NetYCE.
 +
 +Where the Playbook option refers to a pre-existing playbook-file that can be 
 +used as-is, the Scenariobook refers to the NetYCE template-based generated Playbook
 +that can be parameterised using the NetYCE modelling and database.
 +
 +The NetYCE GUI supporting these Scenariobooks will soon be extended with dedicated XML and
 +YAML template types.
 +</​WRAP>​
 +
 +=== Scenario '​signal_json'​ ===
 +<WRAP indent>
 +Scenarios have long had the ability to signal external systems or users using SNMP traps and
 +emails. Now the ability to signal an event using a RESTFUL JSON transaction has been added.
 +
 +The scenario command "​Signal_json"​ is used to issue a JSON transaction to targets of choice.
 +Details for call (like the authentication to be used) is defined in a dedicated configuration
 +file. 
 +
 +See [[menu:​operate:​scenarios:​commands_details#​signal_json|Scenario commands - Signal_json]] for details ​
 +on the use and configuration of this new command.
 +</​WRAP>​
 +
 +=== Context-template syntax ===
 +<WRAP indent>
 +Context templates are used to include sub-templates indirectly based on values in a relation. The
 +relation has for instance the names of the port-templates of a device. Then the context-template
 +will be substituted with the parsed port-templates for all interfaces in the relation.
 +
 +Common line for a context-template is:
 +<​code>​
 +-- include loopback interfaces
 +{<​Port_template@Loopbacks_node>,​ Loopbacks_node}
 +-- include physical interfaces
 +{<​Port_template@Interfaces>,​ Interfaces}
 +</​code>​
 +
 +The syntax for these context-templates has been extended to support multiple conditions. These
 +conditions can be used to filter the appropriate rows of a relation before including the remaining
 +templates. ​
 +
 +These conditions use the syntax:
 +<​code>​
 +{tpl_reference,​ relation, column=value,​ column=value,​ ...}
 +
 +example:
 +{<​Port_template@Port_map>,​ Port_map, Port_class='​Lo'​}
 +{<​Port_template@Port_map>,​ Port_map, Port_class='​Gi',​ Port_id != '​*2*'​}
 +</​code>​
 +
 +All conditions need to be '​true'​ to be included in the output. Conditions can use the operands '​=' ​
 +and '​!='​ and the values support wildcards. Conditions using non-existing column names are
 +ignored. There is no limit on the number of conditions.
 +</​WRAP>​
 +
 +=== Scenario '​send-email'​ ===
 +<WRAP indent>
 +At customer suggestion the '​send_email'​ command for scenarios has been extended with options
 +for setting the To: and From: email addresses of the issued email.
 +
 +The implementation allows for multiple To: addresses. By default are the user mail-address and
 +optionally (by tweak) the user-group mail-addresses used for the To: field.
 +from the operator ​
 +</​WRAP>​
 +
 +=== Scenario '​script_exec'​ ===
 +<WRAP indent>
 +For scenario'​s a new command has been added that allows scenarios to launch scripts. This 
 +facilitates the use of network automation scripts that engineers build to execute specific
 +tasks. By integrating these scripts, the effort that originally went into the development
 +can be reused.
 +
 +The '​script_exec'​ command will execute any file one the NetYCE server for which the interpreter
 +is installed or is a compiled binary. These interpreters include python, perl, bash and sh by default, ​
 +but can be extended. The python support is version 2.6 for CentOS 6.x and version 2.7 and 3.x for
 +CentOS 7.x.
 +
 +The '​script_exec'​ command includes an option for overriding the interpreter and a timeout to
 +forcibly kill the script after an interval. Script arguments can be passed in using multiple options.
 +</​WRAP>​
 +
 +=== Scenario '​grep'​ ===
 +<WRAP indent>
 +When creating scenarios involving list variables, the '​grep'​ function is a useful command to 
 +extract desired items. This function was reworked to accept filter strings in two distinct
 +formats: wildcards ('​*'​ and '?'​) and regular expressions (regex).
 +
 +The wildcard format is widely used and will satisfy most demands. Wildcard filters are case-insensitive:​
 +<​code>​
 +# filter the entries using WILDCARDS in the -c argument
 +<​nie_names>​ := grep -l <​all_names>​ -c "​m*nie"​
 +</​code>​
 +
 +For more advanced filters the more complex and versatile regex format is available. To distinguish
 +the condition argument must be placed between slashes ('/​.../'​) and accepts modifiers like '​i'​
 +to compare case-insensitive:​
 +<​code>​
 +# filter the entries using REGEX
 +<​nie_names>​ := grep -l <​all_names>​ -c "/​m.*nie$/​i"​
 +</​code>​
 +
 +For details and examples, please consult the wiki page on 
 +[[menu:​operate:​scenarios:​commands_details#​grep| Scenario Command details - grep]]
 +
 +</​WRAP>​
 +
 +=== Scenario '​match'​ ===
 +<WRAP indent>
 +In addition to the '​grep'​ function, the '​match'​ function was added to the scenarios.
 +
 +The behaviour of '​match'​ is similar to the '​grep'​ function but with an essential difference: where 
 +the '​grep'​ returns the actual element of the matching condition, the '​match'​ returns only 
 +the matching string. In essence, the '​match'​ is a grep function and a substring function ​
 +rolled into one.
 +
 +When using the regex condition a distinct feature can be used: multiple substring matches. ​
 +Regular expressions have the capability to '​mark'​ matched strings using '​()'​ and refer to 
 +them by position ($1, $2 and so on). The '​match'​ function allows you to use this feature ​
 +to use these marks in the condition regex and have these marked strings returned as a 
 +separate entry in the result.
 +
 +For details and examples, please consult the wiki page on 
 +[[menu:​operate:​scenarios:​commands_details#​match| Scenario Command details - match]]
 +</​WRAP>​
 +
 +=== Supernets for Sites ===
 +<WRAP indent>
 +Automating network designs including IP-address allocations in NetYCE requires the use of IP-plans.
 +IP-plans (ipv4 and ipv6) provide an abstraction for segmenting and standardising ip-spaces that are
 +to be used in the design. They function as a '​blueprint'​ for allocating required subnets needed
 +to '​build'​ the various components of the design.
 +
 +For the implementation an ip-space (or ip-block) in the form of a Supernet is assigned to a 
 +NetYCE Client along with the IP-plan that will be governing it. Many large or small IP-plans can
 +be made available for the networking design of the Client.
 +
 +Because not all designs allow for a '​Client'​ based approach, customers have requested an option
 +to manage IP-plans and Supernets per Site. Starting this release, Client Supernets can now be 
 +'​reserved'​ to be exclusively used by a Site. The Site will from then on be '​owning'​ the ip-space
 +of that Supernet.
 +
 +The assignment to a Site can be done using the GUI by editing a new or existing Supernet in 
 +the Client details form. The Site reservation is made by entering a valid SiteCode within the 
 +Client. Alternatively the Service-type '​ASSIGN-SUPERNET-SITE_CODE'​ can be used to further ​
 +automate this task.
 +
 +When a subnet for a requested Net_name is to be added, the Net_name is initially searched for in
 +all Supernets '​owned'​ by the Site. When available in multiple Supernets (regardless of ip-plan), the 
 +new subnet is allocated from the Supernet that has the least free subnets of that type (thus filling
 +up used supernets before newer). Should the subnet not be available in '​own'​ Supernets, the search
 +will continue in Supernets that are not '​owned'​ by other Sites but are '​shared'​ within the Client.
 +The same logic in selecting the Supernet then applies when multiple provide the Net_name.
 +
 +For existing designs set up using Client-'​shared'​ Supernets nothing will change, the implementation ​
 +of the Site-supernets is fuly downwards compatible.
 +</​WRAP>​
 +
 +=== Infoblox DNS IPv6 ===
 +<WRAP indent>
 +For customers deploying the Infoblox integration another step in integrating IPv6 was implemented.
 +
 +The module for Infoblox DNS management now allows fo the creation of '​host'​ records with IPv6
 +addresses using the NetYCE '​add_host'​ API call and the scenario function '​dns_add_host'​. ​
 +
 +This initial implementation is limited '​host'​ records and the use of a specified IPV6 address. ​
 +Support for AAAA-records or searching for a '​free'​ IPv6 address is planned for a subsequent phase.
 +
 +See the NetYCE Wiki for details in the pages 
 +[[menu:​operate:​scenarios:​commands_details#​dns_add_host|Scenario commands - dns_add_host]]
 +and
 +[[maintenance:​general:​api_plugin_infoblox_dns#​add_host|Infoblox API plugin - add_host]]
 +</​WRAP>​
 +
 +=== Menu defaults ===
 +<WRAP indent>
 +The NetYCE front-end menus '​Design',​ '​Build',​ '​Operate'​ and '​Admin'​ each have a list of forms and tools
 +for the end-user to choose from. It is now possible to configure the system to define which of these
 +items wil be automatically selected when its menu is opened. ​
 +
 +The selection what form or tool(set) to open for each menu is controlled using the Lookup "​Tweaks"​
 +category. The Variables "​Default_menu_design",​ "​Default_menu_build"​ and so on can be set to specify
 +the item. The values to be used are listed in the Wiki article:
 +[[menu:​admin:​lookup:​lookup_tweaks#​default-menu_tweaks|Default-menu tweaks]].
 +
 +These Default-menu tweaks can also be used in combination with the 
 +[[menu:​admin:​lookup:​lookup_tweaks#​default-tool_tweaks|Default-tool tweaks]]
 +that specify which tool from a tool set is selected by default.
 +</​WRAP>​
 +
 +
 +\\ 
 +<WRAP widths 60% box safety>
 +==== Change ====
 +</​WRAP>​
 +
 +=== Scenario '​config_save'​ ===
 +<WRAP indent>
 +The scenario command '​config_save'​ will push the devices'​ startup-configuration to
 +the NetYCE server. This worked properly but uploaded the existing startup file.
 +
 +The '​config-save'​ procedure has been changed such that it will now first create a new
 +startup file and then upload it to the NetYCE server.
 +</​WRAP>​
 +
 +=== Domain permissions ===
 +<WRAP indent>
 +The required privileges to view and edit the Domain form have been modified. Users with Engineer or
 +Modeller permissions were denied access to the various '​password'​ type fields. These levels have
 +been loosened.
 +
 +The following privileges now apply by default:
 +  * Operators can view the non-password fields but all password fields are starred out and disabled.
 +  * Engineers can edit the non-password fields but all password fields are read-only. The password fields appear starred out until it clicked, then the value is displayed.
 +  * Modellers and Managers have full access to all password and non-password fields. Again the passwords
 +appear starred out until the field becomes active, then it can be edited.
 +
 +This default behaviour can be modified using the Auth_permissions table where for each of the fields and
 +for all user levels the edit permissions can be modified using the "Admin - Custom data" tool for
 +the "​Auth_permissions"​ table. ​
 +
 +<​code>​
 +By filtering on "​Controller:​ '​domains'"​ and "Type: '​field'",​ the value
 +for the desired role can be modified in one of 3, 4, or 5: 
 +  '​3'​ denotes '​disabled', ​
 +  '​4'​ results in '​read-only'​
 +  '​5'​ in full access.
 +</​code>​
 +</​WRAP>​
 +
 +=== Help menu items ===
 +<WRAP indent>
 +The '​Help'​ menu item of all four menus has been removed. The link to the NetYCE Wiki is moved
 +to the upper right hand corner of the page. 
 +
 +This change was made to reduce the number of items in the various menus.
 +</​WRAP>​
 +
 +=== Longer Net_names ===
 +<WRAP indent>
 +At customer request, the '​Net_name'​ of ipv4 and ipv6 subnets can now be 50 characters long. 
 +Previously the maximum length was 20 characters.
 +</​WRAP>​
 +
 +=== Scenario arguments ===
 +<WRAP indent>
 +Scenario functions use arguments like '​-a'​ and '​-p'​ and so on. Some of these functions allow
 +for many (repeated) arguments resulting in very long function calls. Although this offers no
 +functional issues, these long lines of can arguments become unwieldy and look untidy.
 +
 +Therefore the scenario arguments can now be split over several lines. All lines starting with 
 +something that looks like an argument flag (like '-a ' or '-p ') will be taken as an argument
 +belonging to the previous function. This allows for a better overview of the arguments and
 +makes editing a lot easier.
 +
 +Be aware that the list of arguments cannot contain comments (starting with '#'​).
 +</​WRAP>​
 +
 +=== Relation test ===
 +<WRAP indent>
 +After the completion of a NetYCE software update or the updating of a system configuration many
 +configuration files are regenerated and processes requiring them are restarted. Because these
 +software updates also install patches on the NetYCE database, the database could be changed in
 +a fashion that potentially renders customer-created Relations non-functional.
 +
 +Therefore, after all changes were mede and processes are restarted, all existing Relations are 
 +checked for errors and the results listed in the Task-log. This process of error-checking of
 +Relations has been modified in two ways:
 +
 +First, the relation will now use relevant data fields for its substitutions (where possible). ​
 +Where previously all variable substitutions used '​0',​ it now uses the values from one of the 
 +nodes in the database. This results in more relevant queries and catches errors that otherwise
 +would be overlooked.
 +
 +Secondly, since the queries now run on the database using actual values, the queries will likely
 +respond in a more representative fashion. For all relations the required execution time of the query
 +is measures and reported. For those queries where this exceeds one second, the relation is marked
 +as "​Slow"​ and should be examined for optimisation.
 +</​WRAP>​
 +
 +=== Operator permissions ===
 +<WRAP indent>
 +Users with Operator-level permissions will find they have now read-only access to Templates where
 +they previously could not access this sesion of the menu. 
 +
 +The same change was made to the Mpls-vrfs forms.
 +</​WRAP>​
 +
 +
 +\\ 
 +<WRAP widths 60% box safety>
 +==== Fix ====
 +</​WRAP>​
 +
 +=== Scenario '​db_query'​ ===
 +<WRAP indent>
 +The scenario command '​db_query'​ did not return any resulting values from the
 +requested database table. It turned out that the result was returned in an
 +incorrect format.
 +The db_query command now returns a single value or list of values (array) correctly.
 +</​WRAP>​
 +
 +=== Telnet Vendor modules ===
 +<WRAP indent>
 +Several vendor modules exhibited some shortcomings when using Telnet instead of SSH. If a 
 +username or password failed, the job did not report the origin of the login failure but just
 +aborted.
 +
 +The issue has been resolved for most vendors, but testing could not be completed for all 
 +vendors at this time. This testing is in progress.
 +</​WRAP>​
 +
 +=== HP VPN warning messages ===
 +<WRAP indent>
 +The HP C5 and C7 devices will issue error messages about already existing VPN's. These messages
 +were incorrectly ignored by NetYCE jobs.
 +
 +These VPN messags are now added to these modules, and depending on the type, marked as an
 +error (stopping additional commands) or a warning (logging it, but continuing).
 +</​WRAP>​
 +
 +=== Export download ===
 +<WRAP indent>
 +Under some circumstances the download of a model export (Service-type,​ Node-type, Node-groups, ​
 +Region, Domain, Relations, Scenarios) would result in an empty file. The use of vpn tunnels or
 +firewall settings could cause this.
 +
 +The issue was caused by the disallowed http data-type '​xml'​. By changing it to the the '​octet-stream'​
 +data-type, the problem was resolved.
 +</​WRAP>​
 +
 +=== Huawei command parse ===
 +<WRAP indent>
 +When using command parsing to extract information from a Huawei device, the command failed because
 +NetYCE incorrectly enabled the privileged mode where show commands are not allowed.
 +
 +By modifying the Huawei State_actions table entries for this command, the issue was resolved.
 +</​WRAP>​
 +
 +=== Template conditions ===
 +<WRAP indent>
 +Over the years the supported template syntax has been extended with many features. These enhancements
 +also affected the capabilities for template conditionals. As a consequence some valid conditional ​
 +syntaxes of the past suddenly became ambiguous or invalid.
 +
 +To prevent these situations, the NetYCE configuration generator would automatically detect these
 +older style conditionals and convert them on the fly. The kind of corrections are mostly like these:
 +<​code>​
 +original
 +|variable|
 +|variable = value|
 +|!variable@relation = value1,​value2|
 +
 +corrected
 +|<​variable>​|
 +|<​variable>​ = '​value'​|
 +|<​variable@relation>​ != ('​value1','​value2'​)|
 +</​code>​
 +
 +Recently the syntax again allowed for more advanced options but were found working fine for 
 +data substitution situations (template text), but produced unpredictable results in conditionals.
 +These were found to be the result of the implicit conditional corrections outlined above. The
 +existing conversion was not suited to deal with these syntaxes and would break an otherwise
 +valid conditional.
 +
 +A complete new conditional converter was created that will still provide the required conversions
 +for the old-style formats while also recognising the sometimes very complicated syntaxes like:
 +<​code>​
 +|<​variable@relation:​column=<​variable2>>​ != (<​variable3>,'​value'​)|
 +|[function(<​variable1@relation1:'​value1'​)] = <​variable2@relation2>​|
 +</​code>​
 +
 +> Update: some valid conditionals in existing templates were incorrectly malformed by the converter.
 +These cases were identified and fixed. After extensive retesing the resulting converter should now
 +be fully downward compatible.
 +</​WRAP>​
 +
 +===  Scenario variables ===
 +<WRAP indent>
 +Variables defined in the [parameter] section of a scenario are shared with the configuration
 +generator and, though it, the relations. These variables are made available to the generator using an 
 +intermediate format. ​
 +
 +A malformed variable caused the generator to abort when reading the intermediate format resulting
 +in empty configurations. Extending the validation of external variables was added to resolve this issue.
 +</​WRAP>​
 +
 +=== Scenario '​dns_clear_host'​ ===
 +<WRAP indent>
 +A problem was found with the '​dns_clear_host'​ call for Infoblox DNS manipulations. It was found that 
 +some hosts addresses could not be found to be removed while the Infoblox front-end clearly showed
 +it existed in the IPAM views.
 +
 +The issue could be resolved when it became understood that the ip-address was not a regular IPAM
 +address, but consisted of an Infoblox constructed object "​FixedAddr"​ that is used for DHCP 
 +address-to-attribute associations like MAC-addresses. ​
 +
 +The scenario '​dns_clear_host'​ and API '​clear_host"​ function now locate and remove the "​FixedAddr"​
 +like a regular ip-address.
 +</​WRAP>​
 +
  
maintenance/releases/7.1.1_20190924.txt ยท Last modified: 2019/09/24 08:07 by yspeerte