User Tools

Site Tools


guides:user:compliance:policies

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
guides:user:compliance:policies [2020/05/22 13:18]
jbosch
guides:user:compliance:policies [2020/06/24 13:58] (current)
pgels [Multiconfig compliance]
Line 1: Line 1:
 +{{indexmenu_n>​2}}
  
 +====== Compliance Policies =====
 +
 +
 +===== Policies =====
 +
 +==== Compliance policies ====
 +The compliance policies form can be found in the Operate->​Compliance form. This is the form where all compliance policies, rules and conditions can be created and maintained. ​
 +
 +{{:​guides:​user:​compliance:​cmpl_policies.png|}}
 +
 +The top-left grid lists all policies, their descriptions and whether they are enabled. The last column gives an overview of the policy'​s rules, conditions, and the number of node groups assigned to it.
 +
 +The new-button creates a new policy. Edit and delete can respectively edit and delete the policy'​s data. Duplicate duplicates a policy, including its rules and conditions (including its node groups). Import imports either a HPNA or netYCE policy export file, and export exports the policy. You can import and export multiple policies at the same time. If an import includes a policy with a name that already exists, the question whether or not to overwrite it is provided. With search you can search through the policies.
 +
 +The top-right grid lists all those node groups and the new-button allows you to assign node groups to this policy. The delete-button allows you to remove node groups from this policy. ​
 +
 +{{:​guides:​user:​compliance:​cmpl_edit_policy.png|}}
 +
 +A policy'​s name is what you get to see in the reports. The description is purely what you get to see in the edit form and below the policies grid for extra clarification. ​
 +
 +A policy that is not enabled will not be checked for compliance. If the "Run cmpl on config change"​-checkbox is checked, whenever the nccmd daemon detects a change in config (through syslog), it checks for compliance. This is on by default, but recommended to be turned off for nodes whose configuration changes often. Policy schedules are an alternative in these cases, which will become available in compliance phase 2.
 +
 +
 +==== Signalling ====
 +
 +A compliance check can have one of these results:
 +  * A compliant node becomes non compliant
 +  * A non-compliant node becomes compliant
 +  * A non-compliant node stays non-compliant
 +  * A compliant node stays compliant
 +
 +These settings allow for signals that just notify on compliance state changes or each time a compliance validation is made.
 +
 +There are four ways we can send signals:
 +    * A trap message
 +    * A syslog message
 +    * An email message
 +    * A REST-API '​post'​ call using Json
 +
 +These settings are available in the front-end when editing a policy using four separate check-boxes allowing for multiple signal types for the same event.
 +
 +These signal messages need setting-up for the environment and mostly requires the definition of the remote targets and priority. Configure these in the file ''​**etc/​signal_cmpl.conf**''​.
 +
 +Details on the configuration of these signalling-types is available in the Reference guide on 
 +[[guides:​reference:​compliance:​cmpl_signalling|Compliance signalling]].
 +
 +Please note signals at the moment are not combined. So be careful when for example creating email signals: if a lot of triggers happen at the same time, a lot of emails will be sent and you could potentially set off a mini ddos attack.
 +
 +===== Rules =====
 +
 +{{:​guides:​user:​compliance:​cmpl_rules_overview.png|}}
 +
 +The rules grid shows the currently selected policy'​s rules. The buttons here are also pretty straightforward,​ however the test-button tests if the logic of the rule's conditions is valid or not for our parser. If not, this rule cannot be used on compliance checks.
 +
 +{{:​guides:​user:​compliance:​cmpl_edit_rule.png|}}
 +
 +The nccmd daemon divides a node's config up in logical blocks. This can be based on indentation,​ line markers or multple lines that have the same start. This also happens hierarchically,​ so blocks within blocks are also recognized. ​
 +
 +Rule_start is meant to recognize these blocks. It can be either plain text or a regex, and it will match the first line of a block against the Rule start. If multiple blocks match, they will all be selected and checked for compliance, so if any of them is not compliant the entire rule isn't. If no Rule start is provided, the entire config will be taken.
 +
 +Hierarchical blocks, so blocks within blocks, can be matched by pasting the first line of all the blocks in the hierarchy in the Rule start field. This is especially useful with Juniper nodes. ​
 +
 +Rule end is optional. If it is set, the matched block'​s last line will be matched. If it doesn'​t match, then the block won't match and will not be checked for compliance. ​
 +
 +If no block is matched, the rule is automatically compliant.
 +
 +Configuration rules will match against a config. Command rules will execute a command on the node and match against its results. Multiconfig will compare the configs of multiple nodes with each other. ​
 +===== Conditions =====
 +
 +{{:​guides:​user:​compliance:​conditions_grid.png|}}
 +
 +The bottom grid is for the rule's conditions. New creates, edit edits, delete deletes, search searches. And since conditions have a sequence, you can move them with the up and down buttons. Note that the edit form is different, depending on the condition'​s type and the rule's type: a logic condition does not need any lines, so the fields for that are not shown.
 +
 +{{:​guides:​user:​compliance:​cmpl_edit_condition.png|}}
 +
 +Conditions take a piece of text, defined by the rule and match it against a number of lines, defined in the first big text area. They either need to match part of each line, a line fully, need to not be present or need to match the block exclusively with no other lines present. They can or cannot contain regex. ​
 +
 +Logical conditions are there to create logic around the conditions with if-, and- and or-statements. A condition also can be disabled. In that case it won't be included in the rule's logic. You can see which conditions are disabled in the condition grid when they are greyed out. 
 +
 +Names can be anything, but a good convention is to give them simple, single-letter names. ​
 +
 +==== Condition Variables ====
 +Starting from compliance phase 2, you can also parse relations and variables. For example, a line:
 +<​code>​
 +hostname <​node>​
 +</​code>​
 +
 +Will parse <​node>​ as the node's hostname. Relations will also be parsed. For more information,​ refer to the [[guides:​reference:​relations:​relations#​parameters|Relations reference]].
 +
 +===== Policy Schedules =====
 +
 +Policy schedules are a way to control the time at which nodes will be checked for compliance. This is especially useful for big machines whose config changes very often and checking after every config change would flood the servers. This functionality will be available in Compliance Phase 2.
 +
 +{{:​menu:​operate:​compliance:​compliance_schedules.png|}}
 +
 +A schedule consists out of a policy, and a time interval. A time interval determines the next interval for this schedule. Whenever a node is scheduled, it calculates the first date in the future that matches this interval. There are four different types:
 +
 +  * **Hour of day**: This schedule will be for every day, at certain hours of the day. You can specify multiple hours (separated by a comma, for example 16,20), and even ranges of hours (separated by a dash, for example 4-6). The first and last checkbox are shortcuts for 0:00 and 23:00 respectively. ​
 +  * **Every x days**: This schedule will take place every x days. Only one number is allowed, and you can set the schedule time for up to the whole hour.
 +  * **Day of week**: This will schedule based on the day of the week, denoted by its two-letter abbreviation:​ mo, tu, we, th, fr, sa and su. Commas to denote multiple days and dashes to denote ranges are also supported. First and last are shortcuts for monday and sunday respectively. You can also set the schedule time for up to the whole hour.
 +  * **Day of month**: This will schedule based on the day of the month, from 1 to 31, where non-existing days like February 30th will be ignored. Commas to denote multiple days and dashes to denote ranges are also supported. First is a shortcut for the first day of the month, and last for the last. You can also set the schedule time for up to the whole hour.
 +
 +A schedule can be enabled or disabled. Only enabled schedules will be used when calculating the next schedule time. If none can be found, the node won't be scheduled and have to wait for another trigger for a compliance check (for example the API, or after a config change if it has been set so). 
 +
 +A schedule can be repeating. Repeating schedules will be scheduled over and over again: when a check finishes, the the next date matching the schedule'​s time interval will be set as the node's next schedule time. If the schedule is not repeating, it will be disabled after the schedule time for its nodes has been set. 
 +===== Command rules =====
 +> NOTE: this feature will be available with the release of Compliance Phase 2
 +
 +Command rules are a rule type that checks the reply of a console command on a node against a number of conditions. ​
 +
 +{{:​menu:​operate:​compliance:​cmd_rule_edit.png|}}
 +
 +The Command field contains the command that will be run on the node. Like all rules, these are limited per vendor type, so in this example, this rule will only be checked for HP C7 nodes. ​
 +
 +Rule start and Rule end can be used to parse part of the reply. Since commands are easier to parse than entire config, these values can be plain text, or contain regular expressions. When using both a rule start and rule end, multiple blocks of text can be matched. Compliance will then be run on all blocks separately, like with configuration rules. Note that whole lines will be matched, so lines that match, but contain some more text will not match. ​
 +
 +For example, a Rule start of:
 +
 +<​code>​
 +Interface
 +</​code>​
 +
 +Will not match:
 +
 +<​code>​
 +Interface interface_1/​1/​1
 +</​code>​
 +
 +However:
 +
 +<​code>​
 +Interface \S+ 
 +</​code>​
 +
 +Will. 
 +
 +Another useful line is:
 +
 +<​code>​
 +^(?![.])
 +</​code>​
 +
 +This matches an empty line. 
 +
 +Command results are fetched along with the NCCM. The nccmd daemon first retrieves the full config of a node, and then evaluates if there are any command rules associated to it (so if it's in a node group that is coupled to a policy that contains command rules). If there are, it executes these, and stores the results in the database. Unlike for the NCCM, there is no history kept for these records. ​
 +
 +When you save a command rule, the nodes that are linked to it are automatically scheduled for a nccm poll, followed by compliance check. It will take the nccmd daemon a few minutes to process this, but the followup process is automated. ​
 +
 +
 +===== Multiconfig compliance =====
 +
 +> NOTE: This functionality was created for the F5 BIGIP vendor module and has been extended to function for other vendor modules as well. The F5 is still a special case however. For these BIGIP configurations some extensive pre-parsing takes place that will remove from the configuration tree all unreferenced segments before comparisons are made. This greatly improves the compliance results given that these configurations can be over 400,000 lines.
 +
 +The Multiconfig compliance is intended for a group of two to four nodes, whose configurations,​ or parts of their configuration need to be equal at all times. ​
 +
 +The Multiconfig uses it own Rule types as they will compare the node's new config to each of the current configs in its node group. A number of restrictions apply:
 +
 +  * The node group cannot contain more than four nodes. The rule will be automatically non-compliant when more than four nodes are detected in the node group. ​
 +  * The node group must contain nodes of the same vendor-type
 +  * It is not advisable for the rule's policy to be run at config change, because this would mean a non-compliance every time one of the nodes changes its config. Instead, we recommend Multiconfig compliance to be run periodically,​ for example daily after office hours. This will be possible with policy schedules, available in Compliance Phase 2.
 +
 +The Rule type has to be set to "​multi-config",​ and its Vendor type to the vendor required. Rules of this type do not have any conditions. If its Rule_start and Rule_end are set, it will only compare the config blocks that match, otherwise it will take the complete configuration. ​
 +
 +In the case of a F5 BIGIP node, all orphans will be logged in the rule's report, to serve as a guide for the operator to help clean them up. 
 +
 +NOTE that a number of blocks are meant to be orphaned. For now these are hard-coded, however if desired we can make this user-generated.
 +
 +
 +===== Nccmd daemon behavior =====
 +
 +This section will give a rough overview of how the nccmd daemon acts. It is a daemon that is responsible for both running the nccm and the compliance. ​
 +
 +It wakes up every five minutes. It then checks if there are any nodes that need to have their nccm checked, and grabs up a sensible number of them. If there are more, these need to wait until the next cycle. A nccm check is necessary if the node in question has sent out a syslog message due to a config change. It can happen because of periodical checks for nccm as dictated by its polling group. Also whenever a node is created or has its name changed, both in the regular yce environment as in the cmdb, it automatically gets scheduled for this as well. 
 +
 +Each node is polled for its configuration and compared with the most recent one in the nccm. In the case of a change they are updated (of course, cyphers and timestamps are filtered out to avoid false positives). If the node has any policies that are set to run compliance upon config changes, these are scheduled for a compliance check for the next cycle of the daemon. ​
 +
 +The daemon also collects the number of nodes that need to be checked for compliance. These processes all run concurrently,​ so it doesn'​t have to wait for every single compliance check to finish before it can get on with actual compliance. A compliance check can be scheduled after nccm, due to a policy schedule, or when a policy gets modified in any way (to check whether its nodes still conform to the new settings).
 +
 +Each combination of node to policy has its own compliance check, and therefore its own report. In the reports form these are bundled per policy or per node. There is no history for past compliance checks: all results you see are from the latest checks, and any previous results are thrown away. The only history is that we keep track of the last time a compliance result has changed for a node, and at what time its last policy check occurred. If a node is removed from a policy through its node group, its compliance results are deleted as well. 
 +
 +Also note that when you update a policy, a flag will automatically be set for the daemon to re-evaluate its compliance for all its nodes. This also happens when you edit (or create/​delete) its rules and conditions, and change its assigned node groups. ​