Differences

This shows you the differences between two versions of the page.

--- maintenance:general:tools:net_setup.pl [2020/11/23 16:05] – [Capabilities] yspeerte
+++ maintenance:general:tools:net_setup.pl [2024/07/03 12:31] (current) – external edit 127.0.0.1
@@ Line 1: / Line 1: @@
+{{indexmenu_n>3}}
+====== net_setup.pl ======
+The ''**net_setup.pl**'' script, located in ''/opt/yce/system/'', is used for setting up the networking environment of the CentOS/Redhat Linux system for NetYCE. It is used at the initial setup of the system or anytime a change to the networking environment is needed.
+The intention of the script is to make the networking setup as smooth as possible, limiting the possibility of errors by manipulating the configurations by hand. It will not allow networking configurations that are not directly supported by the NetYCE appliance.
+The net_setup script will not just setup the networking, it also creates and maintains a NetYCE networking configuration file that is used to configure NetYCE various components and daemons. It gathers information from the system and from the user to write the settings to the system and to setting files of netYCE.
+==== Capabilities ====
+The ''net_setup.pl'' script is intended to configure the networking of CentOS/Redhat 6.x and CentOS/Redhat 7.x systems for use in a NetYCE environment.
+The net_setup supports various settings that can be categorized covering these topics and capabilities:
+  * Setting 'root' and 'yce' user passwords
+  * Hostname and domain name changes
+  * Multiple ethernet interfaces
+  * IPv4 supported on CentOS 6.x and CentOS/Redhat 7.x
+  * IPv6 supported on CentOS/Redhat 7.x if enabled in kernel
+  * IPv4 + IPv6 dual-stack and IPv6-only configurations
+  * Configure secondary addresses (IPv4 and IPv6)
+  * DNS server configuration (IPv4 and IPv6)
+  * Static (fixed) or DHCP (autoconf) ip-addresses
+  * Automatic detection and NetYCE reconfiguration on DHCP ip-address change (IPv4 and IPv6)
+  * Default gateway interface assignment and routing configuration
+  * NetYCE user and devices interface identification
+  * External NAT address assignment for Network devices
+  * NTP server assignment (date and time sync)
+<color red>NOTES</color>:
+> The net_setup script needs to be started as 'root' in order to activate its changes
+> In the dialog with net_setup the (default) values and the user entries are shown using the color '<color green>green</color>' for easy legibility. This colour provides a good contrast on both white and black terminal backgrounds.
+> Default values are shown between square brackets (''['' and '']''). An <enter> suffices to accept de default value. The use of these defaults is to permit the user to enter as few as possible values and re-use existing values where available.
+> At each prompt a help message is available by entering the ''**?**'' as value.
+==== 'net_setup.conf' ====
+The configured and collected networking data net_setup uses is written to: ''/opt/yce/etc/net_setup.conf''. This configuration file is read by the NetYCE setup script ''**yce_setup.pl**'' which generates the dependent configuration files (e.g. for httpd, vsftpd, mysql, mojo, psmon, etc) and restarts their processes.
+The net_setup script requires execution as the 'root' superuser because it will change networking files and activates the settings. When no root privileges are detected, a prompt allows a user to continue, but no changes can be activated. Changes are optionally saved in the ''net_setup.conf'' but are not activated.
+==== Invocation ====
+Execution of net_setup.pl is preferably done using a server console session. The activation of new network settings could result in a lost session resulting in an incomplete setup, or should the new network settings result in an unreachable server the console is the only means to correct it anyway. See the section on [[maintenance:general:tools:net_setup.pl#Network activation|Network activation]].
+Therefore the net_setup.pl script is part of the 'root' login sequence (see below). Manual execution of the net_setup.pl script requires 'root' privileges:
+<code>
+-- as root:
+# /opt/yce/system/net_setup.pl
+-- as yce:
+$ sudo /opt/yce/system/net_setup.pl
+</code>
+==== 'root' login ====
+The net_setup script is part of the 'root' login procedure to remind the user to make networking changes to the NetYCE system on initial installation. The root user is then presented with a 5 second countdown to start the net_setup process. If the user hits <enter> during the countdown the net_setup will start prompting the user, otherwise the net_setup will end.
+<code>
+-- NetYCE Networking setup
+   Hit enter within 5 seconds to start setup .....
+</code>
+<code>
+-- NetYCE Networking setup
+   Hit enter within 5 seconds to start setup
+-- Timeout, skipping setup
+</code>
+==== Setting passwords ====
+Because of its use at the initial 'root' login, net_setup will start prompting for the 'root' and 'yce' passwords.
+<code>
+-- NetYCE Networking setup
+   Hit enter within 5 seconds to start setup ..
+NOTE:
+  When prompted for input help on the question is available by entering '?'.
+  Incorrect responses result in a message on the expect input.
+  Just hitting <enter> will accept the existing or default value '[...]'.
+  The proces can be aborted at any prompt by entering 'quit'.
+   good, root privileges apply
+-- System release
+   identified CentOS - 7.9.2009
+   using setup for Redhat V7
+-- Read Network setup: '/opt/yce/etc/net_setup.conf'
+-- Read NetYCE setup: '/opt/yce/etc/yce_setup.conf'
+-- Setup passwords
+  For the first-time setup it is mandatory to set the 'root' password. You are
+  prompted now to enter the desired root password twice. This will then be the
+  active 'root' password.
+     enter 'root' password:                   ********
+     verify 'root' password:                  ********
+     password done
+  For the first-time setup it is mandatory to set the 'yce' password. You are
+  prompted now to enter the desired yce password twice. This will then be the
+  active 'yce' password.
+     enter 'yce' password:                    ********
+     verify 'yce' password:                   ********
+     password done
+</code>
+Once the passwords are set, the forced password prompts will be replaced for optional password setting prompts.
+<code>
+-- Setup passwords
+   Set the 'root' password?                   [no]
+   Set the 'yce' password?                    [no]
+</code>
+==== Change hostname ====
+The next prompt relates to changing the hostname and the domain of the server. The hostname change will be activated at the same time as the network changes are activated.
+<code>
+-- Setup hostname
+   Full qualified name is 'genesis.netyce.org'
+   Is this full name correct?                 [yes] ?
+  The displayed full-qualified-domain-name should match the hostname and domain
+  of this server and must be unique. Type 'yes' to confirm it is correct or
+  'no' to be prompted for a new hostname and domain.
+   Is this full name correct?                 [yes] no
+     Hostname?                                [genesis] netyce01
+     DNS domain?                              [acme.org]
+   Full qualified name is 'genesis.netyce.org'
+   Is this full name correct?                 [yes]
+   name change: 'genesis.netyce.org' -> 'netyce01.acme.org'
+   Save this configuration?                   [yes]
+   update yce_setup: 'genesis.netyce.org' -> 'netyce01.acme.org'
+</code>
+==== Interface configuration ====
+Prior to prompting for the interface configuration settings, the existing - operational - interface settings are read from the system and presented in a concise table per ethernet interface. Non-ethernet interfaces are ignored.
+The example below shows this interface summary of a system deploying two interfaces, one using fixed IP-addresses, the other DHCP. The image below is shows the use of the colour 'green' for all values as the user would experience it.
+{{:maintenance:general:tools:current_interface_setup.png?500|}}
+When selecting the default [yes], the user enters a set of dialog prompts for the first interface. When those are done confirmation is requested if the entries are correct and the dialog moves to the next interface. Should mistakes have been mode, the same interface is re-prompted.
+The first prompt determines the basic way the interface will be setup:
+<code>
+   Update networking?                         [yes]
+-- Setup interface 'enp0s17'
+   enp0s17 use 'static' ip, 'dhcp' or 'none'? [static] ?
+  Each ethernet interface can use a configuration method that is either
+  'static' (a fixed address), 'dhcp' (a dynamic address), or if
+  not used: 'none'. If IPv6 Autoconf is to be used, choose 'dhcp'.
+   enp0s17 use 'static' ip, 'dhcp' or 'none'? [static]
+</code>
+When the interface is not to be disabled ('none'), the dialog for 'static' and 'dhcp' will prompt for subsequent values for its IPv4 and IPv6 setup. The 'static' variant will include prompts for IP-addresses.
+A sample session where the static IPv4-address is changed. Note that the gateway address is automatically calculated from the prefix.
+<code>
+-- Setup interface 'enp0s17'
+   enp0s17 use 'static' ip, 'dhcp' or 'none'? [static] static
+   enp0s17 = FIXED-IP
+   enp0s17 enable IPv4?                       [yes]
+     enp0s17 IPv4-address?                    [172.17.10.25] 192.168.2.141
+     enp0s17 IPv4-prefix?                     [24] 25
+     enp0s17 gateway address?                 [192.168.2.129] ?
+  Using the assigned ip-address and the prefix the network-address is
+  determined. The first address of the network-address is usually the gateway
+  address used, although any address in the subnet may be used. The default
+  is calculated as indicated. Type 'none' if no gateway address is to be
+  assigned (not recommended).
+     enp0s17 gateway address?                 [192.168.2.129]
+     enp0s17 add IPv4 secondary addresses?    [no]
+   enp0s17 enable IPv6?                       [yes]
+</code>
+The dialog continues for the IPv6 setup and concludes with the DNS server addresses that will be used.
+<code>
+   enp0s17 enable IPv6?                       [yes]
+     enp0s17 IPv6-address?                    [3001::25]
+     enp0s17 IPv6-prefix?                     [64]
+     enp0s17 gateway address?                 [3001::1]
+     enp0s17 add IPv6 secondary addresses?    [no]
+   enp0s17 primary-DNS address?               [2001:4860:4860::8844]
+   enp0s17 secondary-DNS address?             [8.8.8.8]
+   enp0s17 is this setup correct?             [yes]
+</code>
+The DNS servers may use IPv4 and IPv6 addresses, but when completed a validation will check if the DNS addresses can be used by the IP-versions used.
+The dialog for DHCP setup is more limited. It is not possible to setup an interface where IPv4 is static and IPv6 uses dhcp or vice versa. And, although dual-stack is currently quite normal, IPv6-only configurations are supported.
+<code>
+-- Setup interface 'enp0s8'
+   enp0s8 use 'static' ip, 'dhcp' or 'none'?  [dhcp] dhcp
+   enp0s8 = DHCP
+   enp0s8 enable IPv4?                        [yes]
+   enp0s8 enable IPv6?                        [yes]
+     enp0s8 include IPv6 autoconf?            [yes] ?
+  When using IPv6 Autoconf an IPv6 address will be generated using the Router Advertisement (RA).
+     enp0s8 include IPv6 autoconf?            [yes]
+   enp0s8 primary DNS-server address (override dhcp)? [2001:4860:4860::8844]
+   enp0s8 secondary DNS-server address (override dhcp)? [8.8.4.4] ?
+  The DHCP server usually configures the DNS servers the system will use. The
+  optional IPv4/v6-address entered here will override the DNS address provided
+  by the DHCP. Type 'none' to accept the DHCP provided DNS.
+   enp0s8 secondary DNS-server address (override dhcp)? [8.8.4.4] none
+   enp0s8 is this setup correct?              [yes]
+</code>
+When the interface dialogs are completed, the updated interface setup summary is displayed again along with a prompt to save it in the net_update.conf settings file.
+==== Interface roles ====
+NetYCE can assign different roles to the various interfaces. If only one (ethernet) interface is present all roles are automatically assigned to the one interface and the dialog is skipped.
+{{:maintenance:general:tools:current_interface_roles.png?500|}}
+Three roles must currently assigned to the available ethernet interfaces. First there is the (default) "**Gateway**". When using multiple interfaces, only one interface will be the default interface where all traffic not explicitly routed will be forwarded, usually the 'outside world', the 'internet' or the 'corporate network'.
+The remaining roles are NetYCE specific. The "**Users**" interface identifies where the user requests are received from. This is also the interface used to for incoming API requests, the replicating databases communicate over and could be considered the NetYCE 'application interface.
+The final role defines the interface to communicate with the network devices. For the incoming connections from the network devices, optional NAT addresses can be configured for IPv4 and IPv6. Most file transfers between NetYCE servers and the network devices must originate from the device and must be able to connect to the server using its address. When there is an address translation service (NAT) active, the devices must use the external addresses instead. These addresses are configured here when needed.
+<code>
+-- Setup default-gateway interface
+) enp0s17
+) enp0s8
+   default-gateway interface (ipv4 and ipv6)? [2]
+   enp0s8 is default-gateway interface
+-- Setup user front-end interface
+) enp0s17
+) enp0s8
+   users interface (ipv4 and ipv6)?           [1]
+   enp0s17 is users interface
+-- Setup network-devices interface
+) enp0s17
+) enp0s8
+   devices interface (ipv4 and ipv6)?         [1]
+   enp0s17 is devices interface
+   enp0s17 optional device IPv4 transfer (NAT) address? [none] ?
+  File transfers between NetYCE and devices require a connection to be
+  initiated from the device to NetYCE. If the NATted address address the
+  devices must use is different from the ip-address of the interface,
+  specify it here. Otherwise use 'none'
+   enp0s17 optional device IPv4 transfer (NAT) address? [none]
+   enp0s17 optional device IPv6 transfer (NAT) address? [none]
+</code>
+The above session example shows a two-interface setup where one interface (enp0s17) uses fixed ip-addresses for both the user and the device communication, and a second interface (enp0s8) that uses dynamic DHCP addresses to communicate with the outside world and the internet. The latter interface is also the default gateway
+An additional prompt to define NTP source for the systems real-time-clock is also included here. Optionally the 'ntpdate' utility can be set up to synchronize the clock every 10 minutes with an external NTP source. When enabled this source is prompted for.
+<code>
+-- Setup clock-synchronization
+   Use 'ntpdate' to periodically set date and time? [yes]
+     ntp server address?                      [pool.ntp.org]
+</code>
+When satisfied confirm on the 'save' prompt.
+The networking setup then continues to activate this configuration. Should the net_setup not have been invoked as root or with sudo, the network activation is skipped and only the NetYCE re-configuration is executed.
+==== Network activation ====
+After confirming the updated configuration, the network settings must be updated. First several networking configuration files are created or updated, including those setting the hostname and dns resolving.
+<code>
+-- Generating network configuration files
+-- Updating interface configurations
+   generate config '/etc/sysconfig/network-scripts/ifcfg-enp0s17'
+   generate config '/etc/sysconfig/network-scripts/ifcfg-enp0s8'
+-- Updating network config file
+   generate network config '/etc/sysconfig/network'
+-- Updating dns resolv config
+   generate dns config '/etc/resolv.conf'
+-- Updating hostname config
+   generate hostname file '/etc/hostname'
+   setting hostname
+   generate hosts file '/etc/hosts'
+   no hostname change: 'devel7a.netyce.org'
+   update yce_setup: 'devel7a.netyce.org' -> 'devel7a.netyce.org'
+-- Installing 'ntpdate' in crontab
+-- Activating network
+   Restart networking service?                [yes]
+</code>
+Then the network must be restarted. As a network restart could very well affect the existing network sessions (due to new address or routing), the user could find his session terminated at this point.
+Although activation could have continued, the user will be unable to observe it and will timeout after a few minutes. If the session was not impacted, the user will be able to monitor the activation and the subsequent re-configuration of the NetYCE application.
+<code>
+-- Activating network
+   Restart networking service?                [yes]
+   network restart (wait...)
+   completed
+   no hostname change: 'devel7a.netyce.org'
+   update yce_setup: 'devel7a.netyce.org' -> 'devel7a.netyce.org'
+Active roles:
+Gateway
+ | Interface         | Boot
+ |  enp0s8           |  DHCP
+ | IPv4              | IPv6
+ |  172.15.10.1      |  fe80::5054:ff:fe12:3500
+Users
+ | Interface         | Boot
+ |  enp0s17          |  STATIC
+ | IPv4              | IPv6
+ |  172.17.10.24     |  3001::24
+Devices
+ | Interface         | Boot
+ |  enp0s17          |  STATIC
+ | IPv4              | IPv6
+ |  172.17.10.24     |  3001::24
+ | IPv4 nat          | IPv6 nat
+ |  none             |  none
+</code>
+Because of this ambiguous behaviour it is advised to execute potentially session disrupting network modifications using the //server console//. This is also good practice as incorrect network settings might result in an unreachable server. The console is then the only means to correct the network settings.
+If the session-lost issue was experienced it is recommended to re-execute the net_setup after a new communication session was established. Because the basic network is now setup properly, the entire setup procedure can be executed without interruption. Just confirm the prompts which should all reflect the earlier choices, and the system will re-activate correspondingly.
+==== NetYCE re-activation ====
+When the net_setup completes it will automatically execute the ''**yce_setup.pl -r**'' command to update the configurations of the various NetYCE components and restart its daemons. This command will use the NetYCE configuration found in ''/opt/yce/etc/yce_setup.conf'' to quickly re-generate (using the ''-r'' option) and activate this configuration using the new network settings.
+When setting up a new NetYCE server, the NetYCE configuration will use defaults which will still require manual session to properly setup the application.
+Please refer to the article on the [[maintenance:general:tools:yce_setup.pl|yce_setup.pl]] tool for details on its use.
+Below example output of a yce_setup session using the regenerate (''-r'') option. This configuration uses two NetYCE servers using master/master database replication supporting IPv4 and IPv6.
+<code>
+-- Generating and activating NetYCE
+-- ----------------------------------------
+-- Starting 'yce_setup' regenerate
+-- System release
+   identified CentOS - 7.9.2009
+   using setup for Redhat V7
+-- Connected to database at '172.17.10.24' using version '10.2.36-MariaDB-log'
+Current setup:
+devel7a.netyce.org (*)
+  | IP-address  | IPv4             | IPv6
+  |  users      |  172.17.10.24    |  3001::24
+  | Database    | Primary          | Secondary
+  |  id=1       |  devel7a (*)     |  devel7b
+devel7b.netyce.org
+  | IP-address  | IPv4             | IPv6
+  |  users      |  172.17.10.25    |  3001::25
+  | Database    | Primary          | Secondary
+  |  id=2       |  devel7b         |  devel7a (*)
+  local server is marked with (*)
+-- Create configs for server 'devel7a'
+-- Yce: /opt/yce/etc/devel7a_yce.conf
+-- Retrieving file-transfer configurations...
+     can support 'sftp'
+     can support 'scp'
+     can support 'ftp'
+     can support 'tftp'
+-- Mojo: /opt/yce/htdocs/angular/app/host.js
+     mojo url set to 'https://devel7a.netyce.org:8080/'
+     wiki url set to 'http://wiki.netyce.com/'
+-- Yce_psmon: /opt/yce/etc/devel7a_psmon.conf
+-- Crontab: /opt/yce/etc/devel7a_crontab.conf
+-- Httpd: /opt/yce/etc/devel7a_httpd.conf
+-- Mysql: /opt/yce/etc/devel7a_mysql.conf
+     mysql version is '10.2.36'
+     mysql key_buffer set to '376M'
+     mysql tmpdir set to '/var/tmp'
+-- Updating 'devel7a' menu-tree (C)
+     Creating menus for the role(s): "frontend","database"
+     Renewed the menu tree using the default
+     Updating 'devel7a' encryption keys
+     Updating scenario syntax highlighting file
+-- Renewing NMS table permissions
+-- Checking database replication
+     replication local: 172.17.10.24, remote: 172.17.10.25
+-- Updating config-sync setup
+     located '55' config-files in '6' groups
+     updated config_sync.conf has '28' entries
+-- Relaunching NetYCE daemons...
+-- yce_psmon: 18813
+     stop: /bin/sudo /usr/bin/systemctl stop yce_psmon.service
+     wait stop 'yce_psmon':
+     start: /bin/sudo /usr/bin/systemctl start yce_psmon.service
+     wait start 'yce_psmon': 29470
+-- yce_netmon: 20081
+     stop: /opt/yce/system/init/yce_netmon stop
+     wait stop 'yce_netmon':
+     start: /opt/yce/system/init/yce_netmon start
+     wait start 'yce_netmon': 29550
+-- yce_cramer:
+     # ignored: /opt/yce/etc/ignore_cramer
+-- yce_tftpd: 18933
+     stop: /bin/sudo /opt/yce/system/init/yce_tftpd stop
+     wait stop 'yce_tftpd':
+     start: /bin/sudo /opt/yce/system/init/yce_tftpd start
+     wait start 'yce_tftpd': 29594
+-- yce_skulker:
+     # ignored: /opt/yce/etc/ignore_skulker
+-- yce_sched: 18956
+     stop: /opt/yce/system/init/yce_sched stop
+     wait stop 'yce_sched':
+     start: /opt/yce/system/init/yce_sched start
+     wait start 'yce_sched': 29617
+-- yce_nccmd: 18976
+     stop: /opt/yce/system/init/yce_nccmd stop
+     wait stop 'yce_nccmd': 18976
+     wait stop 'yce_nccmd':
+     start: /opt/yce/system/init/yce_nccmd start
+     wait start 'yce_nccmd': 29640
+-- yce_ibd:
+     # disabled
+-- morbo:
+     # disabled
+-- mojo: 18985 25379 26425 28340 28564 28565 28566
+     mojo hot-deploy on pid 18985
+     running 'mojo': 18985 25379 26425 28340 28564 28565 28566
+-- yce_xch: 19034
+     stop: /opt/yce/system/init/yce_xch stop
+     wait stop 'yce_xch':
+     start: /opt/yce/system/init/yce_xch start
+     wait start 'yce_xch': 29703
+-- Completed
+</code>
+==== Dynamic (DHCP) addresses ====
+The NetYCE application supports dynamic IP-addresses using DHCP (and autoconf for IPv6 too). This implies that the various NetYCE components must dynamically be re-configured when an IP-address is allocated or changed.
+If the **net_setup** has interfaces configured for DHCP, the **yce_setup** will automatically configure and launch the ''**yce_netmon**'' daemon. This background process will monitor all network-address changes (of the interfaces in net_setup) and modify the net_setup and yce_setup configuration files accordingly. It then relaunches the ''**yce_setup.pl -r**'' to re-activate the NetYCE components.
+If the DHCP server is slow to issue an IP-address, the NetYCE application might take several minutes to properly activate itself. Should the DHCP server fail to issue an address, the NetYCE application will not function.
+//