====== OS-Repository setup ======
The NetYCE OS-Repository is intended to provide a centralized storage and maintenance facility of networking device Operating-system related files.
The repository consists of two parts. First of all the disk-space where the files are stored and monitored is part of the file-transfer directory tree of the application which allows all your devices access using all common protocols (scp, sftp, ftp, tftp). This disk space is shared using Network File System (NFS) among all NetYCE servers (or any Unix based server) to act as a distributed disk-share without duplication of files. Active monitoring of all file activity (adding, updating, moving, deleting) will automatically adjust the OS repository to reflect these changes.
{{:maintenance:general:os_repo_setup.png|}}
Secondly, the user front-end form, which can be found under 'Operate - OS repository', allows the user to organize the image files into file-sets associated with a Vendor and a Device type. Each file-set created is intended to assist the user in finding or selecting the image files needed to a specific device-type and OS version or feature set. As image files can be used for many device-types, the repository keeps track of shared usage and status.
{{:menu:build:os_repo:os_repo.png|}}
Details can be found in the [[menu:build:osversions|OS-Repository form]] page.
Finally, a couple of Scenario commands were added to locate image files in the repository for use in OS upgrade scenarios. These commands will also allow access to the meta-data of each file-set to simplify copy and activation actions.
Details van be found in the [[guides:user:scenario_calls|Retrieving OS-repository data for OS upgrades]] and [[menu:operate:scenarios:commands#os_upgrades|Scenario OS-Repository commands]]
This document details the setup and operation of the OS-Repository.
===== OS repository setup =====
The OS-Repository is enabled and configured from the cli (ssh session) using the **''yce_setup.pl''** tool in interactive mode (not using the ''-r'' option).
When stepping though the various setup sections, the OS-repository section follows the database-server mapping section. It will show the current configuration before prompting to change the settings as illustrated below for two cases.
OS-repository is disabled
Repository clients:
1) genesis (*)
| import-dir
| /var/opt/shared/public/os
No external clients
local server is marked with (*)
Update OS-repository settings? [no]
OS-repository is enabled
Repository host: genesis (*)
| ipv4 | ipv6
| 172.17.0.24 | 3001::24
| export-dir
| /var/opt/shared/public/os
Repository clients:
1) devel7b
| import-dir
| /var/opt/shared/public/os
No external clients
local server is marked with (*)
Update OS-repository settings? [yes]
===== Prerequisites =====
The OS-repository is supported on CentOS or RedHat 7.x
It requires the NFS packages to be installed. Use these or more recent versions:
nfs4-acl-tools-0.3.3-21.el7.x86_64
nfs-utils-1.3.0-0.68.el7.2.x86_64
libnfsidmap-0.25-19.el7.x86_64
For NetYCE servers, the OS-Repository path is fixed: **''/var/opt/shared/public/os''**.
This path is part of the ''/var/opt/shared'' directory tree on which a ''chroot'' is enforced for ssh, scp, sftp, ftp and tftp when using the 'ycicle' user, preventing file access outside this tree.
This setup also enforces that all files are 'yce'-user owned at all times.
===== Example setup =====
In the following example session the initial setup of a shared OS-repo on two NetYCE servers, ''netyceA'' and ''netyceB'', where server ''netyceA'' will be the central OS-repo host exporting it to ''netyceB''.
The OS-repo host should be setup first to activate the NFS export before trying to mount it as an import on other servers.
==== Server 'netyceA' ====
Server netyceA has its OS-repo initially disabled. Enable it and define it as the OS-repo 'host'-server. This means the server will actually have all files stored on its disk and will export the OS-repo directory tree to other registered servers using NFS. There can be only **one** server in a shared OS-repo setup that is the 'host' server, all others are 'client'-servers.
OS-repository is disabled
Repository clients:
1) netyceA (*)
| import-dir
| /var/opt/shared/public/os
2) devel7b
| import-dir
| /var/opt/shared/public/os
No external clients
local server is marked with (*)
Update OS-repository settings? [no] yes
enable shared OS-repository? [no] yes
1) netyceA (*)
2) netyceB
3) external system
select the repository 'host' server: [] 1
'netyceA' (*) will be repository host
'netyceB' will be repository client
add external client(s) to the repository? [no]
no extern clients
Is the OS-repository setup correctly?: [Y]
When continuing the setup script, the NFS export and import setup will be configured and activated.
-- Setup 'netyceA' as OS-repository host server
unmounting /var/opt/shared/public/os
NFS import file '/etc/fstab' is unchanged
updating NFS export file '/etc/exports'
restarting nfs-server
==== Server 'netyceB' ====
OS-repository is disabled
Repository clients:
1) netyceA
| import-dir
| /var/opt/shared/public/os
2) netyceB (*)
| import-dir
| /var/opt/shared/public/os
No external clients
local server is marked with (*)
Update OS-repository settings? [no] yes
enable shared OS-repository? [no] yes
1) netyceA
2) netyceB (*)
3) external system
select the repository 'host' server: [] 1
'netyceA' will be repository host
'netyceB' (*) will be repository client
Is the OS-repository setup correctly?: [Y]
During the activation the NFS configuration files will be modified and the OS-repo is mounted.
-- Setup 'devel7b' as OS-repository client
NFS export file '/etc/exports' is unchanged
'devel7b' uses OS-repository on 'devel7a.left.netyce.org'
updating NFS import file '/etc/fstab'
unmounting /var/opt/shared/public/os
mounting devel7a.left.netyce.org:/var/opt/shared/public/os
no os files found to migrate to OS-repository
===== Migration =====
As the OS-repo directory tree on this client server may already have some os-files installed, these must be moved to the new OS-repo 'host'. This migration or merger of the OS-repo directories is executed automatically when setting up the OS-repo as a client server.
The files found in the OS-repo tree of the client will be copied to the corresponding vendor directory of the 'host' server and removed from the local disk before mounting its exported OS-repo tree.
This migration is implemented by renaming the ''public/os'' directory to ''public/os-local'', then create a new ''public/os'' directory and mounting the 'host' OS-repo tree. If that succeeded, the files are **moved** one-by-one to the OS-repo tree - effectively using the NFS to create a copy on the 'host' server.
Files that already existed on the OS-repo of the 'host' are skipped. They must be manually removed from the ''public/os-local'' tree.
===== External clients =====
As the underlying fabric is NFS-based, any Unix/Linux server supporting nfs4 can partake as a OS-repo client (or host). During the NetYCE setup of the host server, a prompt asks for any external clients. Any number of clients can be added to access the OS-repository.
A sample session where the external client ''unixC'' is added to the OS-repo 'host' to grant access:
OS-repository is enabled
Repository host: netyceA (*)
| ipv4 | ipv6
| 172.17.0.24 | 3001::24
| export-dir
| /var/opt/shared/public/os
Repository clients:
1) netyceB
| import-dir
| /var/opt/shared/public/os
No external clients
local server is marked with (*)
Update OS-repository settings? [yes]
enable shared OS-repository? [yes]
1) netyceA (*)
2) netyceB
3) external system
select the repository 'host' server: [1]
'netyceA' (*) will be repository host
'netyceB' will be repository client
add external client(s) to the repository? [no] yes
external client name/fqdn: [none] ?
The name of the external server entered here will be used as a label to identify
the server's entry but is not relevant otherwise. The NFS mounting permissions will
be granted based on the IPv4 and IPv6 addresses entered following this prompt.
external client name/fqdn: [none] unixC
'unixc' IPv4-address: [none] 172.17.0.200
'unixc' IPv6-address: [none]
external client name/fqdn: [none]
Is the OS-repository setup correctly?: [Y]
The update is shown in the activation:
-- Setup 'devel7a' as OS-repository host server
unmounting /var/opt/shared/public/os
NFS import file '/etc/fstab' is unchanged
updating NFS export file '/etc/exports'
restarting nfs-server
The resulting ''/etc/exports'' file:
#
# Auto-generated /etc/exports by yce_setup
# 2022-04-11 11:52:00
#
# Extern os-repository client: unixc
/var/opt/shared/public/os 172.17.0.200(rw,sync,all_squash,anonuid=1000,anongid=8000)
#
# NetYCE os-repository client: netyceB - netyceb.right.netyce.org
/var/opt/shared/public/os 172.17.0.25(rw,sync,all_squash,anonuid=1000,anongid=8000)
/var/opt/shared/public/os 3001::25(rw,sync,all_squash,anonuid=1000,anongid=8000)
#
The ''yce_setup'' OS-repository configuration summary will now include the external client too:
OS-repository is enabled
Repository host: netyceA (*)
| ipv4 | ipv6
| 172.17.0.24 | 3001::24
| export-dir
| /var/opt/shared/public/os
Repository clients:
1) netyceB
| import-dir
| /var/opt/shared/public/os
External clients:
2) unixC
| ipv4 | ipv6
| 172.17.0.200 |
local server is marked with (*)
Update OS-repository settings? [yes]
===== External host =====
If the OS-repository is not a NetYCE system or a NetYCE server from another NetYCE environment (like separate Production and Test environments), the setup allows for the definition of an external OS-repo NFS host:
OS-repository is disabled
Repository clients:
1) netyceA
| import-dir
| /var/opt/shared/public/os
2) netyceB (*)
| import-dir
| /var/opt/shared/public/os
No external clients
local server is marked with (*)
Update OS-repository settings? [no] yes
enable shared OS-repository? [no] yes
1) netyceA
2) netyceB (*)
3) external system
select the repository 'host' server: [] 3
external repository host fqdn: [] ?
enter the fqdn of the external repository:
external repository host fqdn: [] unixc.netyce.org
external repository directory: [/var/opt/shared/public/os]
Is the OS-repository setup correctly?: [Y]
At activation time the server will try to mount the external NFS OS-repository. If it fails, a notification is shown.
-- 'unixc.netyce.org' is the OS-repository host server
-- Setup 'netyceB' as OS-repository client
NFS export file '/etc/exports' is unchanged
'netyceB' uses OS-repository on 'unixc.netyce.org'
updating NFS import file '/etc/fstab'
unmounting /var/opt/shared/public/os
-- FAILED: OS-repository on 'unixc.netyce.org' is not reachable for NFS
===== Customization ====
==== File types ====
The OS-Repository can be customized to change or extend the different file-types. Using the 'Admin - Setup - Settings' tool for the class 'Translation', the file-types used can be modified.
By default, five file-types are configured for **''Os_file_type''**
^ String-value ^ Num-value ^
| unknown | 0 |
| os-image | 1 |
| boot-image | 2 |
| license | 3 |
| other | 4 |
The administration uses the numeric values, the string values are the translation for the user. Additional entries with their own number can be added or existing ones renamed. Note however, that the 'os-image'/'1' entry is the default file-type to pick when none is specified.
It is possible to alter the values for the ''Os_file_status'' and ''Os_image_status'', but is not recommended as these numeric values are used in the behaviour of the front-end.
==== Device types ====
Device types are used to identify the hardware to which an OS-image set applies. Vendors traditionally use many, many different formats to identify its hardware. Within a product-family hundreds of different names are in circulation, often changing when different versions of the OS are loaded.
Using this hardware identifier without modification in the OS-repository will easily result in a chaotic number of OS-image sets.
Therefore we have opted to provide a means to moderate the available Device types. You can either lock the Device types or keep them open. When locked, only the 'controlled' types can be selected in the front-end. When unlocked, the front end will still show the existing device types, but allows an operator to add their own which will then become part of the available Device types.
This behaviour is controlled using the Tweak **''Allow_custom_device_types''** which can be modified using the 'Admin - Setup - Settings' tool. When the num value is set to '1', users are allowed to add custom device types for the OS repository. When set to 0, they can only use values that already exist.
The Device types available, moderated or not, are maintained in their own database table, ''YCE.Os_device_types''. It van be viewed and modified using the 'Admin - Custom data' tool.
An initial set of Device types was extracted from the available information already available in the YCE and CMDB databases, mostly relying on the ''Node_model'' information extracted during communication with the device. Similar node models are combined to result in a single Device type.
This initialization and extraction process can be repeated if desired by running the appropriate installation patch in force mode:
go patches
patch_install.pl -p 21102602 -f
Result
Force mode enabled
[21102602] Re-applying YCE patch '21102602'
[21102602] Inserted '131' entries for '19' vendors in YCE.Os_device_types
[21102602] Done