|
principles projects CNMI-Guam old projects |
HFDR /
Installation of Ubuntu 22.04 LTS 64-bit on Generic HFDR field machinesThese master instructions are for installing a system from scratch using the standard ubuntu 22.04 distribution. To assist in radar configuration, we have also prepared a custom iso image, which contains all the configurations required, using the Cubic toolbox. We recommend to always perform installation through this customized image, follwing these simplified instructions. We also recommend installing Cubic on the lab reference machine for creating and saving additional customization. CAUTION! with 22.04, Canonical has changed the management of several packages including firefox, using snap instead of apt. This creates problems because snap makes updates automatically in the background, which is not desired on a remote site which has limited network bandwidth. After installing linux, please disable snap upgrades: snap refresh --hold These master instructions are kept up-to-date. The
instructions are systematic and must be done in order. It is recommended
to copy this wiki page to a new wiki page Site.Installsteps and to
In the instructions below, root# indicates a root prompt and a command to be executed as root, and ruser$ indicate a user prompt and a command to be executed as ruser, R(adar) USER. Pre-formatted text between horizontal lines should be copied verbatim into configuration files. Hardware should consist of a low power i5 PC with at least four cores and 4 Mb cache, 32 Gb memory, 256 Gb SSD system drive and 2 Tb HDD data drive, 1 Gbs ethernet (to switch), USB-3 (for backup disks). We recommend the NUC-13 with i5 processor as optimum. The i7 dissipates too much heat, and the i3 is too slow. A 1Gbit/s ethernet switch is used to interconnect PC, Dtacq, Pi, through the LAN port of a router/gateway to the outside internet. The router should be configured to isolate the local network from the outside world and should have dhcp enabled. If router does not have dhcp, the PC can be configured as a dhcp server. Gateway is assumed to be 192.168.1.254 as seen from the internal private network relayed by the switch. If another private net is used, 192.168.1 has to be updated in all the instructions. 0. First, configure BIOS:
A USB key connected to a USB-3 port (blue) may not be recognized at boot, always use USB-2 port (black). 1. Prepare a USB key with Ubuntu 22.04.1 LTS 64-bit desktop version; from an ubuntu laptop:
2. Open terminal window and
type 3. Select icon "Install Ubuntu 22.04". Choose "Normal installation", "Install 3rd party software", then "Erase disk and install ubuntu". Verify that the SSD disk is partitioned with a small UEFI/fat32 boot partition, and a single ext4 partition: Device Start End Sectors Size Type /dev/sda1 2048 1050623 1048576 512M EFI System /dev/sda2 1050624 488396799 487346176 232.4G Linux filesystem 4. Define user ruser (1000) (group ruser(1000) will be automatically created) and set password (defaut ruser), enable Administrator and AutomaticLogin. 5. Define computer name as radlab-xxxx, the 4 letters of the site name (four letters are preferred by the international network). Some processing scripts assumes that the site name is exactly radlab- followed by 4 letters and may break if another number of characters is used. 6. Set time zone to any US time zone. Do not use another country time zone even if you set up from another country, because this will alter the date format, and several scripts may break down. 7. Computer will ask to restart when installation is complete; remove the USB key; reboot into the new installation and login as ruser. 8. Setup access to the Internet. If using dhcp from the router/gateway, the IP number of the name resolver is provided automatically. If this fails, temporary stop gap for name resolving is to root# vi /etc/resolv.conf and to add the line nameserver 8.8.8.8 then ping www.google.com to verify that name serving works. 9. Install commonly needed packages: Copy and paste in a terminal window: add- apt-repository main add- apt-repository universe add- apt-repository multiverse add- apt-repository restricted apt -y update apt -y full-upgrade apt -y install apache2 apt -y install build-essential apt -y install bvi apt -y install chromium-browser apt -y install csh apt -y install default-jre apt -y install default-jdk apt -y install gifsicle apt -y install gimp apt -y install gnome-session-flashback apt -y install gpart apt -y install gparted wget http://archive.ubuntu.com/ubuntu/pool/universe/h/hddtemp/hddtemp_0.3-beta15-53_amd64.deb apt -y install ./hddtemp_0.3-beta15-53_amd64.deb chmod 6711 /usr/sbin/hddtemp apt -y install imagemagick apt -y install isc-dhcp-server apt -y install jhead apt -y install libio-socket-inet6-perl apt -y install libio-socket-ssl-perl apt -y install libncurses5 apt -y install libnet-ssleay-perl apt -y install lm-sensors apt -y install plocate apt -y install mailutils apt -y install mlocate apt -y install netpbm apt -y install net-tools apt -y install nfs-kernel-server apt -y install nmap apt -y install ntp apt -y install openssh-server apt -y install openjdk-8-jre apt -y install openjdk-8-jdk /bin/rm /etc/alternatives/java ln -s /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java /etc/alternatives apt -y install php libapache2-mod-php php-mysql apt -y install pv apt -y install python3-pip netcdf-bin python3-netcdf4 apt -y install libxcb-xinerama0 apt -y install resolvconf apt -y install screen apt -y install setserial apt -y install sshpass apt -y install smartmontools apt -y install tcsh apt -y install traceroute apt -y install vim apt -y purge modemmanager apt -y purge mysql-server apt -y full-upgrade apt -y autoremove systemctl stop unattended-upgrades.service systemctl disable unattended-upgrades.service apt remove -y unattended-upgrades snap refresh --hold Alternately, download and execute the script root# wget https://www.oceanphysics.com/support/Documents/linux/post_install_22.04.sh root# sh post_install_22.04 For the postfix configuration (TAB), choose "local mail only" to receive crontab notifications. Reboot, login as ruser, open terminal window and sudo su. 10. Add local hosts: root# vi /etc/hosts add at the start of the file after the 127 hosts: # nuc (edit as locally configured) 192.168.1.20 # dtacq (edit as locally configured) 192.168.1.12 dtacq dtacq1 dtacq-master 192.168.1.13 dtacq2 dtacq-slave dtacq-slave1 192.168.1.14 dtacq3 dtacq-slave2 192.168.1.15 dtacq4 dtacq-slave3 # pi (edit as locally configured) 192.168.1.50 pi 11. Set the proper time/clock and locale: root# dpkg-reconfigure tzdata and set timezone to "None of above" then "UTC" through the menus (use TAB then ENTER to go through the menus); this will ensure that the crontabs run on UTC, not local time. It is important to do this now to avoid irreparable confusion in the data files time-tags. If you installed the PC being outside the US, the installer may have outsmarted you by setting the default language based on where your IP is located, and you may see day/month in tagalog, mandarin, spanish or french (this affects date, and any script that would use date, which are many). root# vi /etc/default/locale and change back all entries to "en_US.UTF-8", deleting any other language. In some instances it may be necessary to run root# dpkg-reconfigure locales unselecting any other language, and to add LANG=en_US.UTF-8 to /etc/environment. Correcting a wrongly set locale can be frustrating. 12. Install python: ruser$ pip3 install numpy spyder scipy netCDF4 matplotlib If you have a matlab license, install the matlab package now. 13. Configure apache2: root# vi /etc/apache2/envvars export APACHE_RUN_USER=ruser
export APACHE_RUN_GROUP=ruser
root# chown -R ruser.ruser /var/lock/apache2 /var/www root# /etc/init.d/apache2 restart root# a2enmod include root# a2enmod userdir root# a2enmod cgid root# service apache2 restart root# vi /etc/apache2/mods-available/userdir.conf add new stanza: <Directory /home/*/public_html/cgi-bin> Options +ExecCGI SetHandler cgi-script </Directory> Open a browser, type localhost in the address bar and make sure apache2 answers. The page should say "Apache2 Ubuntu Default Page. It works!" To allow remote inspection of the data acquired through the web server : root# su ruser ruser$ cd /var/www/html ruser$ ln -s /home/ruser/data . 14. Configure interfaces and check connectivity: Under 22.04 on a NUC, we use a single 1Gbit/s ethernet interface. NUC, Rasberry-pi, dtacq, data loggers, are all connected part of the same subnet and interconnected through the LAN port of a router/gateway to outside internet or bridge ubiquity. To find what is the RJ-45 interface name: root# lshw -class network and look for the logical name of the interface which is described as "Ethernet interface" (on the above NUC model, it comes back as eno1). In 22.04, the graphical NetworkManager is a valid method to setup the network. It is best to setup as a dhcp client, and to fix the IP served in the router as 192.168.1.20, based on the MAC address of the PC recorded by the router. Alternately, using the graphical NetworkManager, setup Ethernet interface for eno1, IPv4 settings: Manual, Address 192.168.1.20, Netmask 255.255.255.0, Gateway 192.168.1.254, leaving DNS on Auto. Save, reboot, check that DNS services are available. If not, go back in the graphical network interfaces, and set fixed DNS server as 8.8.8.8, or local ones (from router) if you know them. 15. Enable firewall:
# enable UFW ufw enable # allow all outgoing ufw default allow outgoing # disallow all incoming ufw default deny incoming on eno1 # Allow all from local net (pi, dtacq, ...) ufw allow from 192.168.1.0/24 # Add your external servers as needed ufw allow from xxx.xxx.xxx.xxx port 22 # UFW Status ufw status verbose
16. Listen to outside network time servers:
root# vi /etc/ntp.conf
# minimal /etc/ntp.conf driftfile /var/lib/ntp/ntp.drift broadcast 192.168.1.255 leapfile /usr/share/zoneinfo/leap-seconds.list statsdir /var/log/ntpstats/ statistics loopstats peerstats clockstats filegen loopstats file loopstats type day enable filegen peerstats file peerstats type day enable filegen clockstats file clockstats type day enable pool 0.ubuntu.pool.ntp.org iburst pool 1.ubuntu.pool.ntp.org iburst pool 2.ubuntu.pool.ntp.org iburst pool 3.ubuntu.pool.ntp.org iburst
root# service ntp restart
root# ntptrace
localhost: stratum 2, offset 0.000139, synch distance 0.004383
17. Configure ruser account: To install pre-configured ruser home directory: ruser$ sudo su root# cd /home root# wget https://www.oceanphysics.com/support/Documents/linux/ruser_homedir_22.04.tgz root# tar -xzf ruser_homedir_22.04.tgz root# rm ruser_homedir_22.04.tgz
# m h dom mon dow command # launch acquisition, specify number of seconds as parameter #0,30 * * * * /home/ruser/bin/radar_control 1740 >> /home/ruser/logs/radar_control.log 2>&1 # log of temperatures 0,10,20,30,40,50 * * * * /home/ruser/bin/gettemps # log of dtacq voltages 0 * * * * /home/ruser/bin/getvolts # launch mkdaydir 0 14 * * * /home/ruser/bin/mvdaydir 18. Enable password-less login: To allow pushing the data for automated upload from this NUC machine to target hosts, copy/paste file ~/.ssh/id_rsa.pub (pre-configured with the ruser account, see below) to ~/.ssh/authorized_keys on each target machine. Verify that the password-less login works by doing ruser$ ssh remote_machine and adding key to known_hosts. If port 22 of this machine is directly accessible from the Internet, enable password-less login from a remote host for pulling the data, create a file id_rsa.pub on the remote host, then copy/paste ~/.ssh/id_rsa.pub from the remote host to ~/.ssh/authorized_keys on this machine. To ease maintenance, password-less access from ruser@www.oceanphysics.com should at least be enabled. To disable annoying messages when you ssh into a machine, create an empty file in the home directory with ruser$ touch .hushlogin Remarks: 1. password-less login may fail if the permissions are not the following ruser:~/.ssh$ ls -l -rw------- 1 ruser ruser 558 Feb 23 23:53 authorized_keys -rw------- 1 ruser ruser 2602 Feb 23 23:53 id_rsa -rw-r--r-- 1 ruser ruser 558 Feb 23 23:53 id_rsa.pub -rw------- 1 ruser ruser 2220 Feb 23 23:56 known_hosts 2. the pre-configured ruser account contains a generic set of keys, which allow password-less login across all similarly configured radars. Unless a higher security is required, it is recommended to leave this configuration intact. 19. Configure the D-Tacq Follow the separate instructions. 20. Setting up CS-studio on the NUC: Follow the separate instructions, step 5. 21. Format archive disk We prefer 2.5" USB-3 portable drives (Seagate, Western Digital) as the primary archival medium. At 4 Tb, they will hold about 12 years of raw decimated data at 750 MB/day for 8 antennas at 8 MHz, and 6 years at 1.5 GB/day for 8 antenna at 27 MHz. The factory formatting as NTFS is fraught with problems under Linux (erroneous treatment of symbolic links in particular). new disks MUST always be reformatted as follows: root# sudo gparted
root# cd /media/ruser/HFDRDATA root# mkdir data root# mkdir data/xxxx root# chown -R ruser.ruser . 22. Cloning system disk We recommend doing a verbatim clone of the system disk, and keeping it offline. Because a clone has exactly the same UUIDs as the original, the cloning can only be done when booted from a life USB key.
dd if=/dev/sda of=/dev/sdb bs=17179869184
23. Configure DHCPD (optional) Only if the router connecting to the outside world does not provide dhcp by default; dhcp is not indispensable, but makes radar maintenance, computer swap etc... easier. root# vi /etc/dhcp/dhcpd.conf subnet 192.168.1.0 netmask 255.255.255.0 {
# default gateway
option routers 192.168.1.254;
option subnet-mask 255.255.255.0;
range dynamic-bootp 192.168.1.100 192.168.1.199;
default-lease-time 86400;
max-lease-time 86400;
}
Note that all the '";" are important. Reboot the system to commit changes. 24. Set-up APC UPS daemon The APC UPS have native support in ubuntu/linux and should be programmed to shutdown nicely the PC in case of power failure. The 900W Back-UPS 1500 is a good choice. Connect special USB cable between RJ-45 in back of UPS and host computer. apt-get -y install apcupsd Configure the apcups daemon: root# vi /etc/apcupsd/apcupsd.conf keep all configuration defaults, disable device name, then set to shutdown after 60 sec of power failure UPSCABLE usb UPSTYPE usb #DEVICE /dev/ttyS0 TIMEOUT 60 ANNOY 30 ANNOYDELAY 30 check configuration then start daemon root# apctest (...) root# service apcupsd start |