Consumer/Producer jobqueue with the gnu parallel

The hosts.txt file must have atleast one line:

tail /data/p.txt -f | parallel -k --slf hostnames.txt

the /data/p.txt and hostnames.txt one can change on the fly.

Replacing failed disk in the software raid1

The disk /dev/sda is failing:
cat /proc/mdstat
Remove it from the raid:
cat /proc/mdstat
mdadm --manage /dev/md127 --fail /dev/sda1
mdadm --manage /dev/md126 --fail /dev/sda3
mdadm --manage /dev/md125 --fail /dev/sda2

after replacing the bad disk we should add the new one in to the raid:

dmsetup remove_all
sfdisk -d /dev/sdb | sfdisk /dev/sda
cat /proc/mdstat
mdadm --manage /dev/md127 --add /dev/sda1
mdadm --manage /dev/md126 --add /dev/sda3
mdadm --manage /dev/md125 --add /dev/sda2

Monitor the sync process:
watch cat /proc/mdstat

Every 2.0s: cat /proc/mdstat                                                                                                                                                                             Tue Jun 26 10:33:19 2018

Personalities : [raid1]
md125 : active raid1 sda2[2] sdb2[0]
      2099136 blocks super 1.0 [2/1] [U_]
        resync=DELAYED
      bitmap: 1/1 pages [4KB], 65536KB chunk

md126 : active raid1 sda3[2] sdb3[0]
      477894656 blocks super 1.2 [2/1] [U_]
      [===>.................]  recovery = 19.9% (95154432/477894656) finish=129.8min speed=49130K/sec
      bitmap: 4/4 pages [16KB], 65536KB chunk

md127 : active raid1 sda1[2] sdb1[0]
      8256512 blocks super 1.2 [2/2] [UU]

unused devices:

Ovirt 4.2 clean up the image upload hanging process

After the upgrade, some of the hosts are not going to the maintenance mode.

In our case, it was a hanging process of image upload.

To clean the DB:

su – postgres

psql -d engine

select * from image_transfers;

if not empty

truncate image_transfers;

Now you can proceed w/o problems.

Ovirt upgrade from 4.1 to 4.2

yum clean all
yum install http://resources.ovirt.org/pub/yum-repo/ovirt-release42.rpm
yum-config-manager --save --setopt=ovirt-4.1-centos-gluster38.skip_if_unavailable=true
yum-config-manager --save --setopt=ovirt-centos-ovirt41.skip_if_unavailable=true



And finally:

yum update

centos 7 firewall d routing IB to eth network with multihomed server

Add the following to /etc/sysctl.conf: net.ipv4.ip_forward = 1
Apply the sysctl settings: sysctl -p

Add direct rules to firewalld. Add the --permanent option to keep these rules across restarts.
firewall-cmd --zone=public --add-masquerade --permanent
firewall-cmd --direct --add-rule ipv4 nat POSTROUTING 0 -o ens1.801 -j MASQUERADE
firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -i ib0:3 -o ens1.801 -j ACCEPT
firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -i ens1.801 -o  ib0:3 -m state --state RELATED,ESTABLISHED -j ACCEPT



credits goes to:
https://www.centos.org/forums/viewtopic.php?t=53819

Raspberry pi 3 camera quality vs Moto G5

raspistill --sharpness 100    -q 5 -t 1 -mm average  -ifx denoise -sa 20 -ISO 200 --vstab   -awb  fluorescent  -w 1920 -h 1080    -o /var/www/html/cam/cam.jpg

Enable on Visual Studio 2017 BUILD and RUN option

HDF5 v1.10 or above on the Lustre FS

If you unable to read the files on the lustrefs with the  HDF5 library v1.10 and above you should mount the FS with localflock option. The reason is flock support absence by default.

h5ls --version
h5ls: Version 1.10.1
Error:
h5ls snapshot.hdf5 snapshot_063.hdf5: unable to open file
What?!

mount -tlustrefs  -o localflock 10.10.10.10@o2ib:/lustre /lustre

Now everything is ok!!
 
h5ls snapshot.hdf5
Header                   Group
PartType0                Group 

Performance test: 24 SSD 150 GB Intel on MegaRAID SAS 9361-24i

No RAID, simply jbod mode dstat results:

Reading:
lsscsi | grep SSDSC2BB15| awk '{printf"dd if=%s of=/dev/null bs=16K & \n",$7}'| sh

  0  18  21  60   0   0|6421M    0 | 120B  876B|   0     0 |  36k   53k
  0  18  17  64   0   1|6414M    0 | 120B  346B|   0     0 |  36k   53k
  0  18  14  67   0   1|6413M    0 | 120B  346B|   0     0 |  36k   53k

Writing:
lsscsi | grep SSDSC2BB15| awk '{printf"dd of=%s if=/dev/zero bs=16K & \n",$7}'| sh 

  0   9  11  77   0   3|   0  4503M| 120B  346B|   0     0 |  24k 8055
  0   9  11  77   0   3|   0  4525M| 120B  346B|   0     0 |  24k 8046
  0   9  13  75   0   3|   0  4505M| 120B  346B|   0     0 |  25k 7968

adding more stress:
lsscsi | grep SSDSC2BB15| awk '{printf"dd if=%s of=/dev/null bs=16K & \n",$7}'| sh
lsscsi | grep SSDSC2BB15| awk '{printf"dd of=%s if=/dev/zero bs=16K & \n",$7}'| sh 

  0  19   1  75   0   4|3420M 3401M| 120B  362B|   0     0 |  37k   36k
  0  18   1  77   0   3|3282M 3261M| 180B  362B|   0     0 |  36k   34k
  0  19   2  76   0   3|3297M 3215M| 180B  362B|   0     0 |  35k   33k

Not Bad:)

 

Lustrefs: a big performance hit on lfs find after patch of: CVE-2017-5754 CVE-2017-5753 CVE-2017-5715

Disable: sh set-protection.sh 0

time lfs find /lustre/arm2arm/| wc -l
1454706

real    0m14.941s
user    0m1.633s
sys    0m10.770s

 Enable: sh set-protection.sh 0

time lfs find /lustre/arm2arm/| wc -l
1454706

real    0m10.468s
user    0m0.959s
sys    0m5.521s

Let us hope that the situation will change in the near future....

 And the script set-protection.sh content is:

 #!/bin/bash

[ ! -d  /sys/kernel/debug/x86 ]&& mount -t debugfs debugfs /sys/kernel/debug
echo $1 > /sys/kernel/debug/x86/pti_enabled
echo $1 > /sys/kernel/debug/x86/ibrs_enabled
echo $1 > /sys/kernel/debug/x86/ibpb_enabled

Enable or disable Meltdown and Spectre attack protection on CentOS

After kernel and microcode update one can enable or disable the protection:
Enable:
sh ./set-protection.sh 1
or
Disable:
sh ./set-protection.sh 0 

And the script content is:

 #!/bin/bash

[ ! -d  /sys/kernel/debug/x86 ]&& mount -t debugfs debugfs /sys/kernel/debug
echo $1 > /sys/kernel/debug/x86/pti_enabled
echo $1 > /sys/kernel/debug/x86/ibrs_enabled
echo $1 > /sys/kernel/debug/x86/ibpb_enabled

Q:Why should I disable the protection?
A: If you enable the protection you might accounter the performance degradation.
For some tasks one can enable or disable it:
Synthetic test iperf3 shows network performance over IB degradation about x2:
node01:iperf3 -s
node02:iperf3 -c node01.ib

Enabled:

[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  8.70 GBytes  7.48 Gbits/sec    0             sender
[  4]   0.00-10.00  sec  8.70 GBytes  7.47 Gbits/sec                  receiver

Disabled:
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  17.6 GBytes  15.1 Gbits/sec    0             sender
[  4]   0.00-10.00  sec  17.6 GBytes  15.1 Gbits/sec                  receiver






[node01~]#ibstat
CA 'mlx4_0'
    CA type: MT4099
    Number of ports: 1
    Firmware version: 2.40.7000
    Hardware version: 0
    Node GUID: 0xXXXXXXX
    System image GUID: 0xXXXXXXX
    Port 1:
        State: Active
        Physical state: LinkUp
        Rate: 56
        Base lid: 338
        LMC: 0
        SM lid: 3
        Capability mask: 0xXXXXXX
        Port GUID: 0xXXXXXXX
        Link layer: InfiniBand

Fixing X11 forwarding on CentOS7.4

I have an client Centos 6.5 trying to connect to Centos 7.4 with X11 forwarding, but no luck:

Finally I found that somehow the important option is missing in the server side in /etc/ssh/sshd_config :

X11UseLocalhost no

After adding everything is ok now.

Lustre 2.10.x dkms usage

get the src for the lustre-client-dkms package from the:

https://build.hpdd.intel.com/job/lustre-b2_10/arch=x86_64,build_type=client,distro=el7,ib_stack=inkernel/

1) rpmbuild --rebuild --without servers lustre-client-dkms-2.10.2_RC1-1.el7.src.rpm
2) yum install /root/pmbuild/RPMS/noarch/lustre-client-dkms-2.10.2_RC1-1.el7.centos.noarch.rpm
It will rebuild the modules for the active kernel with the following commands:
 

/bin/bash /sbin/dkms build -m lustre-client -v 2.10.2_RC1 -k 3.10.0-693.5.2.el7.x86_64

PS troubleshooting
if dkms  status shows something like this:

 dkms status
lustre-client, 2.10.2_RC1, 3.10.0-693.5.2.el7.x86_64, x86_64: installed (original_module exists) (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!)

1) dkms --force remove  -m  lustre-client -v 2.10.2_RC1 -k 3.10.0-693.5.2.el7.x86_64
2) rm -fr  /lib/modules/3.10.0-693.5.2.el7.x86_64/extra/lustre
3) find and remove manually the module leftovers: find /lib/modules/| grep lustre
4) dkms --force install -m  lustre-client -v 2.10.2_RC1 -k 3.10.0-693.5.2.el7.x86_64

Deleting the funny 192.168.122.xxx network from CentOS 7.4

In order to stop unnecessary interface on the VM:
 
yum install -y libvirt-client
systemctl start  libvirtd 
virsh net-destroy default
systemctl stop  libvirtd 
systemctl disable  libvirtd 


 

Performance consumer nvme ssd vs samsung evo pro SSD

Micron 1100 256GB M.2 vs Samsung SSD 850 EVO 1TB

Micron vs Samsung

List used ciphers on remote HTTPS server

Nmap is your friend:

nmap -sV --script ssl-enum-ciphers -p 443 remote.server.my

Again lustre upgrade troubles: unable to mount after upgrade to 2.10.x

The failure is following:

LDISKFS-fs (sda): Unrecognized mount option "context="unconfined_u:object_r:user_tmp_t:s0"" or missing value

But Selinux is disabled so we should remove that mount options.
To fix it:

tunefs.lustre --mountfsoptions="user_xattr,errors=remount-ro" /dev/sda

mount a expected.

Recover a single file on the Lustre due to the OST corruption.

Assuming the OST number 4 is corrupted.to get the ost ID on the client one can use lfs df  :

lfs df /archive
UUID                   1K-blocks        Used   Available Use% Mounted on
arch-MDT0000_UUID       99428812    44432680    48286884  48% /archive[MDT:0]
arch-OST0000_UUID    63838042896 46275072544 14344891120  76% /archive[OST:0]
arch-OST0001_UUID    63838042896 46036859640 14583104024  76% /archive[OST:1]
arch-OST0002_UUID    63838042896 34406650692 26213311960  57% /archive[OST:2]
arch-OST0003_UUID    63838042896 39355270936 21264676344  65% /archive[OST:3]
arch-OST0004_UUID    63838042896  7102256308 53517690972  12% /archive[OST:4]

If the OST got corrupted then file attributes are still on MDS so we can filter all corrupted files by following:

lfs find /archive --ost 4  -type f | xargs -I% sh -c "[ ! -f % ]&& echo %" | tee -a recover/todo/OST04-corrupted.txt

this is assuming that files does not contain spaces or nasty characters. For the general case one should use python script for the proper handling of the filenames.

Now once the file list is complete, one can use following script to copy the files over the corrupted files, here we assume that the backup path is /backup and target path is /archive:

 cat recover.sh

#!/bin/bash
file=$1
cat $file   | xargs -I{} sh -c  "[ -f \"/backup{}\" ]&&echo 'unlink \"{}\";cp -a \"/backup{}\" \"{}\"' "| parallel  --progress --eta 

./recover.sh recover/todo/OST04-corrupted.txt

Running ANSYS on LINUX

Check which version is installed:
ls /ansys_inc/
v172
v182

If you want to run v182 then the simple answer is:

/ansys_inc/v182/Framework/bin/Linux64/runwb2

Recover uefi boot on the new laptops: Windows 10 + Linux

On the new laptop somehow the boot loader got back to the windows 10 only, so to recover it simply run the command in the cmd.exe as a Administrator:

 bcdedit /set {bootmgr} path \EFI\ubuntu\grubx64.efi