Translate

2021-02-14

Make your three USB disks keep their device identities (Ubuntu 20.04)

 

I use three USB disks, 1, 1.5 and 2.0 Tb connected to my laptop. They reside in adapter boxes that also can be used for disk duplication (Image attached at the end. Price ~$35). I have backup scripts creating a daily, weekly and monthly backup on them. There are some different approaches to this. My experience has shown, they may lose contact and be remounted as a different device, like "dev/sdc1" may end up being "/dev/sde1" if you're not careful about your handling or if you're unlucky.

The following contains some ideas that may be useful when approaching this scenario.

To counter that problem you may put a file, readable but not writebable (-r--r--r) for ANYBODY, in the disk's root (as high as you can go, still on the same disk). This file is called something meaningful like "label_sdc1" or "label_backupdisk:1". Before copying files to it, your script checks that this file exists.

There is a caveat. Let's say you mount it on /media as "/media/sdc1" or "/media/backupdisk_1". This needs the mountpoints  "/media/sdc1" or "/media/backupdisk_1" to be created in the media directory. 

One has to be really careful and check that the disk *is* mounted like that before you create the "label-"  file. If the disk isn't mounted where you want it, you will write the "label-" file to the directory "/media/sdc1" or "/media/backupdisk_1" in "/media" on your local disk. Your script have no idea that the backup is placed on your local system disk, it will look the same.  Then your backup will be written on the same disk you're trying to back up if the real target disk is off line. Eventually your disk space may run out and you'll get a write error, "No space on device".

So, with the labeled disk powered off, check that you *can't* find the "label_" file and that you *can* find it when it's powered on.

OK, how will I check that the disk I want to write on is really "/dev/sdc1" or whatever your preferred name would be?.

To find out the UUID:s of your disks, for each disk, power on *only* the disk you're interested and then run this (/dev/sda1 is normally your boot drive, I want to check /dev/sdb1 here.):

bash: /home/: blkid <ENTER>

the result will be like:
/dev/sda1: UUID="10b71446-98d2-4234-a177-4c287eae8c4a" TYPE="ext4" PARTUUID="731cd93f-01"
/dev/sdb1: UUID="28beab03-866c-4dbb-bc36-d6ff7ba51ed4" TYPE="ext3"
/dev/sdc1: UUID="f3be9443-74a9-43ee-8ba5-06bac24164f2" TYPE="ext3"
/dev/sdd1: UUID="2b181ab7-630d-4790-a0d8-774f96922b9b" TYPE="ext3"
PARTUUID="9361c7bd-01"

Enter the following in /etc/fstab (if you have *my* three disks on *your* system :):

UUID=28beab03-866c-4dbb-bc36-d6ff7ba51ed4 /media/backup/ ext3 defaults 0  0
UUID=f3be9443-74a9-43ee-8ba5-06bac24164f2 /media/bu_weekly/ ext3 defaults 0  0
UUID=2b181ab7-630d-4790-a0d8-774f96922b9b /media/work/ ext3 defaults 0  0

Reboot with all USB disks powered off. The start will seem to hang in the middle, just standing there stupidly. Then, power on the first disk and wait until you see the light on the adapter box flicker, indicating *a data transfer*.

Then do the same with each of the other disks, waiting for the data transfer on each. Voilá, they are now mounted correctly. *Your* very long UUID-number and mount points will not be like the example above, they will reflect *your* hard disks and *your* mount points. I left them looking like this to illustrate, only.

My backup scripts are run by /etc/crontab. 

If I do :
bash: # cat /etc/crontab <ENTER>

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# Run at system start!
@reboot root /home/xhosa/tools/scripts/myfw

# m h dom mon dow user  command <= This line is useful to have if you forget the meaning of all the stars! You'll have to enter it yourself.

# and here are my backups running and logging into different logfiles (monthly is disabled temporarily):

26 1    * * *   root    /usr/bin/php /home/xhosa/tools/scripts/incrbu_ctrl.php >> /var/log/incrbu/incrbu.log 2>&1
26 23   * * *   root    /usr/bin/php /home/xhosa/tools/scripts/bu_ctrl_weekly2.php >> /var/log/incrbu/bu_weekly.log  2>&1
##26 6  * * *   root    /usr/bin/php /home/xhosa/tools/scripts/bu_ctrl_monthly.php >> /var/log/incrbu/bu_monthly.log  2>&1


Each backup script uses phpmailer.php to send reports to my mail. Getting these reports to say something short and meaningful is a process, in my experience. Good luck with that. If you're not proficient in using bash, use your preferred scripting language and call the bash commands when needed.

In phpmailer I use these settings (since I use Gmail):

//Set the hostname of the mail server
$mail->Host = 'smtp.gmail.com';

//Set the SMTP port number - 587 for authenticated TLS, a.k.a. RFC4409 SMTP submission
$mail->Port = 587;

//Set the encryption system to use - ssl (deprecated) or tls
$mail->SMTPSecure = 'tls';

I use a separate account in Gmail for reports from my backups.

NOTE: In my backup scripts I check if its time to run (weekly and monthly) by checking the latest log file in each case. I write a  a "0" in a file (like: missed_weekly.log) before each backup. If the back up succeeds, I write a "1" in the file. Since all scripts are started each night, if it finds a  "0" in this file the script will try to run again the next night. If you encounter a problem and perhaps take a day or two to fix it, you know for sure that the missed backup will be run the following night.

 I also delete log files older than some date. 

My diskstation, one of three:


I'm sorry but I really can't show my backup scripts. They are *really* messy since they have grown as answers to each new demand. I WILL have to rewrite them in the future to avoid being ashamed showing them. 

To be continued...

2021-02-07

My Blender [Gaming?] PC. BIOS update, installation

 

This is an ongoing finished project. Updates, made during the process, are available at the end of the page.


I tried Blender on my laptop and discovered there was not enough memory. Besides, there was not enough speed. The latter I could live with but not the first.

I have a couple of 3D-printer projects I'd like to do but currently I have no 3D printer and currently no PC equipped to handle the project. I decided to start with the PC-part. There can never be a problem (in my imagination!) with having a fast PC. And I mean REALLY fast.

From my point of view this is a hysterical PC. But: I'd like to see if I can manage Blender AND if I can,  I want to get me a 3D printer . So, let's begin in the beginning.

I bought the following parts:



 

A Ryzen 3:rd gen processor mod 9, 3900 MHz, socket AM4. Note: orientation!


Whatever you do, don't try to mount the motherboard first and then the processor. It sucks royally, trying to fasten the cooler when the board is in the case. Not recommended! Mount the processor and cooler on your table before installing the motherboard. There are two slots for the M2 disks under the gray and black covers to the left of the CPU. Also under those, a fan in the nearest left hand corner to keep them cool. Temperature has a noticeable impact on TBW (Tera Bytes Written) so this is a good idea!



32Gb memory (4x8Gb) 
Read the instructions! If you have two modules they must be installed with one empty slot between them. The first slot is nearest the CPU! There's a cutout made in each module's contact and a corresponding key on the board. Align them correctly.

1Tb Samsung 860 EVO SSD 2.5" disk for storing 3D models

TEAMGROUP T-Force Cardea Zero Z440 1TB NVMe PCIe Gen4 x4 M.2 2280 Solid State memory for system disk. For the place I come from this may seem outlandish but, since we all have to rely on    reviews, I chose this one. It's missing in the image at the top of this page.  https://uk.pcmag.com/ssd/127511/corsair-force-series-mp600

I did not chose an M2 that supports 7000Mbs because I'm skeptical about too much bleeding edge. Buying this sort of volatile product, you'd better find out about TBW and the performance of these disks during different type of loads
and naturally: 




Seasonic 850w modular PSU. Seasonic seem to be a stable company, making power supplies for a long time

There are a *lot* of cables that come with the PSU:


It's bewildering, but all cables are marked so you know what end goes where and to what category of "consumer". A couple of them have SSD contacts, that's self explanatory. Two cables are marked CPU, one big goes in a big place you can't miss and one more with two four pin connectors. Only one of these is used on this motherboard, the other one is left hanging in the air. If you have a graphic card like the GeForce, I suspect two cables marked PCI-e are needed but at this point I 'm just guessing since I have to wait months before I will have one in my hand.

I bought  a Fractal Fractal Design "Meshify C Black". 

Why? It seemed to have places for all the equipment I've ever is going to need, it's good design for a one CPU machine. I've been assembling it for a day and I can say, I'm not disappointed. Using 2.5" disks you can at least have 10! The PSU is supplied with two 4- and one 2-connector cables and this number of disk is supported in the chassis. You may have to buy a hard disk mounting kit like:
to get that working, though. Each kit holds two 2.5" drives. Theoretical maximum 10 Tb. Should be enough.


It comes with 2 standard fans, "Fractal 120mm" pushing inwards at the front and one 120mm sucking air out at the back. I bought one more pushing inwards at the front, 120mm Noctua NF-A12x25 PWM and 2 140mm Noctua NF-A14 PWM  at the top, sucking the warm air out of the box. I'm cool, I want my PC to be cool too. PWM (Pulse Width Modulation), rpm-regulation can't get more effective than that.

If it so happens that you choose to buy this case, I have a suggestion: To install the extra front fan you must remove the front. It has fixed hooks in the upper part and a wide flexible one at the bottom and  smaller hooks, one per side. It's almost impossible to pry the lower end loose, I had to insert my hand from the inside through the hole, where the extra fan will sit, and press outward. That worked but with the extra fan sitting there, this is no option. I sawed away the wide flexible (not very!) hook at the bottom, letting the smaller hooks, on each side half ways up, hold the front. Then it was possible to remove it from the outside.




To run Blender 16Gb is recommended. I've placed an order for a  MSI GeForce RTX 3090 24GB VENTUS 3X OC graphic card. Delivery date (preliminary) is 21-04-13!! I ordered the first on in January but the seller couldn't give me any delivery date, not even a preliminary one. I  cancelled the order at the end of January and found another firm that at least had a preliminary delivering date. The difference, I think, is that the latter firm orders a lot more of this component, making them an Nvidia customer of higher priority. That's just a hunch, though. No blame falls on the first seller, faultlessly serving me for 20 years and being one of the first, where I live, to introduce "order before 14:00 and you'll get it the next day". Well, I'm still waiting for it, that's why this blog post will have a continuation.

During this period of no deliveries, what was produced went to resellers who put them on Ebay. Bidding drove the price to near the double since the rule of the market is a balance between availability and demand. Greedy bastards!




I noticed hat this graphic card has three times the memory available in my current  laptop.

The 1Tb SSD disk Samsung 860 is for storing my projects and for saving TBW on the system disk. If I think my projects are slow, I can move them to the M2  "Cardea Zero". I doubt it. My patience will be sufficient, my problem wasn't the storing, it was the processing. This disk is the regular 2.5" format. No image needed.

Surprisingly enough, what's really heavy work is to thread the cables from point A to point B. It takes time and, as a first timer, you will have to rewire it, until all is perfect. Very trying, indeed. Since the case is manufactured by one company and the motherboard, fans and whatever are manufactured by others, one is bound to face nomenclature problems and generalizations leaving one (me) bewildered. This is the part where your experience help you. If you have none, the Internet may help you.

Still, I'm amazed about how closely these companies follow the rules of the game. Seasoninc will supply enough cables of the right type. Fractal design will have cabling for PWR ON, Reset, HD-led  etc. MSI will have terminals to connect them to. I will not say it's seamless but pretty close! There are many conditions to meet before the parts fit together and everybody has to play along. It's an infrastructure, without which "building your own PC" would not be possible.

The Noctua 140MM fans come with silicon thingamajigs instead of screws (well, screws are also supplied) isolating the fans, vibrationally (if you excuse my language) and my first experience from installing them was, a hassle! But I noticed that instead of putting them in their right places and then trying to install the fans, the correct way to do this was to install the silicone fasteners on the fans and then install them in the chassis. Since this is a part of the cabling problem, one may have to uninstall and reinstall several times so this experience was really useful. Images below. 
A really useful detail: Make some soap water in a glass and apply on thingamajigs to make installation easy. This experience come from changing tires on my motorcycle. Also pliers, thin nosed, are invaluable.

The thingamajigs on the 120mm fan is much easier to handle. Pliers are still an advantage, though!




This was the really easy way of installing the fans,just let the lower part protrude and pry out the upper with a small screw driver.


Suggested pliers:


Future action: 
mounting disks, one on order and one secured. I've not decided where I will put them, there are a lot of options in that field. I'd like the M2 disk to carry the system, 3D models on the SSD and a 2Tb standard 3.5" mechanical disk for backup. I Prefer Linux and will write my own backup scripts so I know exactly what's going on. I'll learn more about that later. A new screen and an HDMI cable would be nice.


Update 2021-03-16: 
Still waiting for the GeForce RTX 3090 and a new monitor, MSI 27" Optix MAG274QRF-QD IPS QHD 165, both said to appear in april. 

Why I didn't buy the 27" Samsung C27G75T, the fastest of them all:
# supercurvature skews squares (excel sheets for example) out in the corners. # white bleed from the backlighting at the bottom, way too much. # The color is not good especially in comparison with MSI 27 "Optix MAG274QRF-QD IPS QHD 165 which I chose instead. The only thing that is *super good* is that it is the fastest on the market, but I do not need that to make 3D models! The "immersive feeling" that it can give does not happen if you sit more than 40cm from the front of the monitor when it is 32"! I wanted to buy a 27" which means I have to sit with my nose against the screen to get that experience. No way!


Update 2021-04-11 BIOS update:
Got my monitor! Yay! Now I will check how the machine runs. I connected first a HDMI cable and then a Display Port cable and none worked. Each time I started, the white led on the motherboard was lit, a white blinding light telling me I was having a problem with the graphics.

I thought maybe the motherboard has been on shelf for a while, there might exist a newer BIOS? Somewhere in a blog post somebody said the last four digits on the small white sticker between the CPU and the memory slots would show the version.


The version seemed to be 2802. Checking ASUS  homepage, under support Ryzen 9 3900x 3.8GHz last version 3801.
I downloaded it (ROG-STRIX-X570-E-GAMING-ASUS-3801.ZIP).



The BIOS file had to be extracted and renamed to SX570EG.CAP. I did not use the BIOSRenamer.exe, naturally, I'm a Linux user. My download directory looked like this, the file SX570EG.CAP is renamed and was originally called ROG-STRIX-X570-E-GAMING-ASUS-3801.CAP 

-rw-rw-r-- 1 snkdb snkdb   128088 nov 12  2019 '  BIOSRenamer.exe'
-rw-rw-r-- 1 snkdb snkdb 21279939 apr 12 12:00  ROG-STRIX-X570-E-GAMING-ASUS-3801.ZIP
-rw-rw-r-- 1 snkdb snkdb 33558528 apr  7 18:35  SX570EG.CAP

It was not unusual for the manufacturer of USB some years back, to add a bit of extra programming, hidden from the users eyes. Aside of it mostly being useless crap it was also the perfect place to store virus. I got suspicious of the stick, it was an old one, so I felt I had to reformat the it using cfdisk. I deleted the partition and created a new one type W95VFAT (= type 'b'). Then I used mkfs.vfat as shown below to create the file system:

mkfs.vfat /dev/sdf                                                               
mkfs.fat 4.1 (2017-01-24)                                                                                            
attribute "partition" not found                                                                                      
mkfs.vfat: Partitions or virtual mappings on device '/dev/sdf', not making filesystem (use -I to override)  
    [Aha! That's what I thought. The problem may well be precisely this extra space].
            
root@pf-ZBook-15:/devel/pic/snk_db2# mkfs.vfat -I /dev/sdf  [
Begone!!                          mkfs.fat 4.1 (2017-01-24)
attribute "partition" not found
Syncing disks.
root@pf-ZBook-15:/devel/pic/snk_db2#

Checking the result with cfdisk again, looked like this:


These actions solved the problem I had, doing the BIOS update. The BIOS update light flashed three or more times and then lit a steady green light which tells you there is a problem. After reformatting the USB stick, it flashed on and off during the entire procedure, around 5 min or so. The machine was powered off but the PSU was powered on.
When the update was done, i shut the PSU off for a while and then back on and started the machine.

Same thing, the white led was lit but the "00" on the display at the bottom of the motherboard told me it had powered on OK, but was still missing something related to the graphics. Sigh.

Back to the Internet. After digging around a good while encountered  
It contained about everything there is to know about 3900x. Like this:

Graphics[edit]

This processor has no integrated graphics. 

Thank you very much. I will have to wait for my graphics card to arrive before liftoff. Arrghh! Well, I learned some things, that's good. I found the root of the problem, that's good. 


Update 2021-04-25
so... waiting forever fo my Geforce card I realized I may have to wait forever so I ordered a Radeon RX6900XT instead from USA. They had 2!!! in their stock. It finally arrived. I installed it and found out that my TEAMGROUP T-Force Cardea Zero Z440 1TB NVMe PCIe Gen4 x4 M.2 2280 was nowhere to be found. It's a DOA. Dead On Arrival. I'm in the process of trying to find out who is going to be the receiver of my warranty complaint. $10 to the one finding it at Amazon. They must be kidding! Do they think the will sneak out of something so selbsverständlich like a warranty?! Fie on them!

Well, that's "work in progress". The next hurdle. I produced an USB installation stick (or should I say 'shtick'?) with the help of https://www.tecmint.com/create-bootable-ubuntu-usb-disk/. Worked nicely with ddrescue, I have no complaints. But, unfortunately it didn't end there.

https://www.amd.com/en/support/kb/release-notes/rn-amdgpu-unified-linux-20-45 AMD explained that (K)Ubuntu 20.04.2 wasn't supported by their Radeon™ Software for Linux® 20.45 . I'll have to wait for their version 20.50. Sucks. Then they rattled on about how I had to back down from Linux kernel 5.8 to 5.4. Sure! I don't even have any output from my Radeon card to my screen, how are you supposing I could do that?

Next problem. On every site displaying download links to 20.04.1, in the directory i found 20.04.2.   
Risking Trojan horses, spy ware, any kind of sh-t you can imagine, I downloaded it. I wouldn't recommend it, but I was  frustrated, so I downloaded anyway. Made a new USB with this system...

and it downloaded into my new machine without me getting ANY indication of the destination! Thank you AMD!

... and : Voilá!
It seems too work. Linux is downloading (have no idea of where, still!) but it will keep me floating.  OK, it ended, the installation. Whooppee! Now I have to stop updating Linux kernel from 5.4 to 5.8. 
It will be investigated, how.

One more NOTE:
Above I said: "I suspect two cables marked PCI-e are needed" [ to the graphic card ].
It was three!



Update 21-04-26
I downloaded Ubuntu 20.04.01 in pure desperation. Then... I found out that running Ubuntu GUI was like running... ugh!... Windows. This won't do. The newest Kubuntu was 19.04. I chose to be relieved of my pains... Oh, my, what a GUI! Currently waiting for it to be installed.

Update 21-05-13
So, I downloaded Kubuntu and ended up with a completely black screen. Kubuntu couldn't find my fat Radeon GPU. I didn't matter if I downloaded kubuntu-plasma-desktop, the minimum, or if I downloaded Kubuntu, the release. I wrote to the Kubuntu contact person explaining, in short terms, what my problem was and went back installing Ubuntu 20.04.2. It worked. AMD says they've fixed their driver so it should work. Now I don't know if that driver is incorporated into Ubuntu, if Ubuntu runs in a safe mode or what. 

During these tumultuous time the system landed om my M2 card. 1000Mb/s instead of 600Mb/s. 67% increase in speed ought to be a noticeable difference even in day use. And yes, the machine is feeling more responsive than before.

I have to try to find out if downloading and installing the drivers directly from AMD will increase performance or not. Just now I'm tailoring my ubuntu to suit my needs, there are always ways to work around things. I've revamped my workspace (cleaned it too!) and for now this is very useful. In the long run, I really do not need one laptop and one super computer. I will have to make a decision later, what to do with the laptop. 

Update 21-07-23
A caveat: Sometimes I had to restart the machine to make it find the console (MSI 27" Optix MAG274QRF-QD IPS QHD 165). The problem is solved by always power on the console and then immediately the machine. If the console has been standing for a while, powered on, the machine can't find it. When the system is up, however, I can power off the console and return later, powering it on, and it's still OK. A little weird but knowing this, it's not a problem anymore.

The 140mm fans from Noctua was *extremely* quiet. The front Noctua 120mm, alone, was noisier than all other fans together!! I have disconnected it, CPU is running around 41ᵒC, I can barely hear it running. Nice!

As one could expect, I moved the whole environment from my laptop to my new machine. Everything now works splendidly.
The take away from all this is: there may be problems using a CPU with no graphic capacity at all together with the latest and fattest GPU available. The Nvidia card I wanted was probably stuck in the "Ever Given" cargo ship in the Suez canal, there was a world wide deficit of computer chips, prices went up horribly. The Radeon card was the next best. I think it's good enough.

To be continued...  Probably not. I think this story ends here.