 |
isorespinner.sh |
This script is a progression of 'isorespin.sh'. Whilst 'isorespin' was created to support Ubuntu and similar Linux distributions on mini PCs, some of the functionality that was developed is now unused. Additionally with Canonical now trialling a new Ubuntu desktop installer and the ISO using a new multi-layer filesystem, rather than hack the earlier code to coerce compatibility I've developed a new script which I've called 'isorespinner.sh' and provides the most useful features and addresses the functionality that is now relevant to respinning.
The syntax of the new script is as follows:
isorespinner.sh -i <ISO> [ -w <directory> | -u | -k <kernel> | --dist-upgrade | --upgrade | --interactive | --all | --disk | --debug | ...
isorespinner.sh ... -b [ GRUB-32 | GRUB-64 ] | -g [ "" | "<kernel boot parameter> ... " ] | c "<cmd> ... " | ...
isorespinner.sh ... --key "<repo> ... " | -r "<repo> ... " | -p "<pkg> ... " | -l "<pkg.deb> ... " | -e "<pkg> ... " | -f [ "<file> | <directory> ... " ]
with a lot of the options unchanged from their previous functionality as described here.
The primary significant difference between 'isorespin.sh' and 'isorespinner.sh' is that the new script no longer adds a GRUB 32-bit bootloader by default so the option '-b GRUB-32' should be used if it is required. Secondly along with removing redundant options, the GUI interface has been dropped so options can only be specified using the command line. As a result all the previous functionality can be replicated using the current options. The prerequisite dependencies are the same as before and if not installed the script will identify any that are required.
The new script includes two new features:
- The script tries to run using memory for the temporary files to alleviate continuous read/writes to storage and thereby reduce drive wear. However to exclude running in memory the option '--disk' should be specified especially if using the script for large and complex respins as currently it is possible to exceed the initial memory allocation which results in running out of 'virtual' space.
- The other key new feature is the support for multi-layer filesystem ISOs such as the new canary Ubuntu Jammy 22.04 ISO. However due to the nature of how such ISOs uses layers, respinning becomes somewhat dependent and/or specific to the individual layers. The default language layer is derived from the 'LANG' shell variable or set as 'English' if either the variable is unset or set to 'C.UTF-8'. A different language layer, assuming it is already supported by the ISO, can be respun by prefixing the respin command with the required 'LANG' variable, so for Spanish for example, use 'LANG=es isorespinner.sh -i ...'. If all language layers require respinning then use the option '--all'. Currently only a 'normal' installation is supported as the 'minimal' installation has not yet been investigated.
Looking at invocation, a good example of why 'isorespinner.sh' is required is the back-porting of the Intel P-State driver fix for Intel Alder Lake processors. Currently the brand new Jammy ISO ships with Linux kernel version 5.15.25 which results in poor performance on Alder Lake due to ITMT support for with P and E core selection. However the Linux kernel version 5.15.35 includes this back-ported fix and this can be incorporated into a respun ISO by using the option '-k v5.15.35' which will then use the Canonical build of the Mainline Test tree prior to the kernel filtering through the kernel rollout process.
Whilst before it was possible to respin an ISO suitable for Intel Atom processors using the single '--atom' option, now the various components have to be first downloaded and respun as specific options e.g. '-b GRUB-32 -l rtl8723bs_4.12.0_amd64.deb -f linuxium-install-UCM-files.sh -f wrapper-linuxium-install-UCM-files.sh -f linuxium-install-broadcom-drivers.sh -f wrapper-linuxium-install-broadcom-drivers.sh -c wrapper-linuxium-install-UCM-files.sh -c wrapper-linuxium-install-broadcom-drivers.sh' as the '--atom' option has been depreciated. For reference these are the 'atom' files:
rtl8723bs_4.12.0_amd64.deb (used for earlier versions of the linux-firmware package)
linuxium-install-broadcom-drivers.sh
linuxium-install-UCM2-files.sh (used for ISOs 20.04 and up - rename as 'UCM' without the '2')
wrapper-linuxium-install-broadcom-drivers.sh
wrapper-linuxium-install-UCM-files.sh
I've also found that the new multi-layer filesystem ISO was too 'heavy' for very low-powered Intel mini PCs. However with a minimum configuration of a Cherry Trail processor and 2GB memory and some specific respinning, both usage and installation was possible. Even then an Intel Compute Stick struggled but using a respun ISO with the options '-b GRUB-32 -g intel_idle.max_cstate=1 -g fsck.mode=skip -p lz4 -c "sed -i 's/^COMPRESS=zstd/COMPRESS=lz4/' /etc/initramfs-tools/initramfs.conf"' prevented both freezing during installation and also running out of memory when creating a new 'initramfs'.
As this is the initial release inevitably there will be improvements required so please donate if you find the script useful using the following link http://goo.gl/nXWSGf as everything helps with development costs.
45 comments:
Hi, I want to install Kubuntu 22.04 on an old Atom tablet. Therefore this script would be perfect for me. But it seems that there are some errors in the script (yes I saw that 22.04 is still experimental).
I started the script with the following command (downloaded all files beforehand):
"-i kubuntu-22.04.1-desktop-amd64.iso -b GRUB-32 -l rtl8723bt_4.12.0_amd64.deb -f linuxium-install-UCM-files.sh -f wrapper-linuxium-install-UCM-files.sh -f linuxium-install-broadcom-drivers.sh -f wrapper-linuxium-install-broadcom-drivers.sh -c wrapper-linuxium-install-UCM-files.sh -c wrapper-linuxium-install-broadcom-drivers.sh"
I don't know exacty if "bs" or "bt" is correct, I used "bt".
I got the following error messages:
"sudo: ar: command not found
tar (child): data.tar.zst: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
mv: cannot stat 'usr/share/cd-boot-images-amd64/images/boot/grub/efi.img': No such file or directory
isorespinner.sh: Adding 32-bit GRUB bootloader ...
du: cannot access 'iso.efi': No such file or directory
mount: /home/marcel/Downloads/isorespinner/mnt: special device iso.efi does not exist.
cp: 'mnt/' and './mnt' are the same file
umount: mnt: not mounted.
cp: missing destination file operand after 'mnt'
Try 'cp --help' for more information.
cp: cannot create regular file 'mnt//boot/bootia32.efi': No such file or directory"
I fixed the typo of "tar" in line 2072, also I deleted a slash after "${EFI_DIRECTORY}" in line 2191 to prevent the double slash (don't know if that makes a difference, though).
Now I got the following error messages:
"tar: Refusing to read archive contents from terminal (missing -f option?)
tar: Error is not recoverable: exiting now
tar (child): data.tar.zst: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
mv: cannot stat 'usr/share/cd-boot-images-amd64/images/boot/grub/efi.img': No such file or directory
isorespinner.sh: Adding 32-bit GRUB bootloader ...
du: cannot access 'iso.efi': No such file or directory
mount: /home/marcel/Downloads/isorespinner/mnt: special device iso.efi does not exist.
cp: 'mnt/' and './mnt' are the same file
umount: mnt: not mounted.
cp: missing destination file operand after 'mnt'
Try 'cp --help' for more information.
cp: cannot create regular file 'mnt/boot/bootia32.efi': No such file or directory"
Can you please help? Would like to test Kubuntu on the tablet since Ubuntu is a little bit slow and Lubuntu is not really made for tablets.
@Marcel: Line 2072 should be 'ar' and not 'tar' as it looks like you first need to install the 'binutils' package that provides this command.
@Linuxium: Sorry, didn't know that. I installed the missing 'binutils' package and the ISO was created successfully. Installing Kubuntu now, thanks!
Hi again, thanks again for the clarification!
Another problem problem I got after installation was no sound. I used the linuxium-install-UCM2-files.sh (yes I renamed it after downloading), since it's "used for ISOs 20.04 and up". After respinning with the other UCM-files sh file sound is working. Do you know why I couldn't use the "correct" file? WiFi is working better with the "rtl8723bt" file, though. Your ISOs do have the "rtl8723bs" which led me to WiFi issues (disconnecting sometimes).
@Marcel: Unfortunately WiFi, Bluetooth and audio are the most temperamental of mini PC functionality and are often affected by package and firmware updates. Basically if you get it to work then quit while you are ahead!
Is isorespinner.sh compatible with the newest 22.10 Ubuntu?
@Marcel: Yes, the latest version (1.0.2) now includes support for 'kinetic' and 'Ubuntu Unity' ISOs.
Thank you for the fast support of 22.10. What I am now experiencing is that just before the end of installation I get the error message "GRUB couldn't be installed". Used the same command as for 22.04 where it worked.
@Marcel: Hopefully I will have some time towards the end of next week to investigate whether I can replicate your issue.
@Marcel: I've just tested respinning Ubuntu 22.10 and then installing it on a 32-bit boot mini PC and I didn't get any errors. Can you try respinning your ISO and then installing it and if there are errors, can you take screenshots so I can see what might be happening?
Hi! Thanks so much for the great work! I was wondering if it might be possible to use this with Ubuntu Server images. I tried but it rejects with a message listing different types of os that are supported, but won't continue.
hi. thanks for hard work. i'm beginner and would like to know:
regards "atom files" do i chmod? which folder ? install utils?"
project asus t100ta and lubuntu 20.04.5
Wow, Linuxium to the rescue! Was trying to install Lubuntu 22.04 LTS on a Gigabyte T5140 media player with AT3N7A-I board (Intel Atom N330): it's UEFI bios doesn't support 64 bit. Respun with '-b GRUB-32' and now works like a charm. Respinning Lubuntu 22.10 also worked fine.
@sybren: If the instructions are too complex then download the Lubuntu ISO from the ISO link at the very top as this has the 'atom files' preconfigured.
@Lachlan: The official Ubuntu server images are in a different format so my script deliberately rejects them. However if you put 'server' in the 'Search blog' top right and then use 'isorespinner.sh' with options along the lines of those in the post you should be able to create a pseudo server ISO. I did support respinning the desktop ISO with a '--server' option but I dropped it due to lack of interest!
thank you for reply linuxium. i made your scripts work, what a very rainy weekend can do for your education. i made it work for 22.04. yesterday, after playing with new system i tried in house upgrade. bad idea. upgrade looked perfect after reboot but processor load was 100%. i quit after 30 minutes on account of low battery. any ideas why.
have gone back to lub 22.04. thanks again.
Hi and sorry for the late reply. I tried respinning and installing 22.10 again. Got the following error (sorry I only could take a photo, screenshot was not possible): https://i.ibb.co/TctT3cv/Linuxium-Ubuntu-22-10-Installation-fails.jpg
@Marcel: Having looked into this I don't think this is a script problem but rather the NVRAM is potentially corrupt. Go to 'https://linuxiumcomau.blogspot.com/2017/06/customizing-ubuntu-isos-documentation.html' and search for 'NVRAM sync issues' and follow the steps from 1 to 6 and then 8 (i.e. don't bother with step 7). Then repeat the installation from your respun 22.10 ISO again and hopefully it will allow 'grub' to successfully install.
@Linuxium: Thanks for the help. But unfortunately, this did not help. Still the same issue after deleting all the bootnum.
@Marcel: My only other suggestion is to first use the respun ISO as a 'LiveUSB' and reformat all the drives on the installation device (note: take a backup of any data you need to keep prior to this) and then repeat the removal of all the NVRAM entries like before. Then try another installation.
@Linuxium: Thanks for your help, wiping the whole volume did the trick! Unfortunately 22.10 did not solve the audio problems. I will try a few other distros. Thanks again!
Hello again, I tried installing Lubuntu, since that seems to be working better (faster), but unfortunately I got the following error:
"Befehl apt install -o Acquire::AllowInsecureRepositories=true -o APT::Get::AllowUnauthenticated=true -o APT::Get::allow-remove-essential=true -y --no-upgrade -o Acquire::gpgv::Options::=--ignore-time-conflict grub-efi-$(if grep -q 64 /sys/firmware/efi/fw_platform_size; then echo amd64-signed; else echo ia32; fi) beendet mit Exit-Code 100.
Ausgabe:
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
Reading package lists...
Building dependency tree...
Reading state information...
Package grub-efi-ia32 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
grub2-common grub-common grub-pc
E: Package 'grub-efi-ia32' has no installation candidate"
Even after deleting bootnum and deleting the drive. Worked with Ubuntu and Xubuntu, though.
@Marcel: I think you may have hit the issue where the most recent distros have started retiring/removing 32-bit packages from their repositories breaking respinning and/or installation. At this stage my only (rather tedious) workaround would be to respin and install a 'focal' based release which hopefully still installs the 32-bit GRUB packages and then manually upgrading using 'do-release-upgrade' or equivalent to get to 'jammy' or later etc.
@Linuxium: I tried it with 22.04 and also 20.04, always the same error. Seems strange that the distro which is meant for devices which are not as fast and older where the installation fails. Any other suggestions? Are you able to install Lubuntu? Maybe I did something wrong.
@Marcel: I've sucessfully installed the Lubuntu ISO from the 'ISOs' link above and I've respun a Lubuntu 22.04 ISO and successfully installed it both on a 32-bit bootloader device (Intel Compute Stick) with neither generating any errors. The only two further suggestions I have is 1. make sure you are connected to the internet during installation and 2. use 'English' as the default language as these are the two testing conditions I've successfully tried.
I wanna respin a Lubuntu 16.04 ISO that was EOF already. Is there any way I can do it with isospinner.sh? Or do you have an image for Lubuntu/Ubuntu 16.04 that can share? It is because I need this distribution for my old product. Many thanks!
@ODBWilson: 'isorespinner.sh' relies on the ISO being current (i.e. not EOL) so that it can download and configure GRUB packages and update 'pool' packages where required. An 'EOL' ISO would break this functionality so 'isorespinner.sh' checks this first.
If you were to use an 'EOL' ISO, you wouldn't get any support including kernel, security and application updates and also, potentially, community support for issues.
Hi. Thanks a lot for this wonderful work. I have a question about your broadcom script. At line 35 there is "sudo cp brcmbt.service /lib/systemd/system". Could you tell me where the brmcbt.service file is coming from? I don't see it in the downloaded archive. My goal is to install these drivers on my favorite distribution which is not ubuntu based. But I am quite new with bash scripts.
Regards
@BA839: The file 'brcmbt.service' is within the script 'linuxium-install-broadcom-drivers.sh' which, after running, extracts it along with several other files.
@Linuxium: yes, exactly, but all the other extracted files in the script come from the tar archive except this one. Where is this 'brmct.service' created or where is this coming from before you cp it? I also see this three lines that I don't understand
"sed '1,/^exit$/d' < ../$0 | base64 -d | sudo tee $0.zip > /dev/null 2>&1
sudo unzip $0.zip > /dev/null 2>&1 > /dev/null 2>&1
sudo rm -f $0.zip"
what are you exactly unziping?
Regards and thanks for the help
@BA839: The whole file for 'linuxium-install-broadcom-drivers.sh' contains the script (upto the 'exit' line) and then appended is an encrypted file. This encrypted file is an ASCII file that has been encoded using 'base64' to convert binary data so that it can be stored and downloaded without issue. The binary data is actually a compressed file using 'zip'. So the lines in the script you don't understand first self-extract the ASCII file ("sed '1,/^exit$/d' < ../$0") i.e. use 'sed' to delete the script from line 1 to the line with 'exit', and then pipe the remaining ASCII data through 'base64' to decode it ("base64 -d") and then pipe to a file called the same as the script ("$0") but with an additional extension of '.zip'. This last part requires 'root' privaleges to overcome any 'write' permission issues and therefore 'tee' is used to duplicate the output to the file ("sudo tee $0.zip") and the actual output is thrown away as it is no longer needed ("> /dev/null 2>&1"). Now that we have the zip file, uncompress it ("sudo unzip $0.zip"), remove it as it is no longer needed ("sudo rm -f $0.zip") and then copy the files it contained including the file 'brmct.service' to the required locations.
You can try this yourself if you have the file 'linuxium-install-broadcom-drivers.sh' already downloaded. First create a temporary directory just to have a clean workspace and then copy 'linuxium-install-broadcom-drivers.sh' into it. Then enter the following commands:
sed '1,/^exit$/d' < linuxium-install-broadcom-drivers.sh | base64 -d | sudo tee linuxium-install-broadcom-drivers.sh.zip > /dev/null 2>&1
sudo unzip linuxium-install-broadcom-drivers.sh.zip > /dev/null 2>&1
Perform a 'ls' and you will see the 'brcmbt.service' file along with the others files.
BTW, the 'cp -n' uses the 'n' flag meaning 'no-clobber' to prevent overwriting any existing files (abit like the '-p' in the 'mkdir' above it). The premise is that if the files already exist, hopefully they are more current that the ones contained in the script.
Thanks a lot. I will need time to understand fully but I can understand the principle. It will be helpfull
I am trying to install ubuntu 22.04 on an intell stick (STCK1A8LFC) but I keep getting a kernel panic after I select "install or try ubuntu" from the grub bootloader (which is loaded from the usb installer)
I am using:
./isorespinner.sh -i ubuntu-22.0.4.3-dekstop-amd64.iso -f linuxium-install-UCM2-files.sh -b GRUB-32
But also tried:
./isorespinner.sh -i ubuntu-22.0.4.3-dekstop-amd64.iso
The script runs fine and creates the linuxium-ubuntu-22.04.3-desktop-amd64.iso file which I am able to write to my usb stick with sudo dd if=linuxium-ubuntu-22.04.3-desktop-amd64.iso of=/dev/sdc bs=4M status=progress
After I try to boot from the usb stick, I manage to get into grub, but after that, I get a kernel panic.
The kernel panic is saying:
Initramfs unpacking failed: write error
failed to execute /init (error -2)
Kernel Panic - not syncinc: No working init found. Try passing init= option to kernel....
What am I doing wrong?
@Skylite: Put simply, Ubuntu has become too big for the STCK1A8LFC stick. Basically, as part of booting the ISO, it needs to decompress the 'init' filesystem. However as the STCK1A8LFC only has 1GB of RAM it runs out of memory before completing the filesystem extraction and subsequently panics or crashes.
Thanks for the reply. Well I've tried Lubuntu, and Xubuntu but all of those do the same thing even when they claim to be 1GB of ram min. requirement. And if I try an older version, like lubuntu 16, I get:
./isorespinner.sh -i lubuntu-16.04.6-desktop-amd64.iso -b GRUB-32 -f linuxium-install-UCM-files.sh
isorespinner.sh: WARNING: Insufficient memory (only 0.5GB free compared to required minimum of 15GB) so checking disk space availability ...
isorespinner.sh: Extracting isorespinner files ...
isorespinner.sh: Extracting ISO ...
isorespinner.sh: ISO is EOL.
isorespinner.sh: Exiting ... ISO not created.
Is there any way I could install a decent linux distro on this old stick?
@Skylite: Have a look at my post 'Adding a 32-bit GRUB bootloader to boot and install ISOs'. You might be able to get something to work using that approach.
Thanks. I ended up installing ubuntu server 22.04 and then installed lxde
Hi, I just wanted to respin Ubuntu Sway Remix (ubuntusway.com) with the options '-b GRUB-32 -g intel_idle.max_cstate=1 -g fsck.mode=skip -p lz4 -c "sed 's/^COMPRESS=zstd/COMPRESS=lz4/' /etc/initramfs-tools/initramfs.conf"' but your script says that ISO structure not compatible with an Ubuntu...desktop ISO. How could I respin that ISO so that I can use it with my ASUS x205ta? Thanks a lot in advance!
@Daniel: Ubuntu Sway Remix is not built the same way as Ubuntu family ISOs so my script cannot be used on it.
@Linuxium: Thanks a lot for your answer. In the mean time I tested your respun ISOs of Lubuntu 22.04 and Ubuntu Jammy canary version, btw. thanks for respinning them!
Both ISOs can be installed without any issues and run pretty well on my x205ta however after a few minutes sound stops working and it just keeps on beeping for a minute or so and then continues working a few minutes until it beeps again. That's really a pity as otherwise everything works fine. But I guess it's a hardware issue.
Super cool script - I just finished respinning an atom image of xubuntu-20.04. I have a question and some feedback. First what is the huge blob of what looks like RSA data at the bottom of the script?
Feedback: when installing dpkg-dev instead of an apt download you can just copy it from iso-directory-structure/pool/main/d/dpkg/dpkg-dev_1.19.7ubuntu3.2_all.deb to save a download and increase compatibility.
Thanks for your script!
@TroyW: The 'huge blob' is just a base64 encoded zip file containing the 32-bit boot manager and other files that are frequently added to an ISO without the need to have to downloading them. You can see from the script how it unpacks and uses them.
Regarding 'dpkg-dev' package, some *buntu ISOs don't (or didn't) include it under pool so it was easier just to download the relevant version regardless.
I should have been more specific. I was building xubuntu 20.04 on lubuntu 22, apt download dpkg-dev would not complete so I just copied it from the xubuntu iso dir.
@TroyW: What was the reason why 'apt download dpkg-dev' failed?
@linuxium it wasn't able to find the package, I don't have the logs anymore, or I'd post the exact message but it was due to it not being able to find it in the remote repos
Post a Comment