Init rcS Speed / Faster MiSTer Boot Time

Kernel, Main, Utilities & Applications, Miscellaneous Devices.
Akuma
Posts: 138
Joined: Sat Dec 11, 2021 9:50 pm
Has thanked: 24 times
Been thanked: 45 times

Init rcS Speed / Faster MiSTer Boot Time

Unread post by Akuma »

Just curious, but why are we not using the ".sh" extension for /etc/init.d/ scripts?

When searching for shell scripts in /etc/init.d with grep "^#!" /etc/init.d/S??* we find that all of them are scripts:

Code: Select all

/etc/init.d/S01syslogd:#!/bin/sh
/etc/init.d/S02klogd:#!/bin/sh
/etc/init.d/S10udev:#!/bin/sh
/etc/init.d/S30dbus:#!/bin/sh
/etc/init.d/S31bluetooth:#!/bin/sh
/etc/init.d/S40network:#!/bin/sh
/etc/init.d/S41dhcpcd:#!/bin/sh
/etc/init.d/S49ntp:#!/bin/sh
/etc/init.d/S50proftpd:#!/bin/sh
/etc/init.d/S50sshd:#!/bin/sh
/etc/init.d/S91smb:#!/bin/sh
/etc/init.d/S99user:#!/bin/sh
It clearly says # Source shell script for speed. in /etc/init.d/rcS

Code: Select all

#!/bin/sh


# Start all init scripts in /etc/init.d
# executing them in numerical order.
#
for i in /etc/init.d/S??* ;do

     # Ignore dangling symlinks (if any).
     [ ! -f "$i" ] && continue

     case "$i" in
	*.sh)
	    # Source shell script for speed.
	    (
		trap - INT QUIT TSTP
		set start
		. $i
	    )
	    ;;
	*)
	    # No sh extension, so fork subprocess.
	    $i start
	    ;;
    esac
done
In my case it saves 30+ seconds

dmesg (before):

Code: Select all

...
[  119.387871] EXT4-fs (loop8): re-mounted. Opts: (null). Quota mode: disabled.
[  119.387919] ext4 filesystem being remounted at / supports timestamps until 2038 (0x7fffffff)
dmesg (after)

Code: Select all

...
[   85.368526] EXT4-fs (loop8): re-mounted. Opts: (null). Quota mode: disabled.
[   85.368563] ext4 filesystem being remounted at / supports timestamps until 2038 (0x7fffffff)
What am I missing?
rhester72
Top Contributor
Posts: 1107
Joined: Thu Jun 11, 2020 2:31 am
Has thanked: 13 times
Been thanked: 169 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by rhester72 »

The "S" isn't for Speed, it is for Shutdown. Since there is no formal shutdown on MiSTer, those scripts will never be called.
Akuma
Posts: 138
Joined: Sat Dec 11, 2021 9:50 pm
Has thanked: 24 times
Been thanked: 45 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Akuma »

rhester72 wrote: Sat Jun 25, 2022 3:20 pm The "S" isn't for Speed, it is for Shutdown. Since there is no formal shutdown on MiSTer, those scripts will never be called.
No, that "S" is for "single user mode", did you understand the question? Let me elaborate: If scripts are renamed to .sh in the /etc/init.d/ folder they are "sourced" rather than "forked". Sourcing is faster.

The question remains, why are those scripts forked and not sourced?
rhester72
Top Contributor
Posts: 1107
Joined: Thu Jun 11, 2020 2:31 am
Has thanked: 13 times
Been thanked: 169 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by rhester72 »

My apologies, you're right - K is the designator for 'Klll' and S is "Start'.

That the case, what difference does it make what the extension is? You can source a script that _doesn't_ end in .sh.
MrMartian
Posts: 16
Joined: Wed Nov 24, 2021 4:04 pm
Has thanked: 14 times
Been thanked: 21 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by MrMartian »

His point is that the current startup scripts will source another startup script that ends in .sh, as opposed to forking it. If the scripts were renamed, the system could start faster.
Malor
Top Contributor
Posts: 860
Joined: Wed Feb 09, 2022 11:50 pm
Has thanked: 64 times
Been thanked: 194 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Malor »

I tried this, and I don't see any difference in boot speed.

Note: don't be dumb like me, and rename everything in the dir to filename.sh. rcS and rcK can't have an .sh extension, or none of the scripts run. If you were using ssh, you'll have to visit the console to unstick it.
User avatar
aberu
Core Developer
Posts: 1144
Joined: Tue Jun 09, 2020 8:34 pm
Location: Longmont, CO
Has thanked: 244 times
Been thanked: 388 times
Contact:

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by aberu »

It's never taken 30 seconds for my MiSTer to boot. It takes a few seconds. When you say it's taking longer, do you mean it's taking longer to do other things like bring up wifi, bluetooth, usb storage, etc...?
birdybro~
Akuma
Posts: 138
Joined: Sat Dec 11, 2021 9:50 pm
Has thanked: 24 times
Been thanked: 45 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Akuma »

Malor wrote: Sun Jun 26, 2022 11:13 pm I tried this, and I don't see any difference in boot speed.

Note: don't be dumb like me, and rename everything in the dir to filename.sh. rcS and rcK can't have an .sh extension, or none of the scripts run. If you were using ssh, you'll have to visit the console to unstick it.
Yes that can be problematic, if I go by memory its: cpu, u-boot, kernel, init, rcS. You can see when its finished by looking at your dmesg output. (just type 'dmesg' in the console/ssh session)
aberu wrote: Mon Jun 27, 2022 5:58 am It's never taken 30 seconds for my MiSTer to boot. It takes a few seconds. When you say it's taking longer, do you mean it's taking longer to do other things like bring up wifi, bluetooth, usb storage, etc...?
The boot process, as you know, has an FPGA part which is quick, and a Linux part that is slower. Its the Linux part that can be optimized. As you can see from my first post it takes some time for the Linux part to finish.

The latter can also be split in two parts:

Some things can be experienced by the user, like moving the S45bluetooth before S40network in my case I renamed it to S31bluetooth. Which means that you can use your bluetooth controllers and don't have to wait before S40network is completed. (this is really noticeable when using WIFI)

Other things are not as noticeable, like the sourcing/forking of init.d scripts or when the boot process is completed in the background. But they are still faster and I was wondering why this was not used. (I assumed there could be a valid reason)

Combined however they still deliver a faster boot process and better user experience imo.
Akuma
Posts: 138
Joined: Sat Dec 11, 2021 9:50 pm
Has thanked: 24 times
Been thanked: 45 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Akuma »

forked_vs_sourced.png
forked_vs_sourced.png (23.9 KiB) Viewed 3200 times
Forked vs Sourced seems negligible


bluetoothd.png
bluetoothd.png (32 KiB) Viewed 3200 times
Putting S45bluetooth before S40network, is noticeably faster when using bluetooth controllers and wifi

Without WIFI, ~7 seconds (you can play as soon as you see the MiSTer menu)

Code: Select all

[    5.932637] EXT4-fs (loop0): recovery complete
[    5.934892] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: disabled.
[    5.934967] ext4 filesystem being mounted at /var/lib/bluetooth supports timestamps until 2038 (0x7fffffff)
[    6.559570] Boot/Init.d sequence complete
With WIFI, it takes ~17 seconds

Code: Select all

[   15.508133] EXT4-fs (loop0): recovery complete
[   15.510023] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: disabled.
[   15.510095] ext4 filesystem being mounted at /var/lib/bluetooth supports timestamps until 2038 (0x7fffffff)
[   16.193493] Boot/Init.d sequence complete
I'm going to investigate, that seems pretty slow (might be normal, I don't know yet)
Malor
Top Contributor
Posts: 860
Joined: Wed Feb 09, 2022 11:50 pm
Has thanked: 64 times
Been thanked: 194 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Malor »

Akuma wrote: Mon Jun 27, 2022 8:08 am Other things are not as noticeable, like the sourcing/forking of init.d scripts or when the boot process is completed in the background. But they are still faster and I was wondering why this was not used. (I assumed there could be a valid reason)
It was probably overlooked because it barely matters. That's a holdover from the Linux-on-80386 days, when the CPU was dismally slow. Even the modest ARM CPU on the DE-10 is fast enough, and has enough RAM, that sourcing versus forking is pretty much irrelevant.

Reordering the boot sequence seems worth working on, and if the scripts can be renamed as .sh for the next release without fouling anything up, might as well, but if there's any pain involved, using .sh scripts is probably not worth doing.

FWIW, renaming everything but rcS and rcK worked fine for me, there just wasn't any noticeable difference from the time I flipped the power switch to when I saw the network icon and time in the title bar. (which is what I consider fully booted.) It seems that the biggest things I'm waiting for are DHCP and NTP; the script execution time doesn't even register.
User avatar
aberu
Core Developer
Posts: 1144
Joined: Tue Jun 09, 2020 8:34 pm
Location: Longmont, CO
Has thanked: 244 times
Been thanked: 388 times
Contact:

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by aberu »

You said it saves you 30 seconds but then your graphs from testing shows 1-2 seconds saved, then you say it takes 7-17 seconds. I'm kinda confused about the data you presented and your findings so far. But hey, if there's no downside to changing it and it's for the better, I would PM sorgelig and see if he would be open to accepting a pull request on it.
birdybro~
Akuma
Posts: 138
Joined: Sat Dec 11, 2021 9:50 pm
Has thanked: 24 times
Been thanked: 45 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Akuma »

aberu wrote: Wed Jun 29, 2022 1:24 am You said it saves you 30 seconds but then your graphs from testing shows 1-2 seconds saved, then you say it takes 7-17 seconds. I'm kinda confused about the data you presented and your findings so far. But hey, if there's no downside to changing it and it's for the better, I would PM sorgelig and see if he would be open to accepting a pull request on it.
Yeah, I used the EXT4 message as a marker. Apparently that does not always gets triggered, but certainly gets triggered if you SSH into the box. So that 30s was my bad. Now I just write a tag to "/dev/kmsg" when rcS has completed. Which you can see in "dmesg".

/etc/init.d/S99z_done

Code: Select all

#!/bin/sh
echo "Boot/Init.d sequence complete" > /dev/kmsg
Malor wrote: Mon Jun 27, 2022 5:18 pm
Akuma wrote: Mon Jun 27, 2022 8:08 am Other things are not as noticeable, like the sourcing/forking of init.d scripts or when the boot process is completed in the background. But they are still faster and I was wondering why this was not used. (I assumed there could be a valid reason)
It was probably overlooked because it barely matters. That's a holdover from the Linux-on-80386 days, when the CPU was dismally slow. Even the modest ARM CPU on the DE-10 is fast enough, and has enough RAM, that sourcing versus forking is pretty much irrelevant.

Reordering the boot sequence seems worth working on, and if the scripts can be renamed as .sh for the next release without fouling anything up, might as well, but if there's any pain involved, using .sh scripts is probably not worth doing.

FWIW, renaming everything but rcS and rcK worked fine for me, there just wasn't any noticeable difference from the time I flipped the power switch to when I saw the network icon and time in the title bar. (which is what I consider fully booted.) It seems that the biggest things I'm waiting for are DHCP and NTP; the script execution time doesn't even register.
I found out that we could scrape a couple of seconds of the DHCP, by adding "noarp" to /etc/dhcpcd.conf.
But I still have to do some testing.

manual
Speed up DHCP by disabling ARP probing

dhcpcd contains an implementation of a recommendation of the DHCP standard (RFC 2131) to verify via ARP if the assigned IP is not used by something else. This is usually not needed in home networks, so it is possible to save about 5 seconds on every connect by disabling it:


/etc/dhcpcd.conf

Code: Select all

noarp
Akuma
Posts: 138
Joined: Sat Dec 11, 2021 9:50 pm
Has thanked: 24 times
Been thanked: 45 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Akuma »

I had to shuffle things around and change rcS and some other scripts to get it working. But I got it down to under 10s for my specific setup, including bluetooth and wifi. (it remains to be seen if this is stable and applicable to the general MiSTer setup)

Is anyone here an expert in UDEV?

Code: Select all

[    1.084770] rcS started: /etc/rc.d/S00boot
[    1.377497] rcS started: /etc/rc.d/S00mister
[    1.407166] rcS started: /etc/rc.d/S01syslogd
[    1.474271] rcS started: /etc/rc.d/S02klogd
[    1.534127] rcS started: /etc/rc.d/S10udev
[    2.283119] rcS started: /etc/rc.d/S30dbus
[    2.849796] rcS started: /etc/rc.d/S31bluetooth
[    4.165037] rcS started: /etc/rc.d/S40network
[    4.220138] rcS started: /etc/rc.d/S41dhcpcd
[    8.235817] rcS started: /etc/rc.d/S49ntp
[    8.424146] rcS started: /etc/rc.d/S50proftpd
[    8.583039] rcS started: /etc/rc.d/S50sshd
[    8.826625] rcS started: /etc/rc.d/S91smb
[    8.906282] rcS started: /etc/rc.d/S92mouse
[    8.998757] rcS started: /etc/rc.d/S98user
[    9.049782] rcS started: /etc/rc.d/S99done
[    9.067168] Boot/Init.d sequence complete
edit:

I rebooted the MiSTer a couple of times to get an average of ~9 seconds:
Attachments
boot_times.png
boot_times.png (28.14 KiB) Viewed 3034 times
Malor
Top Contributor
Posts: 860
Joined: Wed Feb 09, 2022 11:50 pm
Has thanked: 64 times
Been thanked: 194 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Malor »

I've been fooling around a little bit with this today. My Mister has only wired attachments; it's the DE10, 128MB, Digital I/O board, and an external USB switchbox.

My base boot speed was around 6.75 seconds.

Adding noarp to dhcpd.conf increased boot time a bit, to 6.82 seconds.

Moving to .sh files further increased boot times very slightly, to around 6.88 seconds.

My big win on boot speed: going to static assignment on the wired interface, instead of using DHCP. I added this to /etc/network/interfaces:

Code: Select all

auto eth0
iface eth0 inet static
    address A.B.C.15
    netmask 255.255.255.0
    gateway A.B.C.1
I then removed the /etc/resolv.conf symbolic link, and replaced it with a static file pointed at my nameserver. You can copy the existing file somewhere (maybe /root), remove the symlink, and then copy the file back in again.

I then removed the S41dhcpcd file... I moved it to /root.

Result: Mister comes up with the network icon already lit up. My dmesg start time is now 5.35 seconds, but the time saved at the Mister menu, waiting for the network icon, is probably more like 8 or 10 seconds.

The main downside to these changes is that the Mister doesn't use DHCP anymore, so I can't just walk it to another network and have it instantly work. I would have to manually reconfigure it with a new static IP.

There's still a pretty noticeable wait for NTP to kick in and the time to show in the Mister title bar. I tried building another start file to use ntpdate to instantly set the clock, but that slowed boot way down, to like 12 seconds, and then the main Mister screen still waited for NTP sync. I think this delay may be fixable only with code changes to the main Mister binary, or with an RTC(clock) module.
Akuma
Posts: 138
Joined: Sat Dec 11, 2021 9:50 pm
Has thanked: 24 times
Been thanked: 45 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Akuma »

dmesg output has some variability to it, so taking an average would give you a better view.
EDIT:
I would replace ntpd with ntpdate <server>, its fast and it only needs to run once.. This should run in the background and it does not seem to matter if "ntpdate" or "ntpd" is used. Maybe if you run a NTP-server on your local router, which should be quite fast.

EDIT2: especially when you use an IP-address instead of DNS.


Starting ntpdate: OK
real 0m0.100s
user 0m0.028s
sys 0m0.048s

You could use chmod 000 <file> instead of moving files to /root. Could you test what happens when you disable the network service? I think its not needed at all :D. I am almost done rewriting all init scripts and it looks like its paying off:
Attachments
boot_times.png
boot_times.png (21.74 KiB) Viewed 2967 times
Flandango
Core Developer
Posts: 388
Joined: Wed May 26, 2021 9:35 pm
Has thanked: 41 times
Been thanked: 328 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Flandango »

Don't need to move the S files out of there....simply rename them by adding a _ in front of it (ex S41dhcpcd -> _S41dhcpcd) and it will get ignored during bootup.
Akuma
Posts: 138
Joined: Sat Dec 11, 2021 9:50 pm
Has thanked: 24 times
Been thanked: 45 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Akuma »

These are the results for a BT/WIFI-based setup:

Code: Select all

Quickest boot time: 6.97
 Average boot time: 8.19
 Slowest boot time: 8.81
Attachments
boot_average.png
boot_average.png (34.74 KiB) Viewed 2918 times
Malor
Top Contributor
Posts: 860
Joined: Wed Feb 09, 2022 11:50 pm
Has thanked: 64 times
Been thanked: 194 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Malor »

Ok, I have really butchered my startup to try to make it go faster. I'm using only wired interfaces here, so this is what I've done so far:

Disable /etc/init.d/S41dhcpcd.sh, S45bluetooth.sh, and S49ntp.sh by putting underlines in front.(thanks for the reminder, I'd forgotten that anything not starting with S won't be run during startup.)

Commented out the wlan0 and wlan1 clauses from /etc/network/interfaces, and put in a clause for eth0 with static addressing, per above.

Created a static resolv.conf by copying the symlinked file created by dhcp into /root, removing the symlink, copying the file back in as a file, and making sure that all users had read permissions.

I tried to create an S48ntpdate script, but I found this didn't work; apparently the network takes a little while to come up, and ntpdate was running too soon. So I created a script called ntpdate.sh in /etc/network/ifup-d. Here it is, in its staggering complexity:

Code: Select all

#!/bin/bash

ntpdate -b -s A.B.C.D &
.... substitute an NTP server IP for A.B.C.D. -b means 'boot mode', where it just sets the time, no matter how out of sync it is, and exits. -s means to use syslog for any messages. Make sure the file is set executable, or it won't run.

By putting it in if-up.d, it's guaranteed to run after the network is operational. The mount_cifs script puts a script that runs itself in that same dir, which mounts remote SMB filesystems.

I think that's everything. My 100% wired Mister now has a dmesg startup time of 4.8 to 4.9 seconds, testing about five times. With clumsy manual timing, from the instant I hit return on a 'reboot' command to seeing the Mister startup screen is about 8 seconds; actual startup would be a second or two faster, but it's hard for me to hit the physical switch to get a more accurate result. It is 16 seconds from enter after reboot until the time displays in the Mister menu.

I'm not sure where that 8-second delay is coming from.
Malor
Top Contributor
Posts: 860
Joined: Wed Feb 09, 2022 11:50 pm
Has thanked: 64 times
Been thanked: 194 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Malor »

BTW, I don't really think that anything I'm doing here should be brought into the default install, except *maybe* for the ntpdate thing. The default setup is very flexible, and works in almost any network, meaning that novices will have a nice easy time with it. That's very important.

Butchering things like I've done here should be saved for advanced users who want faster bootups, and understand enough to rename files and edit scripts from the command line.

Maybe we could pull some of this info into the wiki?
Malor
Top Contributor
Posts: 860
Joined: Wed Feb 09, 2022 11:50 pm
Has thanked: 64 times
Been thanked: 194 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Malor »

As an aside, I've discovered that putting the config directory on SMB is a bad idea; a cold boot often doesn't have the share up by the time Mister starts, so it can't read its own settings correctly. It's best to keep config in /media/fat.
Akuma
Posts: 138
Joined: Sat Dec 11, 2021 9:50 pm
Has thanked: 24 times
Been thanked: 45 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Akuma »

I found out that udev and network are biting each other both triggering dhcpcd with ifup ifdown. So I rewrote network as well, which is only needed for lo & eth0 since wlan0 is triggered by udev.

It varies greatly with WIRED 7s~12s before the network is actually usable
With WIFI its always around 24s, so I dont think NTP is the problem, because the system just isn't ready yet.

And the weird thing is, everything seems ready after ~8 seconds with WIFI icon popping up a second or two later.

I used netcat --zero option to scan the DNS port on my router.

WARNING: this is an endless loop until the adapter comes up, (if it doesn't it hangs forever)

Code: Select all

start(){
        printf "starting network connection: "|tee /dev/kmsg
        while :; do
                nc -z -w2 <insert_router_ip> 53 > /dev/null 2>&1 && break
                sleep 0.1
        done
        echo OK |tee /dev/kmsg
}
Got a small performance gain :D (edit: and it now has the correct chart)

Code: Select all

Quickest boot time: 7.55
 Average boot time: 7.59
 Slowest boot time: 7.91
Attachments
20220704.png
20220704.png (36.72 KiB) Viewed 2795 times
Akuma
Posts: 138
Joined: Sat Dec 11, 2021 9:50 pm
Has thanked: 24 times
Been thanked: 45 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Akuma »

Malor wrote: Mon Jul 04, 2022 1:31 am Commented out the wlan0 and wlan1 clauses from /etc/network/interfaces, and put in a clause for eth0 with static addressing, per above.
This is the doing of UDEV rules. A default user would require lo, eth0 or wlan0. That needs to be looked at. The problem is that everywhere "ifup -a" is used, instead of ifup <link>. This will up all links instead of what is needed.(WIFI)

EDIT:
also dhcpcd and udhcpc are both used, I still have to figure out if this is in the original linux.img
Malor wrote: Mon Jul 04, 2022 1:31 am I tried to create an S48ntpdate script, but I found this didn't work; apparently the network takes a little while to come up, and ntpdate was running too soon.
Ran into that myself, also related to UDEV rules/ifup -a/dhcpcd
Malor wrote: Mon Jul 04, 2022 1:31 am By putting it in if-up.d, it's guaranteed to run after the network is operational. The mount_cifs script puts a script that runs itself in that same dir, which mounts remote SMB filesystems.
If solved this another way, but I hope I can translate it back to if-pre-up.
Malor wrote: Mon Jul 04, 2022 1:31 am I'm not sure where that 8-second delay is coming from.
That clears up after UDEV rules is properly sorted.
Malor wrote: Mon Jul 04, 2022 1:46 am BTW, I don't really think that anything I'm doing here should be brought into the default install, except *maybe* for the ntpdate thing. The default setup is very flexible, and works in almost any network, meaning that novices will have a nice easy time with it. That's very important.

Butchering things like I've done here should be saved for advanced users who want faster bootups, and understand enough to rename files and edit scripts from the command line.
Good point, that's why I decided not to post my scripts yet, until I have some 1on1 with a dev.

But any user could backup their MiSTer_data/linux/linux.img to MiSTer_data/linux/linux.img.bak. In case anything happens, pop out the sdcard and restore the file behind a pc.
Malor wrote: Mon Jul 04, 2022 1:46 am Maybe we could pull some of this info into the wiki?
What would you like to post there ?
Malor wrote: Mon Jul 04, 2022 7:59 am As an aside, I've discovered that putting the config directory on SMB is a bad idea; a cold boot often doesn't have the share up by the time Mister starts, so it can't read its own settings correctly. It's best to keep config in /media/fat.
Well it can be done, but that would mean a boot delay.You'd still be waiting on the network in order to use MiSTer.
But some people may be interested in this.
Akuma
Posts: 138
Joined: Sat Dec 11, 2021 9:50 pm
Has thanked: 24 times
Been thanked: 45 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Akuma »

Seems udev, ifup, dhcpcd, network scripts were fighting, once that got resolved its been rather stable, however it seems I have some network congestion (transmit data). And its this part that will vary for everyone depending on the environment.

WIRED: ~12s

Code: Select all

Quickest boot time: 8.17
 Average boot time: 12.05
 Slowest boot time: 22.54
wired.png
wired.png (40.66 KiB) Viewed 2763 times
WIFI: ~15s

Code: Select all

Quickest boot time: 8.13
 Average boot time: 14.68
 Slowest boot time: 22.76
wireless.png
wireless.png (39.26 KiB) Viewed 2763 times
Malor
Top Contributor
Posts: 860
Joined: Wed Feb 09, 2022 11:50 pm
Has thanked: 64 times
Been thanked: 194 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Malor »

For some reason, my ntpdate trick just stopped working; it suddenly started getting launched before the interface was actually ready. This was true even if I listed it as a post-up clause in /etc/network/interfaces. It just refuses to do an initial sync.

I ended up giving up on that idea and re-enabling the regular NTP.
Akuma
Posts: 138
Joined: Sat Dec 11, 2021 9:50 pm
Has thanked: 24 times
Been thanked: 45 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Akuma »

Malor wrote: Thu Jul 14, 2022 3:23 pm For some reason, my ntpdate trick just stopped working; it suddenly started getting launched before the interface was actually ready. This was true even if I listed it as a post-up clause in /etc/network/interfaces. It just refuses to do an initial sync.

I ended up giving up on that idea and re-enabling the regular NTP.
You also have to power off the system, to check that it seems. Date/time is persistent with reboots, go figure. You could build in a delay, something like:

Code: Select all

( sleep 5; ntpdate 0.pool.ntp.org;) &
That NTP daemon is active because it is unknown if/when there is internet, users can plug in that wifi dongle any time. Which is also the reason why DHCPCD is used.
Akuma
Posts: 138
Joined: Sat Dec 11, 2021 9:50 pm
Has thanked: 24 times
Been thanked: 45 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Akuma »

Conclusions of this little research project:
  • if .sh files are sourced, their script name is lost in the process. (more difficult to troubleshoot)
  • bluetooth, mouse (eg: peripherals), before networking, so "controllers" are available as soon as possible
  • dhcpcd package is not needed, udhcpc works just fine with some additional configuration
  • static configuration, can have some variability on networks with active dhcp server
  • it takes a while before wlan0 is ready ~4+s
  • it takes a while before dhcp lease is active ~4+s
  • if a dhcp lease is not released, the next request for the same lease will take longer.
These are single boot-ups , I did not average them, so they can vary a little bit, I'm sure.
Attachments
chart.png
chart.png (30.86 KiB) Viewed 2512 times
Akuma
Posts: 138
Joined: Sat Dec 11, 2021 9:50 pm
Has thanked: 24 times
Been thanked: 45 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by Akuma »

Scripting is almost done, Scripting still is a work in progress, and this is what it looks like now:


EDIT: direct video play, i love this forum :D
Attachments
init.mp4
(8.02 MiB) Downloaded 111 times
prenetic
Posts: 41
Joined: Mon Jun 06, 2022 7:06 am
Has thanked: 23 times
Been thanked: 8 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by prenetic »

Akuma wrote: Mon Jul 04, 2022 1:46 pm I found out that udev and network are biting each other both triggering dhcpcd with ifup ifdown. So I rewrote network as well, which is only needed for lo & eth0 since wlan0 is triggered by udev.
I've been noticing really strange network behavior that might be related to this. My DHCP server shows two leases for my Wi-Fi adapter's MAC address -- one named "MiSTer" and the other unnamed. When rebooting the MiSTer, I notice there's maybe a 50/50 chance it'll show either IP address in the Menu core and ifconfig. Network connectivity in general is also a little unstable while this all settles during the first minute or so.

To be clear, I'm just running the stock MiSTer image -- no changes. I know this mentions wlan0 is triggered by udev alone, but I haven't really come to any other conclusions so far. What's also strange is despite only showing one IP address bound to the wlan0 interface with ifconfig, MiSTer responds to the .68 address as well.

Code: Select all

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.10.66  netmask 255.255.255.0  broadcast 192.168.10.255
        ether b4:b0:24:29:08:21  txqueuelen 1000  (Ethernet)
        RX packets 39549  bytes 13755172 (13.1 MiB)
        RX errors 0  dropped 76  overruns 0  frame 0
        TX packets 9195  bytes 2014996 (1.9 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Code: Select all

PS C:\Users\user> ping mister

Pinging mister.domain [192.168.10.68] with 32 bytes of data:
Reply from 192.168.10.68: bytes=32 time=67ms TTL=64
Reply from 192.168.10.68: bytes=32 time=85ms TTL=64
Reply from 192.168.10.68: bytes=32 time=4ms TTL=64
Reply from 192.168.10.68: bytes=32 time=26ms TTL=64
Screenshot 2022-08-13 131658.png
Screenshot 2022-08-13 131658.png (21.15 KiB) Viewed 2321 times
User avatar
RealLarry
Top Contributor
Posts: 768
Joined: Mon May 25, 2020 4:04 am
Location: San Junipero/DE/Earth
Has thanked: 86 times
Been thanked: 329 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by RealLarry »

prenetic wrote: Sat Aug 13, 2022 8:18 pm I've been noticing really strange network behavior that might be related to this. My DHCP server shows two leases for my Wi-Fi adapter's MAC address -- one named "MiSTer" and the other unnamed. When rebooting the MiSTer, I notice there's maybe a 50/50 chance it'll show either IP address in the Menu core and ifconfig. Network connectivity in general is also a little unstable while this all settles during the first minute or so.
Sounds like a similar issue I had, too. My DHCP server never showed two leases, but my DHCP and DNS server nagged about an "already used lease" and refused to give a fresh/new FQDN. Turned out that S40network isn't releasing it's lease when rebooting MiSTer.
I wrote an issue about that, but it was declined: https://github.com/MiSTer-devel/Linux_I ... /issues/15
Contributor of tty2oled, author of tty2tft, tty2rpi and update_tty2xxx
prenetic
Posts: 41
Joined: Mon Jun 06, 2022 7:06 am
Has thanked: 23 times
Been thanked: 8 times

Re: Init rcS Speed / Faster MiSTer Boot Time

Unread post by prenetic »

RealLarry wrote: Sun Aug 14, 2022 4:09 am
prenetic wrote: Sat Aug 13, 2022 8:18 pm I've been noticing really strange network behavior that might be related to this. My DHCP server shows two leases for my Wi-Fi adapter's MAC address -- one named "MiSTer" and the other unnamed. When rebooting the MiSTer, I notice there's maybe a 50/50 chance it'll show either IP address in the Menu core and ifconfig. Network connectivity in general is also a little unstable while this all settles during the first minute or so.
Sounds like a similar issue I had, too. My DHCP server never showed two leases, but my DHCP and DNS server nagged about an "already used lease" and refused to give a fresh/new FQDN. Turned out that S40network isn't releasing it's lease when rebooting MiSTer.
I wrote an issue about that, but it was declined: https://github.com/MiSTer-devel/Linux_I ... /issues/15
Interesting, there are dozens of other devices on my network that don't get a chance to release their IP address before getting powered off (IoT/smart devices are a great example), and they gracefully start back up receiving their existing lease from the DHCP server -- it seems like the MiSTer should be able to do the same.

I just noticed I can see both addresses with "ip address", with the .66 address considered valid indefinitely (which seems strange since leases on my network are only valid for 24 hours). Tried removing it with "ip addr del 192.168.10.66/24 dev wlan0" to see if that takes care of it, but it came back after a reboot.

Code: Select all

3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether b4:b0:24:29:08:21 brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.66/24 brd 192.168.10.255 scope global wlan0
       valid_lft forever preferred_lft forever
    inet 192.168.10.68/24 brd 192.168.10.255 scope global secondary dynamic noprefixroute wlan0
       valid_lft 74946sec preferred_lft 64146sec
Also tried disabling the dhcpcd init.d script, and neither .66 nor .68 came up so I logged in via the console and noticed there was a new address all by itself (.65) but packet loss was 100% -- I had to re-enable the S41 init.d script and restart. Now I'm back up and running again, but once again have two addresses -- now .65 instead of .66, in addition to the original .68 address.

Code: Select all

3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether b4:b0:24:29:08:21 brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.65/24 brd 192.168.10.255 scope global wlan0
       valid_lft forever preferred_lft forever
    inet 192.168.10.68/24 brd 192.168.10.255 scope global secondary dynamic noprefixroute wlan0
       valid_lft 86303sec preferred_lft 75503sec
Perhaps someone with more experience can confirm whether this is expected behavior? Is this happening for anyone else's MiSTer and possibly not being enough of a hindrance to notice/be bothered by it? Seems like it should only be running with one address per interface, but maybe there's a reason for this.

Also, if this an issue separate from what was outlined in this thread please let me know and I can start a new thread and create an issue on GitHub if appropriate.
Post Reply