[New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Kernel, Main, Utilities & Applications, Miscellaneous Devices.
evilsim
Posts: 12
Joined: Mon May 25, 2020 4:40 am
Has thanked: 9 times
Been thanked: 2 times

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by evilsim »

with Clonezilla

Interesting idea. I was thinking about trying CloneZilla or Macrium Reflect recently..

Macrium can expand a certain volume to any size based on available space on the destination drive/device, just before actually doing the restore.

It would also be good to use a serial console (if the mister has this? kind of like an enterprise network switch) to show where the boot process gets to, on cards which dont boot after running the migrate_sd.sh script through. Sometimes I can run the script a second time - and it works. This smells like a dying SD card.. The script is flawless it cant really mess up!

User avatar
MiSSes
Posts: 5
Joined: Mon May 25, 2020 8:56 pm
Has thanked: 4 times
Been thanked: 1 time

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by MiSSes »

I transferred my data from a 400 to a 512gb card. I guess all went well. My mister starts and reacts as it should. Thank you!
Now I'm looking at my files on my new 512 gb card and I see a folder "found.000".
It contains 667 files named "FILE0667.CHK" It contains 4,29gb of data.
What is this folder good for and was this folder on my old card as well?
I do believe that it wasn't before the migration.

jca
Top Contributor
Posts: 1911
Joined: Wed May 27, 2020 1:59 pm
Has thanked: 145 times
Been thanked: 454 times

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by jca »

It looks like there was some corruption on your SD card. These files contain the data from the corrupted files.

Fed
Posts: 40
Joined: Mon May 25, 2020 6:31 am
Has thanked: 9 times
Been thanked: 2 times

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by Fed »

Upgraded from a 256 to 512 card, worked great, thanks for this functionality !

heinzgruber
Posts: 35
Joined: Thu Mar 23, 2023 8:12 am
Has thanked: 15 times
Been thanked: 1 time

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by heinzgruber »

hi,
i've used migrate_sd.sh and migrate data from a 256 gb card to a 512 gb card. the operation finished successfully.

but now on both sd cards something seems wrong: my daemonbite adapter isn't working anymore. seems like migrate_sd broke Daemonbite support. the adapter get power and my wireless 8bitdo-controller connects but MiSTer doesn't recognize inputs neither in games nor in the menu. When i open up the Definde Joystick Button menu no input is recognized either.

Also the "Switch to USB" menu entry is greyed out. So I guess something with the USB communication is wrong. But my via usb connected bluetooth and snac adapter are still working.

Edit:
Already tried d different USB ports und rund Update_all but the problem is still there.

jca
Top Contributor
Posts: 1911
Joined: Wed May 27, 2020 1:59 pm
Has thanked: 145 times
Been thanked: 454 times

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by jca »

If you do not have any USB drive connected there is no reason to have "Switch to USB" enabled. As you see USB is working with your Bluetooth. SNAC adapters are not connected via USB but via User GPIO which uses a USB connector but is not USB.

heinzgruber
Posts: 35
Joined: Thu Mar 23, 2023 8:12 am
Has thanked: 15 times
Been thanked: 1 time

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by heinzgruber »

Okay. Thanks for your Tipps. So my deamonbite problem is very strange. Especially that it is on both sd cards after using the script.

heinzgruber
Posts: 35
Joined: Thu Mar 23, 2023 8:12 am
Has thanked: 15 times
Been thanked: 1 time

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by heinzgruber »

Suddenly my deamonbite adapter is working again so i've no idea whats happened. It never happened before. sorry for the confusion.

tomvandeusen
Posts: 1
Joined: Thu Oct 26, 2023 1:07 am

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by tomvandeusen »

Hi there,

I'm not sure if it's an issue with my MiSTer, but when I try to load your script in the Scripts menu it just flashes some text and goes back to the MiSTer main menu. I assume from other replies that this has some sort of GUI associated with it, and I've never used a script with a GUI. I ran update_all relatively recently but it still is having this issue. Is there something I'm completely missing? My SD Card is fat32 formatted and it's plugged in via USB, but this seems more like an issue of my MiSTer not even reading your script properly.

Thanks so much for your help, and apologies since I don't really run scripts aside from update_all

UPDATE - nevermind, I figured it out. I had it on a CRT and fb_terminal was set to 0 instead of 1 (probably some legacy thing, I've had a MisTer since 2019)

User avatar
Longtime4321
Posts: 103
Joined: Fri Aug 12, 2022 5:17 am
Has thanked: 8 times
Been thanked: 15 times

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by Longtime4321 »

I wouldn't reccomend interfacing with scripts over a CRT. Many don't work with one.

Fed
Posts: 40
Joined: Mon May 25, 2020 6:31 am
Has thanked: 9 times
Been thanked: 2 times

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by Fed »

Longtime4321 wrote: Thu Oct 26, 2023 6:01 pm

I wouldn't reccomend interfacing with scripts over a CRT. Many don't work with one.

Never had any issues using scripts on a CRT

MEGAMILK
Posts: 1
Joined: Sun Dec 10, 2023 4:41 pm

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by MEGAMILK »

Would it be alright to cancel the script half way through (by rebooting the MiSTer)? I want to rerun it without including the games folder as it’s only going 50Kb/s.

RowanDDX
Posts: 2
Joined: Fri Feb 11, 2022 12:30 am
Has thanked: 1 time

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by RowanDDX »

Worked a treat! Thanks.

Took 1hr25min to copy what was basically a full 32GB card over to a new Sandisk 128GB card. Lots of little files, obviously takes ages.

Only thing that tripped me up initially was that the script failed because I hadn't formatted the microSD card first. Error given was "unable to successfully partition the disk". So I used Windows to format it to whatever the default was (exFat I think), then the script was happy after that. So to be pedantic, to say "no PC required" .. well the PC was required in this case to format the new card. Or maybe there's a MiSTer script to format cards?

lroby74
Posts: 113
Joined: Tue May 26, 2020 2:59 pm
Has thanked: 131 times
Been thanked: 15 times

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by lroby74 »

I've 3 Misters and each one has 32GB microSD with only arcades, ScummVM with games and nothing else.
All my games are on NAS so i can play them from any Mister and are protected with Raid 5 from losing them for faulty HDDs :D

User avatar
RealLarry
Top Contributor
Posts: 811
Joined: Mon May 25, 2020 4:04 am
Location: San Junipero/DE/Earth
Has thanked: 100 times
Been thanked: 353 times

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by RealLarry »

lroby74 wrote: Thu Dec 14, 2023 9:01 am

I've 3 Misters and each one has 32GB microSD with only arcades, ScummVM with games and nothing else.
All my games are on NAS so i can play them from any Mister and are protected with Raid 5 from losing them for faulty HDDs :D

That's it! Same here...but I'm having all cores and its stuff on a NAS. WiFi speed is fast enough to wait just a few seconds.

Contributor of tty2oled, author of tty2tft, tty2rpi and update_tty2xxx
disq
Posts: 1
Joined: Tue Dec 19, 2023 1:44 pm
Has thanked: 1 time

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by disq »

First time it failed to partition the card (brand new sandisk card, just unpackaged and stuck right in, not sure what kind of partition table was on it) but I insisted (by rebooting) and it's chugging along now. Thanks for the script!

User avatar
lagerfeldt
Posts: 162
Joined: Mon Jan 03, 2022 8:32 pm
Location: Denmark
Has thanked: 32 times
Been thanked: 82 times
Contact:

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by lagerfeldt »

Natrox wrote: Sun Oct 30, 2022 12:00 pm

I hope you will find this script useful. Please let me know if you have any questions, problems or nice words.

Works like a charm, thanks! Much easier than doing DD and having to copy block by block of unused data on a 512 GB card.

I'm linking to your script from here: https://misterreplay.com/vault.html

One thing that did surprise me a bit after inserting the newly migrated card for the first time into the MiSTer was that the startup sound and BGM didn't seem to work. Silence.

It appears that the first time a newly migrated card is inserted into the MiSTer and it's powered up it takes a bit of time before something syncs up (20-30 seconds in my case).

Any idea why this is so?

Patron of MiSTer + theypsilon + Jotego Team + Coin-Op Collection + Pierco + Nullobject

MiSTer RePlay website

Ultimate SID Collection website

User avatar
LamerDeluxe
Top Contributor
Posts: 1206
Joined: Sun May 24, 2020 10:25 pm
Has thanked: 853 times
Been thanked: 270 times

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by LamerDeluxe »

I had an odd bug where the script supposedly finished copying 400GB in one minute. The next time, I opted to do a manual selection of folders to copy and that seems to be working correctly. (got interrupted by MiSTer SAM, which I had turned off before, but sometimes it behaves a bit like a virus).

Tried to clone with AOMEI partition assistent before that. Chose exact clone, which, after five hours, I found out was the wrong decision. You can't resize exFAT partitions. It had placed the recovery(?) partition right between a 400GB sized one and the rest of the almost 600GB unused space. This turned out to be unfixable.

The quick option in AOMEI might have worked correctly (it mentions something about being able to resize a partition). I'm sure it would also take five hours. Though I expect this migration utility to take much longer, judging by the average copying speed. Major advantage being that it should partition correctly. And it's better to leave the MiSTer on than a whole PC, just to copy files.

Psygho
Posts: 34
Joined: Thu Nov 19, 2020 11:46 pm
Has thanked: 2 times

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by Psygho »

Is t normal to display nothing ? First a got lots of scrolling text... Wich files are copied.. Had to leave for an 45min, so i put tv off. and switched it on.....
But no screen.... and mine mster a is been showing the green and red led alternating, dim, bright dim, and alternate between green and red.

Come from a 200gb filled 256 micro sd, to a 512mb micro sd.

Update
Well it is not. Reset mine mister, and choose manual, en then run, I can see wich file is copied and how fast it is in doeing so..
Runs for more then 2 hours, so i let it run overnight..

Update 2
Let it run overnight, and get a message that X amount was copied, with 4mb/s.
Guess all went perfect the second time

User avatar
AfterBurnett
Posts: 32
Joined: Sun Mar 13, 2022 7:40 am
Has thanked: 6 times
Been thanked: 2 times

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by AfterBurnett »

You absolute legend! I am currently migrating from a 512GB card to a 1TB so I guess the MiSTer is gonna be busy for quite some time, haha. Thanks heaps for this!!!!

rhester72
Top Contributor
Posts: 1205
Joined: Thu Jun 11, 2020 2:31 am
Has thanked: 14 times
Been thanked: 183 times

Re: [New version!] SD Card Migration Utility for MiSTer: An easy way to migrate to a new SD card, no PC required

Unread post by rhester72 »

Migrated from 1TB to 1.5TB without issue - thank you!

Post Reply