SafeSysUpdater vs. sysUpdater on partial downgrades, or SafeSysUpdater is not really safe when it partial downgrades
I asked my brother (finishing his Statistics and Computer Science degrees) to run through some numbers we extracted from the logs in #3dshacks @ Rizon, our IRC chat. The stats are in a PDF here.
Note, the analysis does nothing to show success rates, it just analyses what happens when a partial downgrade takes place. The null-hypothesis test assumes both fail the same way. Since there are two main failure types (able to access browser, and not being able to access browser) the analysis assumes both are equally as likely to happen in the case both updaters are the same. The data I parsed from the #3dshacks logs (I never want to read so many logs again, and I only got through the first two weeks or so of data) indicated 14 safesysupdater partial downgrades, and 5 sysupdater partial downgrades, all of these on N3DSs (looks like N3DSs are more prone to failure, but this is not analyzed). Before people start saying the sample size is too small, read the analysis and/or take it up to a statistician, then come back and argue with me :).
The null-hypothesis is that both updaters are the same and thus should fail equally. The summary of the null-hypothesis test is that there is a 3% chance of rejecting the null-hypothesis incorrectly when looking at the sysupdater numbers, and a way under 1% chance when looking at the safesysupdater numbers. There is another analysis that is beyond what I know of statistics, but according to my brother it just indicates that sysupdater fails one way, safesysupdater fails another. In other words, they're not equal. sysupdater failures let users still access the browser and settings menu, but safesysupdater failures do not. In addition, I took note that sysupdater downgrades NATIVE_FIRM nearly last, while safesysupdater does not (more like near the 20th CIA), which indicates that they do not behave the same. In conclusion, safesysupdater is not really safe in the case of a partial downgrade, while sysupdater, while more difficult to get going, partials in a safer way. In the case of a success, both work the same way, though, as far as we can tell in #3dshacks.
tl;dr For the love of the gods, don't use safesysupdater to downgrade, only to check checksums, unless you like suffering and/or have Oot3dhax set up for 9.2 or Ninjhax.
EDIT: This is a known good copy of sysUpdater. It is fickle to start it (takes a long time to get it going due to it freezing a lot), but once it gets going it's usually good. If it fails halfway through the downgrade, statistically speaking one should still be able to access the browser and settings.
Posted Feb 21, 2016, 8:55 PM UTC · edited