[DISCUSSION] Windows Subsystem for Linux (WSL)

Discussion in 'Windows 10' started by Tito, Jul 30, 2016.

  1. Tito

    Tito Super Mod / Adviser
    Staff Member

    Nov 30, 2009
    18,681
    18,589
    340
    Well, its all about gathering & sharing knowledge.

    :hug2:
     
  2. qwerty12

    qwerty12 MDL Novice

    Aug 31, 2010
    49
    68
    0
    #22 qwerty12, Jul 31, 2016
    Last edited by a moderator: Apr 20, 2017
  3. jygoro

    jygoro MDL Senior Member

    Sep 21, 2011
    254
    142
    10
    Latest Windows 10 "Create a system image" (sdclt.exe /BLBBACKUPWIZARD) feature also preserves EA on files and folders. I guess it also uses DISM? Anyway my backup from 19 July already has a lxss folder with its EA on it. :)
     
  4. Prince_Charles

    Prince_Charles MDL Novice

    May 10, 2007
    38
    33
    0
    #24 Prince_Charles, Jul 31, 2016
    Last edited: Jul 31, 2016
    @qwerty12

    Good solution. My approach (I don't know if it's feasible!) would have been to patch the respective urls in lxrun.exe to retrieve ubuntu.ico and trusty-server-cloudimg-amd64-root.tar.gz from a local web server.
     
  5. dmex

    dmex MDL Junior Member

    Apr 20, 2011
    95
    105
    0
  6. qwerty12

    qwerty12 MDL Novice

    Aug 31, 2010
    49
    68
    0
    Looks useful :)

    If you want to install WSL from scratch, sure. I assumed the reason for backing up an existing install is to preserve the updates, customisations etc. applied to it.

    If you want to use LxRun for the task, then sure. (Unless I overlooked something, which is likely.) Like I said earlier, it doesn't use the downloaded tarball as-is and makes changes to the files and appears to create a new Linux user if run by a non-administrative Windows user. I mean, sure, the latter is easily done but for the former, looks like you'd have to maintain the modifications LxRun makes with every WSL release. Honestly, for me - and this is coming from someone who'll happily admit he knows very little about REing stuff - I think I'd be able to patch LxRun faster...
     
  7. T-S

    T-S MDL Guru

    Dec 14, 2012
    3,984
    1,331
    120
    Another good question could be what if you replace %LOCALAPPDATA%\lxss\ with a symlink (a windows one I mean) or a junction pointing somewhwere else before installing linux, will it install? Will it work?

    Mounting a sparate partition in the lxss folder could be a further test worth to try
     
  8. dmex

    dmex MDL Junior Member

    Apr 20, 2011
    95
    105
    0
    #28 dmex, Aug 2, 2016
    Last edited by a moderator: Apr 20, 2017
    Good idea. I'll look at adding something to backup and restore the folder.

    I've spent the last few hours RE'ing and performing traces - I wasn't able to find lxrun.exe modifying any files. From a developer standpoint this route of modifying files using their installer has some major pitfalls:

    1. Canonical would not be able to update their files without Microsoft first pushing an update via WU for the installer.
    2. These files are licenced under GPL and Microsoft hasn't made these changes available anywhere.

    #1 can get messy very quickly with both maintenance and reliability.
    #2 likely ends up with Microsoft violating the GPL licence.

    If there are changes then they were very likely made to the archive before release since it would simplify maintenance and patching. If their installer or anything else is modifying files then you would hope Microsoft or someone makes those changes available somewhere in order to comply with the GPL - which hasn't happened and makes me doubt that there are changes to these files.

    I could be wrong about this so if you do find modifications then post them here as it may help determine if Microsoft has violated the GPL.

    LxRun.exe creates a new user when executed by both administrators and standard users... It seems to be executing a script via bash.exe after installation.

    Note: I've likely trimmed too much 'garbage' from the string (including spaces by accident), I don't know bash scripting so uh... I could use some help cleaning this up (or getting the full script from LxRun.exe if I've trimmed way too much already) in order to create an installer for Fedora or other distributions :p

    Code:
    /bin/echo $'#!/bin/shexit 101' > /usr/sbin/policy-rc.d/bin/bash-c/bin/chmod 0744
    /usr/sbin/policy-rc.d/usr/bin/dpkg-divert --local --rename --add /sbin/initctl &> /dev/null ln -f -s /bin/true 
    /sbin/initctl &> /dev/null.UTF-8/usr/bin/apt update &> /dev/null/usr/bin/apt install -y language-pack-%s/usr/sbin/update-locale LANG=~rootfs\rootrootfs\home /usr/bin/aptupdate%d/usr/lib/update-notifier/apt-check --human-readable &> /usr/lib/update-notifier/apt-check-cache
    %lsHOSTTYPE=x86_64TERM=xtermLANG=en_US.UTF-8SHELL=/bin/bash 
    
    "/bin/bash -c ""/mnt//usr/sbin/adduser--quiet--force-badname--disabled-password--gecos" 
    
    /usr/sbin/usermod-aG/usr/sbin/deluser /usr/bin/id abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890_-%TEMP%\uid.txt/usr/bin/id -u  &> %TEMP%\gid.txt/usr/bin/id -g --uid--gidDefaultUidDefaultGidDefaultUsername/usr/bin/passwd/usr/sbin/addgroup/usr/sbin/delgroup
    The only other thing I could see being performed on the Windows side was this registry key being created:
    HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Lxss

    Values created:
    "DefaultUid"
    "DefaultGid"
    "DefaultUsername"

    bash.exe seems to use the "DefaultUid" during startup but not the other values (but I might have missed the other calls in my traces).

    I also noted the "C:\Windows\System32\lxss\LxssManager.dll" includes the 'init' ELF binary in its resource section(?) and it will likely overwrite the init included in the lxss directory... Init does the work of creating these files after startup of bash.exe:
    "zoneinfo"
    "timezone"
    "localtime"
    "resolv"

    I don't think we don't need to worry about what LxssManager and init does; So if we wanted to make an installer for other distributions like Fedora then it should be as simple as:

    1. Download rootfs.tar.gz
    2. Extracts tar contents into %LOCALAPPDATA%\lxss\ and populate the LXATTRB attributes based on the metadata stored in the archive.
    3. Execute "bash.exe -c <script_here>" to create the user and other post-install setup tasks.
    4. Set the above 3 registry keys based on the \lxss\rootfs\etc\passwd file for UID, GID and username.
    5. Run bash.exe and start using the environment like normal.

    It works both ways, you'll need to maintain your modifications after every Windows update :p
     
  9. T-S

    T-S MDL Guru

    Dec 14, 2012
    3,984
    1,331
    120

    I don't think so.

    From the ubuntu POV what counts is the root folder. It doesn't give a crap about how the NTFS fs is organized. Surely it's able to browse it, but linux sees the NTFS FS as a mounted device.

    The LXSS path is hardcoded in a single exe, so the only problem that can happen is an update to the exe itself that would restore the path to it's default, but at that point would be pretty easy to repatch it
     
  10. qwerty12

    qwerty12 MDL Novice

    Aug 31, 2010
    49
    68
    0
    #30 qwerty12, Aug 2, 2016
    Last edited by a moderator: Apr 20, 2017
    Not directly, but looking quickly at the disassembly, I remember seeing /usr/sbin/policy-rc.d policy mentioned. I assume it calls it in the WSL environment during install. I don't have the tarball and WSL install around anymore (rolled back a VM snapshot to test my crude "classic Sticky Notes" installer), but I'm pretty sure the WSL install on the filesystem had a /usr/sbin/policy-rc.d file that was simply a script (so no GPL violation here) that just exits. Unless my memory is failing me, I'm pretty sure policy-rc.d in the tarball is a binary. Oh, dpkg-divert is also called, presumably to stop a Ubuntu update installing it again. Actually, the bash script you posted also has the change so I guess you saw it too ;)

    Ah, I assumed that was for non-admins, 'cause I'm always signed in as root, and I don't recall any non-standard user being added to any groups in /etc/groups. I only took a cursory look at getent passwd, though...

    Sorry, my bash is rudimentary at best, and I need to focus the little energy I have left tonight on working out whatever hidden COM start menu API there is in Windows, 'cause I'm tired of unpinning the multitudes of s**t like Candy Crush MS won't get rid of for me myself when I import the Registry entries to disable the CDM :( I don't roll with modified ISOs and since my user is already created, I can't use the StartLayout PowerShell scriptlets, either. Unless there's an easier way (if Microsoft populated the Metro apps folder with those placeholder icons, I could programmatically invoke the unpin verb :(). Sorry, going off on a tangent here. I wish I had some drink around...

    EDIT: I had a go. Serious verification is needed, but if I didn't mess up, hopefully it's a start :)

    Code:
    #!/usr/bin/env bash
    export HOSTTYPE=x86_64 TERM=xterm LANG=en_US.UTF-8 SHELL=/bin/bash # I think these are set for every process spawned by LxRun. The "/bin/bash -c" lines shouldn't be needed; I assume they're there in LxRun.exe to make redirection work
    /bin/echo $'#!/bin/sh\nexit 101' > /usr/sbin/policy-rc.d
    /bin/chmod 0744 /usr/sbin/policy-rc.d
    /usr/bin/dpkg-divert --local --rename --add /sbin/initctl &> /dev/null
    ln -f -s /bin/true /sbin/initctl &> /dev/null
    # Some guesswork follows, verification needed
    somevar=%s # Language code determined from mapping the output of GetUserDefaultLocaleName()
    /usr/bin/apt update &> /dev/null
    /usr/bin/apt install -y "language-pack-$somevar" &> /dev/null
    /usr/sbin/update-locale LANG="$somevar.UTF-8"
    #~rootfs\rootrootfs\home I think some files are deleted in here, but I can't verify this until I install WSL again
    /usr/lib/update-notifier/apt-check --human-readable &> /usr/lib/update-notifier/apt-check-cache
    
    
    #Sorry, I don't understand how the user adding works, so I'll leave it. id is called before adduser, I think to determine if a UNIX user is already present
    exit 0
    
    #%ls 
    
    "/bin/bash -c ""/mnt//usr/sbin/adduser--quiet--force-badname--disabled-password--gecos" 
    
    /usr/sbin/usermod-aG/usr/sbin/deluser /usr/bin/id abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890_-%TEMP%\uid.txt/usr/bin/id -u  &> %TEMP%\gid.txt/usr/bin/id -g --uid--gidDefaultUidDefaultGidDefaultUsername/usr/bin/passwd/usr/sbin/addgroup/usr/sbin/delgroup
    Ah, didn't notice that! I wondered how WSL was going to be dealing with the very Linux kernel-centric systemd (if Ubuntu is using it currently)

    If that's in reply to my post saying that changes in the installer would have to be kept up to date, the anal-retentive in me still insists LxRun should be tracked once in a while, just to keep up with the modifications. I mean, your very post shows the commands it runs to setup the environment after the downloaded tarball is unpacked (including replacing at least one file - more files being replaced by MS in the future to retain compatibility doesn't seem unlikely to me). A shame this is all hardcoded in LxRun.exe...

    (Also, I know you're using Fedora as an example, and you did say "other distributions", but I'm still going to say it anyway: Prioritise Arch, plz. The slowness of yum and now dnf make me cry. apt and pacman on the hand fly. Plus, the hypocrisy of Fedora saying Chromium can't be distributed there because of the amount of bundled libraries it uses over the system's, and then coming out with Flatpak is not lost on me...)

    Oh, definitely, I just meant that it wouldn't take me long to find the new offsets (thank you, name mangling). What I posted was a hack anyway (I believe I was quite candid about the time it would take and the fact that it wouldn't hold up for future versions); your technical knowledge surpasses mine (I promise this is the last time that I'll swoon, but I've never liked Process Explorer while Process Hacker was good enough to make my default task manager on the first install), so it's good to see, hopefully, the beginnings of something better (I mean, no NSudo for something that writes to %LOCALAPPDATA% can only be a plus)! :cheers:
     
  11. moderate

    moderate MDL Guru

    Aug 31, 2009
    3,383
    2,479
    120
    #31 moderate, Aug 2, 2016
    Last edited by a moderator: Apr 20, 2017
  12. cruxq

    cruxq MDL Junior Member

    Mar 31, 2010
    78
    24
    0
    Heh, nice!
    Now, if only we could replace window's desktop manager with something else, gnome, KDE, whatever. :)
     
  13. T-S

    T-S MDL Guru

    Dec 14, 2012
    3,984
    1,331
    120
    You can since ages. Just download the native win32 kde and set it as a default shell
     
  14. Garbellano

    Garbellano MDL Addicted

    Aug 13, 2012
    947
    248
    30
    that looks insane
     
  15. jygoro

    jygoro MDL Senior Member

    Sep 21, 2011
    254
    142
    10
    Before build 14388 was released you could even use an external bash.vhdx file attached to an empty %AppData%/Local/lxss folder and it worked great.
    Since build 14388 you can install WSL but at the last stage when it runs bash.exe it says Error: 0x80070003. It seems now it tests the folder and if lxss is a symlink it refuses to run.

    Installing WSL after symlinking the LocalAppdata or AppData folders works though.
     
  16. biatche

    biatche MDL Addicted

    Nov 14, 2009
    569
    143
    30
    there a way to enable this in windows via dism?
     
  17. T-S

    T-S MDL Guru

    Dec 14, 2012
    3,984
    1,331
    120

    I can understand the poeple scared by reading a 500 pages thread, but this one....

    (This is for offline, if you need online adjust the command accordingly)

    Dism /Image:C:\test\offline /Enable-Feature /FeatureName:Microsoft-Windows-Subsystem-Linux /all
     
  18. biatche

    biatche MDL Addicted

    Nov 14, 2009
    569
    143
    30

    thanks, mind telling me what's the role of /all here?
     
  19. abbodi1406

    abbodi1406 MDL KB0000001

    Feb 19, 2011
    16,226
    84,921
    340
    to enable any parent features (WSL don't have any parent though)
     
  20. Mr.X

    Mr.X MDL Guru

    Jul 14, 2013
    8,575
    15,646
    270
    #40 Mr.X, Aug 2, 2016
    Last edited: Aug 2, 2016