Installing Windows Server 2019 from a remotely mounted ISO

Installing Windows Server 2019 from a remotely mounted ISO

[Alexander Amelkin]

[-- Attachment #1: Type: text/plain, Size: 479 bytes --]

Hi all!

We're trying to install Windows Server 2019 from an ISO image mounted as 
remote media via OpenBMC WebUI.

The Windows installer boots, but then says that a media driver is missing.

Are there any hints on how to accomplish the task?

Attached is the screenshot of the message we get with Windows Server 2019.

With best regards,
Alexander.

P.S. We're using Intel-BMC version of OpenBMC for the task, but it looks 
like the remote media part is more or less the same.


[-- Attachment #2: winserv2019-remote-media.jpg --]
[-- Type: image/jpeg, Size: 43397 bytes --]

Re: Installing Windows Server 2019 from a remotely mounted ISO

[Thaj]

[-- Attachment #1: Type: text/plain, Size: 673 bytes --]

Hi Alexander, I am also facing the same issue. Did you find any
workaround for this ?

On Mon, May 25, 2020 at 4:04 PM Alexander Amelkin <a.amelkin@yadro.com>
wrote:

> Hi all!
>
> We're trying to install Windows Server 2019 from an ISO image mounted as
> remote media via OpenBMC WebUI.
>
> The Windows installer boots, but then says that a media driver is missing.
>
> Are there any hints on how to accomplish the task?
>
> Attached is the screenshot of the message we get with Windows Server 2019.
>
> With best regards,
> Alexander.
>
> P.S. We're using Intel-BMC version of OpenBMC for the task, but it looks
> like the remote media part is more or less the same.
>
>

[-- Attachment #2: Type: text/html, Size: 1019 bytes --]

Re: Installing Windows Server 2019 from a remotely mounted ISO

[Maciej Lawniczak]

[-- Attachment #1.1: Type: text/plain, Size: 1054 bytes --]

Mounting DVD-based ISOs in Intel-BMC Virtual Media is not supported by 
underlying USB Mass Storage gadget. To mount such iso files they need to 
be converted to MBR/FAT32 image. Attached instruction shows two ways of 
preparing such image.

On 10/21/2020 9:32 AM, Thaj wrote:
> Hi Alexander, I am also facing the same issue. Did you find any 
> workaround for this ?
>
> On Mon, May 25, 2020 at 4:04 PM Alexander Amelkin <a.amelkin@yadro.com 
> <mailto:a.amelkin@yadro.com>> wrote:
>
>     Hi all!
>
>     We're trying to install Windows Server 2019 from an ISO image
>     mounted as
>     remote media via OpenBMC WebUI.
>
>     The Windows installer boots, but then says that a media driver is
>     missing.
>
>     Are there any hints on how to accomplish the task?
>
>     Attached is the screenshot of the message we get with Windows
>     Server 2019.
>
>     With best regards,
>     Alexander.
>
>     P.S. We're using Intel-BMC version of OpenBMC for the task, but it
>     looks
>     like the remote media part is more or less the same.
>

[-- Attachment #1.2: Type: text/html, Size: 36756 bytes --]

[-- Attachment #2: mounting_windows_iso.md --]
[-- Type: text/plain, Size: 5501 bytes --]

# Mouting Windows ISO image in OpenBMC VirtualMedia

OpenBMC implementation of VirtualMedia functionality is based on open source
 [Mass Storage Gadget](https://www.kernel.org/doc/html/latest/usb/mass-storage.html).
It has known limitation of no support for DVD ISO images.

By design VirtualMedia will mount all served images as USB Mass Storage device
for compatibility. While this doesn't affect modern ISO images for Linux
distributions, Windows installer is sensitive to the medium it's booted from.

To verify if ISO you're trying to mount is a CD/DVD image execute following
command:
```bash
file <path to iso>
# CD/DVD image will report 'ISO 9660 CD-ROM`
# Mass Storage compatible ISO will usually report 'DOS/MBR'
```

## Windows installation in OpenBMC VirtualMedia
Windows ISO images are meant to be mounted in following ways:
- DVD burned directly from ISO
- USB device prepared by Windows Media Creation Tool

None of these are applicable out of the box for OpenBMC VirtualMedia, so
following preparation is needed:

### Option 1: Create USB installation disk on Windows and dump drive image
```
Image size consideration

Please note that this method will create image of the whole partition. By default it means that resultant image will have size of your whole USB drive used for the operation (16GB image for 16GB USB drive).

To keep image size as small as possible it's best to use use smallest drive possible (preferably 8GB) for image creation.
Another (more advanced) option is to create/shrink partition on USB drive to preferred size, but keep in mind that it would require rolling back after operation to restore full USB drive capacity.

If you want to have full control over image size and don't want to modify your USB drive, please check 'Option 2', as it covers manual conversion of image without need of any USB device.
```

If Windows Media Creation Tool is not available for your version of Windows,
follow [instructions from Microsoft](https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/install-windows-from-a-usb-flash-drive).
In short:
1. Format your USB drive as FAT32 'Active' partition
2. Mount ISO in the system (Windows have built-in support for that)
3. Using linked instruction copy over ISO contents to your USB drive
   - copy all files less than 4GB directly
   - split `sources/install.wim` file into multiple `.swm` files

Dump newly populated FAT32 partition into image. There are various tools to achieve
that, you can use following solution based on `Cygwin`:
1. Install Cygwin from https://cygwin.com/install.html
2. Find your FAT32 partition by executing command:
```bash
cat /proc/partitions
```
   - Find device descriptor for your USB Drive, in this system it has letter ‘D:\’ assigned
```bash
sdb1   D:\
```
3. Create image of the partition with following command,
```bash
dd if=/dev/*** conv=sync,noerror bs=64K status=progress of=***
```
Assuming your drive is at `/dev/sdb1`, and image should be placed at `C:\Users\Admin\Documents\disk_image.img`:
```bash
dd if=/dev/sdb1 conv=sync,noerror bs=64K status=progress of=/cygdrive/c/Users/Admin/Documents/disk_image.img
```

### Option 2: Convert ISO to FAT32 disk image on Linux
```
Following instruction is based on Ubuntu 18.04, but should be applicable
to wide range of modern GNU/Linux operating systems.

It requires wimtools and 7zip, which can be installed as such:
sudo apt install p7zip-full wimtools
```
#### Unpack image to 'iso' directory
```bash
# 7zip package should be available in all modern Linux distributions
7z x <path_to_iso> -oiso
```

#### Create empty image from_iso.img
```bash
# 8 gb should be suitable for double density DVD, but can be decreased to fit the image
dd if=/dev/zero of=from_iso.img count=8 bs=1G
```

#### Create FAT32 partition
```bash
# Interactive tool will require following inputs:
# n <enter> <enter> <enter> <enter> t c w
fdisk from_iso.img
```

```bash
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x84a6bd20.

# type 'n'
Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
# type <enter>
Select (default p):

Using default response p.
# type <enter>
Partition number (1-4, default 1):
# type <enter>
First sector (2048-16777215, default 2048):
# type <enter>
Last sector, +sectors or +size{K,M,G,T,P} (2048-16777215, default 16777215):

Created a new partition 1 of type 'Linux' and of size 8 GiB.

# type 't'
Command (m for help): t
Selected partition 1
# type 'c'
Hex code (type L to list all codes): c
Changed type of partition 'Linux' to 'W95 FAT32 (LBA)'.

# type 'w'
Command (m for help): w
The partition table has been altered.
Syncing disks.
```

#### Format filesystems
```bash
mkfs.vfat from_iso.img
```

#### Modify ISO contents to fit into FAT32 partition
```bash
wimlib-imagex split "./iso/sources/install.wim" "$(pwd)/iso/sources/install.swm" 4000
```

```
WARNING:
Next commands should be performed as 'root' user.
Type 'sudo -s' to switch to root session, type 'exit' afterwards.
```

#### Copy ISO contents to newly created image
```bash
mkdir mounted
mount from_iso.img mounted
rsync -rv --exclude="sources/install.wim" ./iso/ ./mounted
```

#### Unmount devices
```bash
# Ensure that everything has been written to disk
sync
sync
umount mounted
```

Re: Installing Windows Server 2019 from a remotely mounted ISO

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 442 bytes --]

On Thu, Oct 22, 2020 at 01:37:17PM +0200, Maciej Lawniczak wrote:
> Mounting DVD-based ISOs in Intel-BMC Virtual Media is not supported by 
> underlying USB Mass Storage gadget. To mount such iso files they need to 
> be converted to MBR/FAT32 image. Attached instruction shows two ways of 
> preparing such image.

Hi Maciej,

Nice write up.  Would you be able to contribute this to the openbmc/docs
repo?

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Installing Windows Server 2019 from a remotely mounted ISO

[Paul Fertser]

Hi Konstantin,

On Tue, Aug 03, 2021 at 04:26:52PM +0300, Konstantin Klubnichkin wrote:
> It's about Option 2 (preparing image on Linux).
> The sequence in the instruction is:
> 1. Create empty image using dd;
> 2. Create partition table on it with FAT32 partition;
> 3. Format image to FAT32 filesystem;
> 4. Prepare files from Windows DVD;
> 5. Mount image to some directory;
> 6. Copy files to that directory;
> 7. Unmount image.
>  
> I can't get why to create partition table on the image(step 2) if the next step
> (formatting image with mkfs.vfat, step 3) destroys it?

Looks like a typo, s/image/partition/.

That said, booting windows installer shouldn't be anything special
with UEFI, basically you create GPT, EFI System partition on it, put
the windows EFI loader there with the right name plus some files it
needs, and then copy everything else to NTFS partition.

This should explain it with enough details:

https://techbit.ca/2019/02/creating-a-bootable-windows-10-uefi-usb-drive-using-linux/

HTH
-- 
Be free, use free (http://www.gnu.org/philosophy/free-sw.html) software!
mailto:fercerpav@gmail.com

Re: Installing Windows Server 2019 from a remotely mounted ISO

[i.kononenko]

Hello Konstantin,

I have several patches [1] to solves mounting DVD-based ISOs through Virtual Media.
The published changes is WIP until had approved by the Linux Foundation community,
but in the YADRO, we use these changes to support installing the OS Windows via
Virtual Media.


I hope your will found it helpful that information to support installing Windows-based OS.

Links:
[1] - https://patchwork.kernel.org/project/linux-usb/list/?series=507583

On 03.08.2021 16:26, Konstantin Klubnichkin wrote:
> Hello Maciej,
> I've got a question regarding the instruction.
> It's about Option 2 (preparing image on Linux).
> The sequence in the instruction is:
> 1. Create empty image using dd;
> 2. Create partition table on it with FAT32 partition;
> 3. Format image to FAT32 filesystem;
> 4. Prepare files from Windows DVD;
> 5. Mount image to some directory;
> 6. Copy files to that directory;
> 7. Unmount image.
> I can't get why to create partition table on the image(step 2) if the next step 
> (formatting image with mkfs.vfat, step 3) destroys it?
> Could you please clarify this point?
> Thank you!
> 22.10.2020, 14:39, "Maciej Lawniczak" <maciej.lawniczak@linux.intel.com>:
> 
>     Mounting DVD-based ISOs in Intel-BMC Virtual Media is not supported by
>     underlying USB Mass Storage gadget. To mount such iso files they need to be
>     converted to MBR/FAT32 image. Attached instruction shows two ways of
>     preparing such image.
> 
>     On 10/21/2020 9:32 AM, Thaj wrote:
> 
>         Hi Alexander, I am also facing the same issue. Did you find any
>         workaround for this ?
>         On Mon, May 25, 2020 at 4:04 PM Alexander Amelkin <a.amelkin@yadro.com
>         <mailto:a.amelkin@yadro.com>> wrote:
> 
>             Hi all!
> 
>             We're trying to install Windows Server 2019 from an ISO image mounted as
>             remote media via OpenBMC WebUI.
> 
>             The Windows installer boots, but then says that a media driver is
>             missing.
> 
>             Are there any hints on how to accomplish the task?
> 
>             Attached is the screenshot of the message we get with Windows Server
>             2019.
> 
>             With best regards,
>             Alexander.
> 
>             P.S. We're using Intel-BMC version of OpenBMC for the task, but it looks
>             like the remote media part is more or less the same.
> 
> -- 
> Best regards,
> Konstantin Klubnichkin,
> lead firmware engineer,
> server hardware R&D group,
> Yandex Moscow office.
> tel: +7-903-510-33-33
> 

-- 
Best regards,

Igor Kononenko

Re: Installing Windows Server 2019 from a remotely mounted ISO

[Paul Fertser]

On Tue, Aug 03, 2021 at 05:17:52PM +0300, Konstantin Klubnichkin wrote:
>   Looks like a typo, s/image/partition/.
> 
> I don't think so as formatting partition would require play with losetup and
> probably some boot loader installation onto the image.

kpartx creates appropriate loop devices automatically, so if your
image has a partition table you can easily get the suitable loop
devices to run mkfs.vfat on them, then mount it etc.

For UEFI bootloader installation on removable media is just copying a
file to a FAT32 partition, nothing fancy.

-- 
Be free, use free (http://www.gnu.org/philosophy/free-sw.html) software!
mailto:fercerpav@gmail.com

Re: Installing Windows Server 2019 from a remotely mounted ISO

[i.kononenko]

On 03.08.2021 18:21, Konstantin Klubnichkin wrote:
> Works like a charm with all ISO I had on hands, thank you!

Konstantin,

Please note, I got feedback from some issues about using CDFS images.
I work to solve the issue of using such ISO formats.

-- 
Best regards,

Igor Kononenko

RE: Installing Windows Server 2019 from a remotely mounted ISO

[Troy Lee]

Hi Igor,

Currently, the remote virtual media emulate the gadget driver as usb thumb drive instead of cd-rom.
Please have a look on my patch:
https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/42986

Thanks,
Troy Lee

-----Original Message-----
From: openbmc <openbmc-bounces+troy_lee=aspeedtech.com@lists.ozlabs.org> On Behalf Of i.kononenko
Sent: Tuesday, August 3, 2021 11:40 PM
To: Konstantin Klubnichkin <kitsok@yandex-team.ru>
Cc: openbmc@lists.ozlabs.org
Subject: Re: Installing Windows Server 2019 from a remotely mounted ISO

On 03.08.2021 18:21, Konstantin Klubnichkin wrote:
> Works like a charm with all ISO I had on hands, thank you!

Konstantin,

Please note, I got feedback from some issues about using CDFS images.
I work to solve the issue of using such ISO formats.

-- 
Best regards,

Igor Kononenko

Re: Installing Windows Server 2019 from a remotely mounted ISO

[i.kononenko]

Hello Troy,

On 04.08.2021 04:50, Troy Lee wrote:
> Hi Igor,
> 
> Currently, the remote virtual media emulate the gadget driver as usb thumb drive instead of cd-rom.
> Please have a look on my patch:
> https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/42986
Please note, the phosphor-misc:usb-ctrl have a way to specify mode usb|usb-ro|hdd|dvd which is useful
to pick between both usb or cd-rom devices. Please, refer to the changes [1].

The implementation of usb-gadget:mass-storage has a many limitations about cd-rom: 
 * Image size should not be significant then 2.1Gb,
 * Supports only CD-ROM profile
That does following to failure of installing Windows-like OS, because without a number of DVD-ROM 
profile's features and correct SCSI-command handling(TOC/ATIP/PMA) a Windows OS can't load appropriate 
UDF-driver(iso-13346) and will stucks.

Summary the mentioned in the current subject patchset should solve the described issue.

> 
> Thanks,
> Troy Lee

Links:
* https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-misc/+/36499


-- 
Best regards,

Igor Kononenko

RE: Installing Windows Server 2019 from a remotely mounted ISO

[Troy Lee]

Hi Igor,

> -----Original Message-----
> From: i.kononenko <i.kononenko@yadro.com>
> Sent: Wednesday, August 4, 2021 10:11 AM
> To: Troy Lee <troy_lee@aspeedtech.com>; Konstantin Klubnichkin
> <kitsok@yandex-team.ru>
> Cc: openbmc@lists.ozlabs.org
> Subject: Re: Installing Windows Server 2019 from a remotely mounted ISO
> 
> Hello Troy,
> 
> On 04.08.2021 04:50, Troy Lee wrote:
> > Hi Igor,
> >
> > Currently, the remote virtual media emulate the gadget driver as usb
> thumb drive instead of cd-rom.
> > Please have a look on my patch:
> > https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/42986
> Please note, the phosphor-misc:usb-ctrl have a way to specify mode
> usb|usb-ro|hdd|dvd which is useful to pick between both usb or cd-rom
> devices. Please, refer to the changes [1].
> 
This looks very useful, good information to know. 
Do frontend (web-vue/phosphor-webui) and backend (bmcweb) support it already?

> The implementation of usb-gadget:mass-storage has a many limitations
> about cd-rom:
>  * Image size should not be significant then 2.1Gb,
>  * Supports only CD-ROM profile
> That does following to failure of installing Windows-like OS, because without
> a number of DVD-ROM profile's features and correct SCSI-command
> handling(TOC/ATIP/PMA) a Windows OS can't load appropriate
> UDF-driver(iso-13346) and will stucks.
> 
> Summary the mentioned in the current subject patchset should solve the
> described issue.
> 
I'll give it a try, this might also solve our virtual media stress issues.

> >
> > Thanks,
> > Troy Lee
> 
> Links:
> * https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-misc/+/36499
> 
> 
> --
> Best regards,
> 
> Igor Kononenko

Thanks,
Troy Lee

Re: Installing Windows Server 2019 from a remotely mounted ISO

[i.kononenko]

Troy,

On 04.08.2021 06:17, Troy Lee wrote:
> Hi Igor,
> 
>> -----Original Message-----
>> From: i.kononenko <i.kononenko@yadro.com>
>> Sent: Wednesday, August 4, 2021 10:11 AM
>> To: Troy Lee <troy_lee@aspeedtech.com>; Konstantin Klubnichkin
>> <kitsok@yandex-team.ru>
>> Cc: openbmc@lists.ozlabs.org
>> Subject: Re: Installing Windows Server 2019 from a remotely mounted ISO
>>
>> Hello Troy,
>>
>> On 04.08.2021 04:50, Troy Lee wrote:
>>> Hi Igor,
>>>
>>> Currently, the remote virtual media emulate the gadget driver as usb
>> thumb drive instead of cd-rom.
>>> Please have a look on my patch:
>>> https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/42986
>> Please note, the phosphor-misc:usb-ctrl have a way to specify mode
>> usb|usb-ro|hdd|dvd which is useful to pick between both usb or cd-rom
>> devices. Please, refer to the changes [1].
>>
> This looks very useful, good information to know. 
> Do frontend (web-vue/phosphor-webui) and backend (bmcweb) support it already?

AFAIK, the bmcweb and WebUI don't support to specify Virtual Media type, 
but I have a number of patches that bring such features but there are not 
published yet.

In the past, I have published a bmcweb patch for supports VM mode for the nbd-proxy,
but the change was not framed properly and has been abandoned [1].

> 
>> The implementation of usb-gadget:mass-storage has a many limitations
>> about cd-rom:
>>  * Image size should not be significant then 2.1Gb,
>>  * Supports only CD-ROM profile
>> That does following to failure of installing Windows-like OS, because without
>> a number of DVD-ROM profile's features and correct SCSI-command
>> handling(TOC/ATIP/PMA) a Windows OS can't load appropriate
>> UDF-driver(iso-13346) and will stucks.
>>
>> Summary the mentioned in the current subject patchset should solve the
>> described issue.
>>
> I'll give it a try, this might also solve our virtual media stress issues.
> 
>>>
>>> Thanks,
>>> Troy Lee
>>
>> Links:
>> * https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-misc/+/36499
>>
>>
>> --
>> Best regards,
>>
>> Igor Kononenko
> 
> Thanks,
> Troy Lee
> 

Links:
1 - https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/36475
-- 
Best regards,

Igor Kononenko

Re: Installing Windows Server 2019 from a remotely mounted ISO

[i.kononenko]

On 18.10.2022 13:44, Konstantin Klubnichkin wrote:
> «Внимание! Данное письмо от внешнего адресата!»
> 
> Hello, Igor!
> Did you succeed to include this pachset into upstream?
> Thank you!

Hello Konstantin!

Unfortunately, I've suspended work with publishing patches into upstream
of the usb-gadget fmc.  The community point is the big rework of  the   
usb-gadget:fmc subsystem is not required, and moreover we should avoid  
that, because there is  a user-space gadget driver that may provide     
a wider number of features and could be more agile.

My opinion at this point is that the OpenBMC VirtualMedia implementation
could be based on the GadgetFS driver with ConfigFS feature. I was plan to      
prepare a design-doc proposal with PoC, but today I have too wide list 
of critical tasks.

> 03.08.2021, 15:53, "i.kononenko" <i.kononenko@yadro.com>:
> 
>     Hello Konstantin,
> 
>     I have several patches [1] to solves mounting DVD-based ISOs through Virtual
>     Media.
>     The published changes is WIP until had approved by the Linux Foundation
>     community,
>     but in the YADRO, we use these changes to support installing the OS Windows via
>     Virtual Media.
> 
> 
>     I hope your will found it helpful that information to support installing
>     Windows-based OS.
> 
>     Links:
>     [1] - https://patchwork.kernel.org/project/linux-usb/list/?series=507583
>     <https://patchwork.kernel.org/project/linux-usb/list/?series=507583>
> 
>     --
>     Best regards,
> 
>     Igor Kononenko
> 
> -- 
> Best regards,
> Konstantin Klubnichkin,
> lead firmware engineer,
> server hardware R&D group,
> Yandex Moscow office.
> tel: +7-903-510-33-33

-- 
Best regards,

Igor Kononenko

Re: Installing Windows Server 2019 from a remotely mounted ISO

[Paul Fertser]

Hi Konstantin,

On Tue, Oct 18, 2022 at 12:44:49PM +0200, Konstantin Klubnichkin wrote:
> Did you succeed to include this pachset into upstream?

FWIW, we're using those patches on a rather old fork, and they work
great.

But there seems to be a serious throughput limitation for an AST2500
getting data over NCSI. With typical OpenBMC load on CPU we are
limited to about 1.0 MB/s (testing with netcat running on BMC
receiving a file of fixed length with TCP and directing it to
/dev/null), so installation from virtual media is quite slow.

Can you please benchmark your system for comparison? Do you use NCSI
for the management channel?

For the reference, USB mass storage gadget is able to stream data to
the host OS at about 11.5 MB/s.

-- 
Be free, use free (http://www.gnu.org/philosophy/free-sw.html) software!
mailto:fercerpav@gmail.com