From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gigi W Subject: Re: Integrity checking fails with Atmel SHA hw accelerator enabled Date: Mon, 26 Feb 2018 08:31:31 +0200 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1805525051288673233==" Cc: device-mapper development , Linux Crypto Mailing List To: Gilad Ben-Yossef Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com List-Id: linux-crypto.vger.kernel.org --===============1805525051288673233== Content-Type: multipart/alternative; boundary="001a113f6ad82be368056617a934" --001a113f6ad82be368056617a934 Content-Type: text/plain; charset="UTF-8" Thanks again for feedback ! Yes, I can do some additional tests in order to isolate the problem. I'm not that familiar with the code responsible for this so I'll ask for your help with a patch. It will be quicker that way. Best regards Mircea On Sat, Feb 24, 2018 at 10:59 AM, Gilad Ben-Yossef wrote: > Hi, > > I'm adding the linux crypto mailing list because it seems relevant. > > > On Fri, Feb 23, 2018 at 2:25 PM, Gigi W wrote: > > Thanks for the input! > > > > See below > > > > > > On Fri, Feb 23, 2018 at 10:53 AM Gilad Ben-Yossef > > wrote: > >> > >> On Fri, Feb 23, 2018 at 10:30 AM, Gigi W wrote: > >> > Hi > >> > > >> > I'm having some trouble using dm-verity for a squashfs root file > system > >> > that > >> > seems to be related to the > >> > Atmel SHA hw accelerator in the kernel, CONFIG_CRYPTO_DEV_ATMEL_SHA > >> > > >> > Some info about my setup: > >> > * I'm using a board with a SAMA5D4 CPU. > >> > * I'm using Yocto rocko for building an image for that device. > >> > > >> > The idea is that Using the 4.14.14 Kernel, Integrity checking using > >> > Kernel > >> > crypto fails with Atmel SHA hw accelerator enabled in kernel. > >> > By disabling it, `CONFIG_CRYPTO_DEV_ATMEL_SHA=n`, and using the > software > >> > sha256 algo, integrity checking works as expected. > >> > This is my kernel config [3] > >> > > >> > Using the 4.8.4 Kernel and Atmel SHA hw accelerator enabled, > everything > >> > was > >> > ok. > >> > > >> > This is what triggers the error during verified boot: > >> > > >> > status=`veritysetup create vroot $root_dev $verity_dev --hash-offset > >> > $hashoffset $root_hash` > >> > > >> > mount /dev/mapper/vroot /mnt/ > >> > mount_ok=`cat /proc/mounts | grep mnt` > >> > if [ -z "$mount_ok" ] ; then > >> > echo "Failed to mount $root_dev on mnt/" > >> > else > >> > echo "Switch rootfs" > >> > exec switch_root -c /dev/console /mnt /sbin/init > >> > fi > >> > > >> > The mount operation fails: > >> > > >> > device-mapper: verity: 179:4: metadata block 2 is corrupted > >> > EXT4-fs (dm-0): unable to read superblock > >> > device-mapper: verity: 179:4: metadata block 2 is corrupted > >> > EXT4-fs (dm-0): unable to read superblock > >> > device-mapper: verity: 179:4: metadata block 2 is corrupted > >> > EXT4-fs (dm-0): unable to read superblock > >> > device-mapper: verity: 179:4: metadata block 2 is corrupted > >> > SQUASHFS error: squashfs_read_data failed to read block 0x0 > >> > squashfs: SQUASHFS error: unable to read squashfs_super_block > >> > device-mapper: verity: 179:4: metadata block 2 is corrupted > >> > FAT-fs (dm-0): unable to read boot sector > >> > mount: mounting /dev/mapper/vroot on /mnt/ failed: Input/output error > >> > Failed to mount /dev/mmcblk0p4 on mnt/ > >> > reboot: Restarting system > >> > Reboot failed -- System halted > >> > > >> > Using veritysetup to verify the integrity against the hashes is > >> > successful, > >> > as it's not using the kernel for that ... > >> > > >> > > >> > So it looks like it something changed from 4.8.4 to 4.14.14. > >> > >> If I am not mistaken the Atmel SHA hw accelerator is an async (read: > >> off CPU) crypto accelerator. > >> Up until 4.12 (I think...) DM-Verity did not use async crypto > >> accelerators (even if present and have high > >> priority). I've changed this is commit d1ac3ff008fb ("dm verity: > >> switch to using asynchronous hash crypto API"). > > > > > > This would explain some things, like the same speeds while reading from a > > verity device, having the CONFIG_CRYPTO_DEV_ATMEL_SHA enabled and then > > disabled, on the 4.8.4 kernel -> it was always using the sync API. > > > >> > >> > >> Is it possible that whatever issue you are seeing has always been > >> there and when DM-Verity started using > >> async. accelerators it was only exposed? > > > > > > It looks like it. > > > > From my understandings + tests described above, Atmel SHA hw accelerator > > works correctly - output hashes are ok, dm-verity with other async crypto > > accelerators is working ok, > > but dm-verity + Atmel SHA hw accelerator don't play nice together. > > > > I couldn't find anyone else complaining about this. > > > > I agree that that the possibility that there is something wrong in the > Atmel SHA accelerator is one possible direction. > The other one is that there is something wrong in DM-Verity that only > manifests under certain conditions when working with async crypto HW > providers. > I don't think it is the case, because I tested DM-Verity after my > changes with the CryptoCell async HW provider and did not get any > other bug reports, but I'd like to be sure. > > Can you do a little experiment? add debug printk to show the data > being hashed, the hash produced by atmel and the expected > pre-calculated hash. > > If the theory that there is something wrong with atmel accelerator, we > can calculate the hash on the data with other means (software) and > should get a different hash. > > If you are having trouble adding the printk's in the right place let > me know and I'll create a patch for you to test. > > Cheers, > Gilad > > > -- > Gilad Ben-Yossef > Chief Coffee Drinker > > "If you take a class in large-scale robotics, can you end up in a > situation where the homework eats your dog?" > -- Jean-Baptiste Queru > --001a113f6ad82be368056617a934 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks again for feedback !
Y= es, I can do some additional tests in order to isolate the problem.
I'm not that familiar with the code responsible for this so I'll = ask for your help with a patch. It will be quicker that way.

B= est regards
Mircea


On Sat, Feb 24, 2018 at 10:59 AM, Gilad Ben-Yossef = <gilad@benyossef.com> wrote:
Hi,

I'm adding the linux crypto mailing list because it seems relevant.


On Fri, Feb 23, 2018 at 2:25 PM, Gigi W <gigitekwork@gmail.com> wrote:
> Thanks for the input!
>
> See below
>
>
> On Fri, Feb 23, 2018 at 10:53 AM Gilad Ben-Yossef <gilad@benyossef.com>
> wrote:
>>
>> On Fri, Feb 23, 2018 at 10:30 AM, Gigi W <gigitekwork@gmail.com> wrote:
>> > Hi
>> >
>> > I'm having some trouble using dm-verity for a squashfs ro= ot file system
>> > that
>> > seems to be related to the
>> > Atmel SHA hw accelerator in the kernel, CONFIG_CRYPTO_DEV_ATM= EL_SHA
>> >
>> > Some info about my setup:
>> > * I'm using a board with a SAMA5D4 CPU.
>> > * I'm using Yocto rocko for building an image for that de= vice.
>> >
>> > The idea is that Using the 4.14.14 Kernel, Integrity checking= using
>> > Kernel
>> > crypto fails with Atmel SHA hw accelerator enabled in kernel.=
>> > By disabling it, `CONFIG_CRYPTO_DEV_ATMEL_SHA=3Dn`, and = using the software
>> > sha256 algo, integrity checking works as expected.
>> > This is my kernel config [3]
>> >
>> > Using the 4.8.4 Kernel and Atmel SHA hw accelerator enabled, = everything
>> > was
>> > ok.
>> >
>> > This is what triggers the error during verified boot:
>> >
>> > status=3D`veritysetup create vroot $root_dev $verity_dev --ha= sh-offset
>> > $hashoffset $root_hash`
>> >
>> >=C2=A0 =C2=A0 =C2=A0mount /dev/mapper/vroot /mnt/
>> >=C2=A0 =C2=A0 =C2=A0mount_ok=3D`cat /proc/mounts | grep mnt` >> >=C2=A0 =C2=A0 =C2=A0if [ -z "$mount_ok" ] ; then
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0echo "Failed to mount $= root_dev on mnt/"
>> >=C2=A0 =C2=A0 =C2=A0else
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0echo "Switch rootfs&quo= t;
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0exec switch_root -c /dev/con= sole /mnt /sbin/init
>> >=C2=A0 =C2=A0 =C2=A0fi
>> >
>> > The mount operation fails:
>> >
>> > device-mapper: verity: 179:4: metadata block 2 is corrupted >> > EXT4-fs (dm-0): unable to read superblock
>> > device-mapper: verity: 179:4: metadata block 2 is corrupted >> > EXT4-fs (dm-0): unable to read superblock
>> > device-mapper: verity: 179:4: metadata block 2 is corrupted >> > EXT4-fs (dm-0): unable to read superblock
>> > device-mapper: verity: 179:4: metadata block 2 is corrupted >> > SQUASHFS error: squashfs_read_data failed to read block 0x0 >> > squashfs: SQUASHFS error: unable to read squashfs_super_block=
>> > device-mapper: verity: 179:4: metadata block 2 is corrupted >> > FAT-fs (dm-0): unable to read boot sector
>> > mount: mounting /dev/mapper/vroot on /mnt/ failed: Input/outp= ut error
>> > Failed to mount /dev/mmcblk0p4 on mnt/
>> > reboot: Restarting system
>> > Reboot failed -- System halted
>> >
>> > Using veritysetup to verify the integrity against the hashes = is
>> > successful,
>> > as it's not using the kernel for that ...
>> >
>> >
>> > So it looks like it something changed from 4.8.4 to 4.14.14.<= br> >>
>> If I am not mistaken the Atmel SHA hw accelerator is an async (rea= d:
>> off CPU) crypto accelerator.
>>=C2=A0 Up until 4.12 (I think...) DM-Verity did not use async crypt= o
>> accelerators (even if present and have high
>> priority). I've changed this is commit d1ac3ff008fb ("dm = verity:
>> switch to using asynchronous hash crypto API").
>
>
> This would explain some things, like the same speeds while reading fro= m a
> verity device, having the CONFIG_CRYPTO_DEV_ATMEL_SHA enabled and then=
> disabled, on the 4.8.4 kernel -> it was always using the sync API.<= br> >
>>
>>
>> Is it possible that whatever issue you are seeing has always been<= br> >> there and when DM-Verity started using
>> async. accelerators it was only exposed?
>
>
> It looks like it.
>
> From my understandings + tests described above, Atmel SHA hw accelerat= or
> works correctly - output hashes are ok, dm-verity with other async cry= pto
> accelerators is working ok,
> but dm-verity + Atmel SHA hw accelerator don't play nice together.=
>
> I couldn't find anyone else complaining about this.
>

I agree that that the possibility that there is something wrong= in the
Atmel SHA accelerator is one possible direction.
The other one is that there is something wrong in DM-Verity that only
manifests under certain conditions when working with async crypto HW
providers.
I don't think it is the case, because I tested DM-Verity after my
changes with the CryptoCell async HW provider and did not get any
other bug reports, but I'd like to be sure.

Can you do a little experiment? add debug printk to show the data
being hashed, the hash produced by atmel and the expected
pre-calculated hash.

If the theory that there is something wrong with atmel accelerator, we
can calculate the hash on the data with other means (software) and
should get a different hash.

If you are having trouble adding the printk's in the right place let me know and I'll create a patch for you to test.

Cheers,
Gilad


--
Gilad Ben-Yossef
Chief Coffee Drinker

"If you take a class in large-scale robotics, can you end up in a
situation where the homework eats your dog?"
=C2=A0-- Jean-Baptiste Queru

--001a113f6ad82be368056617a934-- --===============1805525051288673233== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============1805525051288673233==--