Git Mailing List Archive on lore.kernel.org
 help / color / Atom feed
From: Michael Felt <aixtools@felt.demon.nl>
To: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Cc: "brian m. carlson" <sandals@crustytoothpaste.net>,
	git@vger.kernel.org, Dan Shumow <shumow@gmail.com>,
	Junio C Hamano <gitster@pobox.com>, Jeff King <peff@peff.net>
Subject: Re: Is detecting endianness at compile-time unworkable?
Date: Tue, 31 Jul 2018 12:39:36 +0200
Message-ID: <93056823-2740-d072-1ebd-46b440b33d7e@felt.demon.nl> (raw)
In-Reply-To: <87wotdt649.fsf@evledraar.gmail.com>

A small step back...


On 7/30/2018 11:39 AM, Ævar Arnfjörð Bjarmason wrote:
> On Sun, Jul 29 2018, Michael wrote:
>
>> On 29/07/2018 22:06, brian m. carlson wrote:
>>> On Sun, Jul 29, 2018 at 09:48:43PM +0200, Michael wrote:
>>>> On 29/07/2018 21:27, brian m. carlson wrote:
>>>>> Well, that explains it.  I would recommend submitting a patch to
>>>>> https://github.com/cr-marcstevens/sha1collisiondetection, and the we can
>>>>> pull in the updated submodule with that fix.
>>>> Not sure I am smart enough to do that. I'll have to download, build, and see
>>>> what it says.
>>> The issue is that somewhere in lib/sha1.c, you need to cause
>>> SHA1DC_BIGENDIAN to be set.  That means you need to figure out what
>>> compiler macro might indicate that.
>> I remember - roughly - a few decades back - having an assignment to
>> write code to determine endianness. PDP and VAC were different iirc,
>> and many other micro-processors besides the 8088/8086/z85/68k/etc..
>>
>> If you are looking for a compiler macro as a way to determine this -
>> maybe you have one for gcc, but not for xlc. I do not know it - currently :)
> I'm not familiar with AIX, but from searching around I found this
> porting manual from IBM:
> http://www.redbooks.ibm.com/redbooks/pdfs/sg246034.pdf
This is from July 2001 - when AIX 5L, for Linux affinity, was new. AIX 
was (nearly) the #1 posix system, and linux was a minor player in the 
data center (in or out (now as IAAS)). IMHO, the recommendations made in 
2001 are probably no longer applicable (64-bit was fairly new, e.g., 
rather than common).
>
> There they suggest either defining your own macros, or testing the
> memory layout at runtime (see section "2.2.2.3 Technique 3: Testing
> memory layout" and surrounding sections).
>
> Perhaps it's worth taking a step back here and thinking about whether
> this whole thing is unworkable. It was hard enough to get this to work
> on the combination of Linux, *BSD and Solaris, but I suspect we'll run
> into increasingly obscure platforms where this is hard or impossible
> (AIX, HP/UX etc.)
>
> The reason we're in this hole is because we use this
> sha1collisiondetection library to do SHA-1, and the reason we have
> issues with it specifically (not OpenSSL et al) is because its only
> method of detecting endianness is at compile time.
Cannot speak for the "others", but as I have mentioned before - as AIX 
in only on POWER it is also only Big Endian - so a compiletime #if 
testing for _AIX will work fine
> This didn't use to be the case, it was changed in this commit:
> https://github.com/cr-marcstevens/sha1collisiondetection/commit/d597672
>
> Dan Shumow: Since the commit message doesn't say why, can you elaborate
> a bit on why this was done, i.e. is determining this at runtime harmful
> for performance? If not, perhaps it would be best to bring this back, at
> least as an option.
>
> And, as an aside, the reason we can't easily make it better ourselves is
> because the build process for git.git doesn't have a facility to run
> code to detect this type of stuff (the configure script is always
> optional). So we can't just run this test ourselves.
On AIX - I am required to run configure, and frankly, I am amazed that 
not everyone is running it. Among other things I an modifying the prefix 
(to /opt) and many of the others to different /var/git/* areas as I do 
not want to "polute" the BOS (base OS) and/or other packages/packagers. 
Officially, according the Linux FHS-3.0 I sould be using /opt/aixtools 
as prefix.

FYI: my current build process is:
wget git-{version}.tar.xz
xz -dc git-{version}.tar.xz
cd git-{version}.tar.xz
mv Makefile Makefile.git #my build scripts only run configure when 
Makefile does not exist
./configure ...
ln Makefile.git Makefile
make

I am amazed, as it rarely happens (maybe git is my first encounter) - 
that configure does not create a Makefile. This also complicates 
building git "out of tree".
>
> Junio: I've barked up that particular tree before in
> https://public-inbox.org/git/87a7x3kmh5.fsf@evledraar.gmail.com/ and I
> won't bore you all by repeating myself, except to say that this is yet
> another case where I wish we had a hard dependency on some way of doing
> checks via compiled code in our build system.
For AIX: again - the determination is simple. If _AIX is set to 1 then 
use BigEndian, or, use:
michael@x071:[/home/michael]uname
AIX
  i.e., something like:
$(uname) == "AIX" && BigEndian=1

  parent reply index

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-07-29 16:44 git broken for AIX somewhere between 2.13.2 and 2.13.3 Michael
2018-07-29 18:10 ` brian m. carlson
2018-07-29 19:46   ` Michael
2018-07-29 20:05     ` Ævar Arnfjörð Bjarmason
2018-07-29 21:40       ` Andreas Schwab
2018-07-30  6:22         ` Michael
     [not found]   ` <2309fa7f-c2d8-ee57-aff5-b9e32d2da609@felt.demon.nl>
     [not found]     ` <20180729192753.GD945730@genre.crustytoothpaste.net>
2018-07-29 19:48       ` Michael
2018-07-29 20:06         ` brian m. carlson
2018-07-29 20:50           ` Michael
2018-07-30  9:39             ` Is detecting endianness at compile-time unworkable? Ævar Arnfjörð Bjarmason
2018-07-30 14:54               ` Junio C Hamano
2018-07-30 18:32                 ` Junio C Hamano
2018-07-30 18:39                   ` Daniel Shumow
2018-07-31 10:06                     ` Michael Felt
2018-08-01  1:35                   ` Eric Wong
2018-08-01  7:16                 ` Ævar Arnfjörð Bjarmason
2018-07-31 10:39               ` Michael Felt [this message]
2018-08-01  7:31                 ` Ævar Arnfjörð Bjarmason
2018-08-02 20:50                   ` [PATCH] sha1dc: update from upstream Ævar Arnfjörð Bjarmason
2018-08-02 21:29                     ` Michael Felt (aixtools)
2018-08-02 21:32                     ` Stefan Beller
2018-07-31 12:32               ` Is detecting endianness at compile-time unworkable? Michael Felt
2018-07-31 14:01               ` Michael Felt
2018-07-31 14:25                 ` Ævar Arnfjörð Bjarmason
2018-07-31 20:06                   ` Michael

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=93056823-2740-d072-1ebd-46b440b33d7e@felt.demon.nl \
    --to=aixtools@felt.demon.nl \
    --cc=avarab@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=peff@peff.net \
    --cc=sandals@crustytoothpaste.net \
    --cc=shumow@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Git Mailing List Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/git/0 git/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 git git/ https://lore.kernel.org/git \
		git@vger.kernel.org
	public-inbox-index git

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.git


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git