From: Arthur Moore <Arthur.Moore.git@cd-net.net>
To: git@vger.kernel.org
Subject: Bug: git p4 clone fails if non unicode files with +k or+ko attributes are present
Date: Fri, 28 May 2021 04:18:18 +0000 [thread overview]
Message-ID: <01000179b131beca-113a6722-77cb-4320-8930-6904b197a003-000000@email.amazonses.com> (raw)
Hello,
I would like to report a problem which is preventing me from using
git's Perforce integration.
Problem Command:
git p4 clone //path/to/files/
Steps to reproduce:
* Create/have a configured and running Perforce server and client.
* Create a file with any encoding aside from UTF-8 or UTF-16
containing at least one character outside the ASCII range.
* Adjust the file's type to text and include the "+k" or "+ko" attributes.
* This can be done in the graphical p4v client by right clicking the
file and selecting "Change Filetype"
* Submit that file to Perforce.
* Attempt to run `git p4 clone //path/to/files/`
Expected behavior:
The Perforce repository is successfully cloned.
Actual behavior:
The clone failed with the following error:
```
Traceback (most recent call last):
File "/usr/libexec/git-core/git-p4", line 4441, in <module>
main()
File "/usr/libexec/git-core/git-p4", line 4435, in main
if not cmd.run(args):
File "/usr/libexec/git-core/git-p4", line 4187, in run
if not P4Sync.run(self, depotPaths):
File "/usr/libexec/git-core/git-p4", line 4053, in run
self.importHeadRevision(revision)
File "/usr/libexec/git-core/git-p4", line 3761, in importHeadRevision
self.commit(details, self.extractFilesFromCommit(details), self.branch)
File "/usr/libexec/git-core/git-p4", line 3311, in commit
self.streamP4Files(files)
File "/usr/libexec/git-core/git-p4", line 3155, in streamP4Files
p4CmdList(["-x", "-", "print"],
File "/usr/libexec/git-core/git-p4", line 784, in p4CmdList
cb(entry)
File "/usr/libexec/git-core/git-p4", line 3142, in streamP4FilesCbSelf
self.streamP4FilesCb(entry)
File "/usr/libexec/git-core/git-p4", line 3090, in streamP4FilesCb
self.streamOneP4File(self.stream_file, self.stream_contents)
File "/usr/libexec/git-core/git-p4", line 3035, in streamOneP4File
text = ''.join(decode_text_stream(c) for c in contents)
File "/usr/libexec/git-core/git-p4", line 3035, in <genexpr>
text = ''.join(decode_text_stream(c) for c in contents)
File "/usr/libexec/git-core/git-p4", line 184, in decode_text_stream
return s.decode() if isinstance(s, bytes) else s
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xde in position
6276: invalid continuation byte
```
Failure Analysis:
The problem appears to be partly in `streamOneP4File` and partly in `
p4_keywords_regexp_for_type`.
Perforce allows embedding special keywords in files that are expanded
at checkout, and this function strips the expansion.
To do so, it converts the binary data to a real string, performs a
regex replace, then converts it back to binary data.
Unfortunately, that assumes the file is encoded in UTF-8, and when
this is not the case an uncaught exception is thrown.
Importantly, perforce also allows injecting those keywords in raw
binary files, so this goes beyond a simple "support more encodings"
issue.
https://community.perforce.com/s/article/3482
[System Info]
git version 2.31.1
[Enabled Hooks]
not run from a git repository - no hooks to show
Sincerely,
Arthur Moore
reply other threads:[~2021-05-28 4:18 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=01000179b131beca-113a6722-77cb-4320-8930-6904b197a003-000000@email.amazonses.com \
--to=arthur.moore.git@cd-net.net \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.