All of lore.kernel.org
 help / color / mirror / Atom feed
* Bug: git p4 clone fails if non unicode files with +k or+ko attributes are present
@ 2021-05-28  4:18 Arthur Moore
  0 siblings, 0 replies; only message in thread
From: Arthur Moore @ 2021-05-28  4:18 UTC (permalink / raw)
  To: git

Hello,

I would like to report a problem which is preventing me from using
git's Perforce integration.

Problem Command:
git p4 clone //path/to/files/

Steps to reproduce:
* Create/have a configured and running Perforce server and client.
* Create a file with any encoding aside from UTF-8 or UTF-16
containing at least one character outside the ASCII range.
* Adjust the file's type to text and include the "+k" or "+ko" attributes.
  * This can be done in the graphical p4v client by right clicking the
file and selecting "Change Filetype"
* Submit that file to Perforce.
* Attempt to run `git p4 clone //path/to/files/`

Expected behavior:
The Perforce repository is successfully cloned.

Actual behavior:

The clone failed with the following error:
```
Traceback (most recent call last):
  File "/usr/libexec/git-core/git-p4", line 4441, in <module>
    main()
  File "/usr/libexec/git-core/git-p4", line 4435, in main
    if not cmd.run(args):
  File "/usr/libexec/git-core/git-p4", line 4187, in run
    if not P4Sync.run(self, depotPaths):
  File "/usr/libexec/git-core/git-p4", line 4053, in run
    self.importHeadRevision(revision)
  File "/usr/libexec/git-core/git-p4", line 3761, in importHeadRevision
    self.commit(details, self.extractFilesFromCommit(details), self.branch)
  File "/usr/libexec/git-core/git-p4", line 3311, in commit
    self.streamP4Files(files)
  File "/usr/libexec/git-core/git-p4", line 3155, in streamP4Files
    p4CmdList(["-x", "-", "print"],
  File "/usr/libexec/git-core/git-p4", line 784, in p4CmdList
    cb(entry)
  File "/usr/libexec/git-core/git-p4", line 3142, in streamP4FilesCbSelf
    self.streamP4FilesCb(entry)
  File "/usr/libexec/git-core/git-p4", line 3090, in streamP4FilesCb
    self.streamOneP4File(self.stream_file, self.stream_contents)
  File "/usr/libexec/git-core/git-p4", line 3035, in streamOneP4File
    text = ''.join(decode_text_stream(c) for c in contents)
  File "/usr/libexec/git-core/git-p4", line 3035, in <genexpr>
    text = ''.join(decode_text_stream(c) for c in contents)
  File "/usr/libexec/git-core/git-p4", line 184, in decode_text_stream
    return s.decode() if isinstance(s, bytes) else s
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xde in position
6276: invalid continuation byte
```

Failure Analysis:

The problem appears to be partly in `streamOneP4File` and partly in `
p4_keywords_regexp_for_type`.

Perforce allows embedding special keywords in files that are expanded
at checkout, and this function strips the expansion.
To do so,  it converts the binary data to a real string, performs a
regex replace, then converts it back to binary data.
Unfortunately, that assumes the file is encoded in UTF-8, and when
this is not the case an uncaught exception is thrown.

Importantly, perforce also allows injecting those keywords in raw
binary files, so this goes beyond a simple "support more encodings"
issue.

https://community.perforce.com/s/article/3482

[System Info]
git version 2.31.1

[Enabled Hooks]
not run from a git repository - no hooks to show

Sincerely,
Arthur Moore

^ permalink raw reply	[flat|nested] only message in thread

only message in thread, other threads:[~2021-05-28  4:18 UTC | newest]

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-28  4:18 Bug: git p4 clone fails if non unicode files with +k or+ko attributes are present Arthur Moore

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.