Subject: Re: [PATCH v4 1/2] git-p4: add test case for "Translation of file content failed" error

From: Junio C Hamano <[email-masked]>
Date: Mon, 21 Sep 2015 11:09:31 -0700

[email-masked] writes:

> From: Lars Schneider <[email-masked]>
>
> A P4 repository can get into a state where it contains a file with
> type UTF-16 that does not contain a valid UTF-16 BOM. If git-p4
> attempts to retrieve the file then the process crashes with a
> "Translation of file content failed" error.
>
> More info here: http://answers.perforce.com/articles/KB/3117
>
> Signed-off-by: Lars Schneider <[email-masked]>
> ---
>  t/t9825-git-p4-handle-utf16-without-bom.sh | 50 ++++++++++++++++++++++++++++++
>  1 file changed, 50 insertions(+)
>  create mode 100755 t/t9825-git-p4-handle-utf16-without-bom.sh
>
> diff --git a/t/t9825-git-p4-handle-utf16-without-bom.sh
> b/t/t9825-git-p4-handle-utf16-without-bom.sh
> new file mode 100755
> index 0000000..65c3c4e
> --- /dev/null
> +++ b/t/t9825-git-p4-handle-utf16-without-bom.sh
> @@ -0,0 +1,50 @@
> +#!/bin/sh
> +
> +test_description='git p4 handling of UTF-16 files without BOM'
> +
> +. ./lib-git-p4.sh
> +
> +UTF16="\227\000\227\000"
> +
> +test_expect_success 'start p4d' '
> +	start_p4d
> +'
> +
> +test_expect_success 'init depot with UTF-16 encoded file and artificially remove BOM' '
> +	(
> +		cd "$cli" &&
> +		printf "$UTF16" >file1 &&
> +		p4 add -t utf16 file1 &&
> +		p4 submit -d "file1"
> +	) &&
> +
> +	(
> +		cd "db" &&
> +		p4d -jc &&
> +		# P4D automatically adds a BOM. Remove it here to make the file invalid.
> +		sed -e "$ d" depot/file1,v >depot/file1,v.new &&

Do you need the space between the address $ (i.e. the last line) and
operation 'd' (i.e. delete it)?  I am asking because that looks very
unusual at least in our codebase.

> +		mv -- depot/file1,v.new depot/file1,v &&
> +		printf "@$UTF16@" >>depot/file1,v &&
> +		p4d -jrF checkpoint.1
> +	)
> +'
> +
> +test_expect_failure 'clone depot with invalid UTF-16 file in verbose mode' '
> +	git p4 clone --dest="$git" --verbose //depot &&
> +	test_when_finished cleanup_git &&
> +	(
> +		cd "$git" &&
> +		printf "$UTF16" >expect &&
> +		test_cmp_bin expect file1
> +	)
> +'
> +
> +test_expect_failure 'clone depot with invalid UTF-16 file in non-verbose mode' '
> +	git p4 clone --dest="$git" //depot
> +'
> +
> +test_expect_success 'kill p4d' '
> +	kill_p4d
> +'
> +
> +test_done