From: Eric Ju <eric.peijian@gmail•com>
To: git@vger•kernel.org
Cc: calvinwan@google•com, jonathantanmy@google•com,
chriscool@tuxfamily•org, eric.peijian@gmail•com,
karthik.188@gmail•com, toon@iotcl•com, jltobler@gmail•com
Subject: [PATCH v11 0/8] cat-file: add remote-object-info to batch-command
Date: Fri, 21 Feb 2025 14:04:41 -0500 [thread overview]
Message-ID: <20250221190451.12536-1-eric.peijian@gmail.com> (raw)
In-Reply-To: <20240628190503.67389-1-eric.peijian@gmail.com>
This patch series is a continuation of Calvin Wan’s (calvinwan@google•com)
patch series [PATCH v5 0/6] cat-file: add --batch-command remote-object-info
command at [1].
Sometimes it is beneficial to retrieve information about an object without
having to download it completely. The server logic for retrieving size has
already been implemented and merged in "a2ba162cda (object-info: support for
retrieving object info, 2021-04-20)"[2]. This patch series implement the client
option for it.
This patch series add the `remote-object-info` command to
`cat-file --batch-command`. This command allows the client to make an
object-info command request to a server that supports protocol v2.
If the server uses protocol v2 but does not support the object-info capability,
`cat-file --batch-command` will die.
If a user attempts to use `remote-object-info` with protocol v1,,
`cat-file --batch-command` will die.
Currently, only the size (%(objectsize)) is supported in this implementation.
The type (%(objecttype)) is not included in this patch series, as it is not yet
supported on the server side either. The plan is to implement the necessary
logic for both the server and client in a subsequent series.
The default format for remote-object-info is set to %(objectname) %(objectsize).
Once %(objecttype) is supported, the default format will be unified accordingly.
If the batch command format includes unsupported fields such as %(objecttype),
%(objectsize:disk), or %(deltabase), the command will terminate with an error.
Changes since V10
================
- Add a check on command input to prevent overflow.
- Add other checks to prevent potential abuse.
Calvin Wan (4):
fetch-pack: refactor packet writing
fetch-pack: move fetch initialization
serve: advertise object-info feature
transport: add client support for object-info
Eric Ju (4):
git-compat-util: add strtoul_ul() with error handling
cat-file: add declaration of variable i inside its for loop
t1006: split test utility functions into new "lib-cat-file.sh"
cat-file: add remote-object-info to batch-command
Documentation/git-cat-file.adoc | 24 +-
Makefile | 1 +
builtin/cat-file.c | 125 ++++-
connect.c | 34 ++
connect.h | 8 +
fetch-object-info.c | 85 ++++
fetch-object-info.h | 22 +
fetch-pack.c | 51 +-
fetch-pack.h | 2 +
git-compat-util.h | 20 +
object-file.c | 11 +
object-store-ll.h | 3 +
serve.c | 4 +-
t/lib-cat-file.sh | 16 +
t/t1006-cat-file.sh | 13 +-
t/t1017-cat-file-remote-object-info.sh | 664 +++++++++++++++++++++++++
transport-helper.c | 11 +-
transport.c | 28 +-
transport.h | 11 +
19 files changed, 1065 insertions(+), 68 deletions(-)
create mode 100644 fetch-object-info.c
create mode 100644 fetch-object-info.h
create mode 100644 t/lib-cat-file.sh
create mode 100755 t/t1017-cat-file-remote-object-info.sh
Range-diff against v10:
1: a4a5aefa3e = 1: 814c53b402 git-compat-util: add strtoul_ul() with error handling
2: c67e79804e = 2: 04f41100c4 cat-file: add declaration of variable i inside its for loop
3: 7f0b824714 = 3: 3af67e6648 t1006: split test utility functions into new "lib-cat-file.sh"
4: 0d22d6af6e = 4: cb1088e436 fetch-pack: refactor packet writing
5: 34c34c7464 = 5: 614daac4bb fetch-pack: move fetch initialization
6: 54dd237c45 = 6: 4bc403fa2c serve: advertise object-info feature
7: 90a3d987d5 ! 7: adae08d5a8 transport: add client support for object-info
@@ transport.c: static int fetch_refs_via_pack(struct transport *transport,
args.reject_shallow_remote = transport->smart_options->reject_shallow;
+ args.object_info = transport->smart_options->object_info;
+
-+ if (transport->smart_options && transport->smart_options->object_info
++ if (transport->smart_options->object_info
+ && transport->smart_options->object_info_oids->nr > 0) {
+ struct packet_reader reader;
+ struct object_info_args obj_info_args = { 0 };
8: 9d932c2cb2 ! 8: 975d39cb6a cat-file: add remote-object-info to batch-command
@@ builtin/cat-file.c
+#include "alias.h"
+#include "remote.h"
+#include "transport.h"
++
++/* Maximum length for a remote URL. While no universal standard exists,
++ * 8K is assumed to be a reasonable limit.
++ */
++#define MAX_REMOTE_URL_LEN (8*1024)
++/* Maximum number of objects allowed in a single remote-object-info request. */
++#define MAX_ALLOWED_OBJ_LIMIT 10000
++/* Maximum input size permitted for the remote-object-info command. */
++#define MAX_REMOTE_OBJ_INFO_LINE (MAX_REMOTE_URL_LEN + MAX_ALLOWED_OBJ_LIMIT * (GIT_MAX_HEXSZ + 1))
enum batch_mode {
BATCH_MODE_CONTENTS,
@@ builtin/cat-file.c: static void parse_cmd_info(struct batch_options *opt,
+{
+ int count;
+ const char **argv;
++ char *line_to_split;
++
++ if (strlen(line) >= MAX_REMOTE_OBJ_INFO_LINE)
++ die(_("remote-object-info command input overflow "
++ "(no more than %d objects are allowed)"),
++ MAX_ALLOWED_OBJ_LIMIT);
+
-+ char *line_to_split = xstrdup_or_null(line);
++ line_to_split = xstrdup(line);
+ count = split_cmdline(line_to_split, &argv);
++ if (count < 0)
++ die(_("split remote-object-info command"));
++
+ if (get_remote_info(opt, count, argv))
+ goto cleanup;
+
--
2.48.1
next prev parent reply other threads:[~2025-02-21 19:05 UTC|newest]
Thread overview: 175+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-06-28 19:04 [PATCH 0/6] cat-file: add remote-object-info to batch-command Eric Ju
2024-06-28 19:04 ` [PATCH 1/6] fetch-pack: refactor packet writing Eric Ju
2024-07-04 16:59 ` Karthik Nayak
2024-07-08 15:17 ` Peijian Ju
2024-07-10 9:39 ` Karthik Nayak
2024-07-15 16:40 ` Peijian Ju
2024-06-28 19:04 ` [PATCH 2/6] fetch-pack: move fetch initialization Eric Ju
2024-06-28 19:05 ` [PATCH 3/6] serve: advertise object-info feature Eric Ju
2024-06-28 19:05 ` [PATCH 4/6] transport: add client support for object-info Eric Ju
2024-07-09 7:15 ` Toon claes
2024-07-09 16:37 ` Junio C Hamano
2024-07-13 2:32 ` Peijian Ju
2024-07-13 2:30 ` Peijian Ju
2024-07-10 10:13 ` Karthik Nayak
2024-07-16 2:39 ` Peijian Ju
2024-06-28 19:05 ` [PATCH 5/6] cat-file: add declaration of variable i inside its for loop Eric Ju
2024-07-10 10:16 ` Karthik Nayak
2024-07-16 2:59 ` Peijian Ju
2024-06-28 19:05 ` [PATCH 6/6] cat-file: add remote-object-info to batch-command Eric Ju
2024-07-09 1:50 ` Justin Tobler
2024-07-12 17:41 ` Peijian Ju
2024-07-09 7:16 ` Toon claes
2024-07-13 2:35 ` Peijian Ju
2024-07-10 12:08 ` Karthik Nayak
2024-07-17 2:38 ` Peijian Ju
2024-07-20 3:43 ` [PATCH v2 0/6] " Eric Ju
2024-07-20 3:43 ` [PATCH v2 1/6] fetch-pack: refactor packet writing Eric Ju
2024-09-24 11:45 ` Christian Couder
2024-09-25 20:42 ` Peijian Ju
2024-07-20 3:43 ` [PATCH v2 2/6] fetch-pack: move fetch initialization Eric Ju
2024-07-20 3:43 ` [PATCH v2 3/6] serve: advertise object-info feature Eric Ju
2024-07-20 3:43 ` [PATCH v2 4/6] transport: add client support for object-info Eric Ju
2024-09-24 11:45 ` Christian Couder
2024-09-24 17:29 ` Junio C Hamano
2024-09-25 18:29 ` Peijian Ju
2024-07-20 3:43 ` [PATCH v2 5/6] cat-file: add declaration of variable i inside its for loop Eric Ju
2024-07-20 3:43 ` [PATCH v2 6/6] cat-file: add remote-object-info to batch-command Eric Ju
2024-09-11 13:11 ` Toon Claes
2024-09-25 18:18 ` Peijian Ju
2024-09-24 12:13 ` Christian Couder
2024-09-25 18:12 ` Peijian Ju
2024-08-22 21:24 ` [PATCH 0/6] " Peijian Ju
2024-09-26 1:38 ` [PATCH v3 " Eric Ju
2024-09-26 1:38 ` [PATCH v3 1/6] fetch-pack: refactor packet writing Eric Ju
2024-09-26 1:38 ` [PATCH v3 2/6] fetch-pack: move fetch initialization Eric Ju
2024-09-26 1:38 ` [PATCH v3 3/6] serve: advertise object-info feature Eric Ju
2024-09-26 1:38 ` [PATCH v3 4/6] transport: add client support for object-info Eric Ju
2024-10-23 9:48 ` Christian Couder
2024-10-24 20:23 ` Peijian Ju
2024-09-26 1:38 ` [PATCH v3 5/6] cat-file: add declaration of variable i inside its for loop Eric Ju
2024-09-26 1:38 ` [PATCH v3 6/6] cat-file: add remote-object-info to batch-command Eric Ju
2024-10-23 9:49 ` Christian Couder
2024-10-23 20:25 ` Taylor Blau
2024-10-24 20:28 ` Peijian Ju
2024-10-24 20:28 ` Peijian Ju
2024-10-24 20:53 ` [PATCH v4 0/6] " Eric Ju
2024-10-24 20:53 ` [PATCH v4 1/6] fetch-pack: refactor packet writing Eric Ju
2024-10-25 9:52 ` karthik nayak
2024-10-25 16:06 ` Peijian Ju
2024-10-24 20:53 ` [PATCH v4 2/6] fetch-pack: move fetch initialization Eric Ju
2024-10-24 20:53 ` [PATCH v4 3/6] serve: advertise object-info feature Eric Ju
2024-10-24 20:53 ` [PATCH v4 4/6] transport: add client support for object-info Eric Ju
2024-10-25 10:12 ` karthik nayak
2024-10-28 5:39 ` Peijian Ju
2024-10-24 20:53 ` [PATCH v4 5/6] cat-file: add declaration of variable i inside its for loop Eric Ju
2024-10-24 20:53 ` [PATCH v4 6/6] cat-file: add remote-object-info to batch-command Eric Ju
2024-10-25 10:53 ` karthik nayak
2024-10-25 13:55 ` Christian Couder
2024-10-25 20:56 ` [PATCH v4 0/6] " Taylor Blau
2024-10-27 3:54 ` Peijian Ju
2024-10-28 0:01 ` Taylor Blau
2024-10-28 20:34 ` [PATCH v5 " Eric Ju
2024-10-28 20:34 ` [PATCH v5 1/6] fetch-pack: refactor packet writing Eric Ju
2024-11-05 17:44 ` Christian Couder
2024-11-06 1:06 ` Junio C Hamano
2024-11-06 18:00 ` Peijian Ju
2024-11-06 19:50 ` Peijian Ju
2024-10-28 20:34 ` [PATCH v5 2/6] fetch-pack: move fetch initialization Eric Ju
2024-10-28 20:34 ` [PATCH v5 3/6] serve: advertise object-info feature Eric Ju
2024-10-28 20:34 ` [PATCH v5 4/6] transport: add client support for object-info Eric Ju
2024-10-28 20:34 ` [PATCH v5 5/6] cat-file: add declaration of variable i inside its for loop Eric Ju
2024-10-28 20:34 ` [PATCH v5 6/6] cat-file: add remote-object-info to batch-command Eric Ju
2024-11-08 16:24 ` [PATCH v6 0/6] " Eric Ju
2024-11-08 16:24 ` [PATCH v6 1/6] cat-file: add declaration of variable i inside its for loop Eric Ju
2024-11-08 16:24 ` [PATCH v6 2/6] fetch-pack: refactor packet writing Eric Ju
2024-11-08 16:24 ` [PATCH v6 3/6] fetch-pack: move fetch initialization Eric Ju
2024-11-08 16:24 ` [PATCH v6 4/6] serve: advertise object-info feature Eric Ju
2024-11-08 16:24 ` [PATCH v6 5/6] transport: add client support for object-info Eric Ju
2024-11-08 16:24 ` [PATCH v6 6/6] cat-file: add remote-object-info to batch-command Eric Ju
2024-11-11 4:38 ` [PATCH v6 0/6] " Junio C Hamano
2024-11-18 16:28 ` Peijian Ju
2024-11-19 0:16 ` Junio C Hamano
2024-11-19 6:31 ` Patrick Steinhardt
2024-11-19 6:48 ` Junio C Hamano
2024-11-19 16:35 ` Peijian Ju
2024-11-20 1:19 ` Junio C Hamano
2024-11-25 5:36 ` [PATCH v7 " Eric Ju
2024-11-25 5:36 ` [PATCH v7 1/6] cat-file: add declaration of variable i inside its for loop Eric Ju
2024-11-25 9:51 ` Patrick Steinhardt
2024-12-03 19:26 ` Peijian Ju
2024-11-25 5:36 ` [PATCH v7 2/6] fetch-pack: refactor packet writing Eric Ju
2024-11-25 9:51 ` Patrick Steinhardt
2024-12-03 19:09 ` Peijian Ju
2024-11-25 5:36 ` [PATCH v7 3/6] fetch-pack: move fetch initialization Eric Ju
2024-11-25 5:36 ` [PATCH v7 4/6] serve: advertise object-info feature Eric Ju
2024-11-25 5:36 ` [PATCH v7 5/6] transport: add client support for object-info Eric Ju
2024-11-25 9:51 ` Patrick Steinhardt
2024-12-03 3:15 ` Peijian Ju
2024-11-25 5:36 ` [PATCH v7 6/6] cat-file: add remote-object-info to batch-command Eric Ju
2024-11-25 9:51 ` Patrick Steinhardt
2024-12-03 19:23 ` Peijian Ju
2024-12-05 9:50 ` Patrick Steinhardt
2024-12-05 10:34 ` Christian Couder
2024-12-23 23:25 ` [PATCH v8 0/6] " Eric Ju
2024-12-23 23:25 ` [PATCH v8 1/6] cat-file: add declaration of variable i inside its for loop Eric Ju
2024-12-23 23:25 ` [PATCH v8 2/6] fetch-pack: refactor packet writing Eric Ju
2024-12-23 23:25 ` [PATCH v8 3/6] fetch-pack: move fetch initialization Eric Ju
2024-12-23 23:25 ` [PATCH v8 4/6] serve: advertise object-info feature Eric Ju
2024-12-23 23:25 ` [PATCH v8 5/6] transport: add client support for object-info Eric Ju
2025-01-07 18:31 ` Calvin Wan
2025-01-07 18:53 ` Junio C Hamano
2025-01-08 15:55 ` Peijian Ju
2024-12-23 23:25 ` [PATCH v8 6/6] cat-file: add remote-object-info to batch-command Eric Ju
2025-01-07 21:29 ` Calvin Wan
2024-12-26 21:56 ` [PATCH v8 0/6] " Junio C Hamano
2024-12-30 23:25 ` Peijian Ju
2025-01-08 18:37 ` [PATCH v9 0/8] cat-file: " Eric Ju
2025-01-08 18:37 ` [PATCH v9 1/8] git-compat-util: add strtoul_ul() with error handling Eric Ju
2025-01-10 11:33 ` Christian Couder
2025-01-14 1:39 ` Peijian Ju
2025-01-08 18:37 ` [PATCH v9 2/8] cat-file: add declaration of variable i inside its for loop Eric Ju
2025-01-10 11:39 ` Christian Couder
2025-01-14 1:36 ` Peijian Ju
2025-01-08 18:37 ` [PATCH v9 3/8] cat-file: split test utility functions into a separate library file Eric Ju
2025-01-10 14:26 ` Christian Couder
2025-01-14 1:33 ` Peijian Ju
2025-01-08 18:37 ` [PATCH v9 4/8] fetch-pack: refactor packet writing Eric Ju
2025-01-08 18:37 ` [PATCH v9 5/8] fetch-pack: move fetch initialization Eric Ju
2025-01-08 18:37 ` [PATCH v9 6/8] serve: advertise object-info feature Eric Ju
2025-01-08 18:37 ` [PATCH v9 7/8] transport: add client support for object-info Eric Ju
2025-01-08 18:37 ` [PATCH v9 8/8] cat-file: add remote-object-info to batch-command Eric Ju
2025-01-10 11:20 ` Christian Couder
2025-01-14 1:24 ` Peijian Ju
2025-01-14 2:14 ` [PATCH v10 0/8] " Eric Ju
2025-01-14 2:14 ` [PATCH v10 1/8] git-compat-util: add strtoul_ul() with error handling Eric Ju
2025-01-14 2:14 ` [PATCH v10 2/8] cat-file: add declaration of variable i inside its for loop Eric Ju
2025-01-14 2:14 ` [PATCH v10 3/8] t1006: split test utility functions into new "lib-cat-file.sh" Eric Ju
2025-01-14 2:14 ` [PATCH v10 4/8] fetch-pack: refactor packet writing Eric Ju
2025-01-14 2:14 ` [PATCH v10 5/8] fetch-pack: move fetch initialization Eric Ju
2025-01-14 2:14 ` [PATCH v10 6/8] serve: advertise object-info feature Eric Ju
2025-01-14 2:14 ` [PATCH v10 7/8] transport: add client support for object-info Eric Ju
2025-02-01 2:08 ` Jeff King
2025-02-20 22:52 ` Peijian Ju
2025-01-14 2:15 ` [PATCH v10 8/8] cat-file: add remote-object-info to batch-command Eric Ju
2025-02-01 2:03 ` Jeff King
2025-02-21 15:34 ` Peijian Ju
2025-02-24 23:45 ` Jeff King
2025-03-12 19:53 ` Peijian Ju
2025-02-21 19:04 ` Eric Ju [this message]
2025-02-21 19:04 ` [PATCH v11 1/8] git-compat-util: add strtoul_ul() with error handling Eric Ju
2025-02-21 19:04 ` [PATCH v11 2/8] cat-file: add declaration of variable i inside its for loop Eric Ju
2025-02-21 19:04 ` [PATCH v11 3/8] t1006: split test utility functions into new "lib-cat-file.sh" Eric Ju
2025-02-21 19:04 ` [PATCH v11 4/8] fetch-pack: refactor packet writing Eric Ju
2025-02-21 19:04 ` [PATCH v11 5/8] fetch-pack: move fetch initialization Eric Ju
2025-02-21 19:04 ` [PATCH v11 6/8] serve: advertise object-info feature Eric Ju
2025-02-21 19:04 ` [PATCH v11 7/8] transport: add client support for object-info Eric Ju
2025-02-21 19:04 ` [PATCH v11 8/8] cat-file: add remote-object-info to batch-command Eric Ju
2025-02-24 20:46 ` Junio C Hamano
2025-03-11 23:10 ` Peijian Ju
2025-02-24 23:47 ` Jeff King
2025-03-12 2:19 ` Peijian Ju
2025-03-13 6:02 ` Jeff King
2025-03-21 18:24 ` Peijian Ju
2025-03-24 3:39 ` Jeff King
2026-03-12 21:41 ` [GSoC] " Pablo Sabater
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250221190451.12536-1-eric.peijian@gmail.com \
--to=eric.peijian@gmail$(echo .)com \
--cc=calvinwan@google$(echo .)com \
--cc=chriscool@tuxfamily$(echo .)org \
--cc=git@vger$(echo .)kernel.org \
--cc=jltobler@gmail$(echo .)com \
--cc=jonathantanmy@google$(echo .)com \
--cc=karthik.188@gmail$(echo .)com \
--cc=toon@iotcl$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox