From: "brian m. carlson" <sandals@crustytoothpaste•net>
To: <git@vger•kernel.org>
Cc: Junio C Hamano <gitster@pobox•com>,
Patrick Steinhardt <ps@pks•im>,
Ezekiel Newren <ezekielnewren@gmail•com>
Subject: [PATCH 13/14] rust: add a small wrapper around the hashfile code
Date: Mon, 27 Oct 2025 00:44:03 +0000 [thread overview]
Message-ID: <20251027004404.2152927-14-sandals@crustytoothpaste.net> (raw)
In-Reply-To: <20251027004404.2152927-1-sandals@crustytoothpaste.net>
Our new binary loose object map code avoids needing to be intimately
involved with file handling by simply writing data to an object
implement Write. This makes it very easy to test by writing to a Cursor
wrapping a Vec for tests, and thus decouples it from intimate knowledge
about how we handle files.
However, we will actually want to write our data to an actual file,
since that's the most practical way to persist data. Implement a
wrapper around the hashfile code that implements the Write trait so that
we can write our loose object map into a file.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste•net>
---
Makefile | 1 +
src/csum_file.rs | 81 ++++++++++++++++++++++++++++++++++++++++++++++++
src/lib.rs | 1 +
src/meson.build | 1 +
4 files changed, 84 insertions(+)
create mode 100644 src/csum_file.rs
diff --git a/Makefile b/Makefile
index 2081b13780..8eb31aeed2 100644
--- a/Makefile
+++ b/Makefile
@@ -1521,6 +1521,7 @@ CLAR_TEST_OBJS += $(UNIT_TEST_DIR)/unit-test.o
UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/test-lib.o
+RUST_SOURCES += src/csum_file.rs
RUST_SOURCES += src/hash.rs
RUST_SOURCES += src/lib.rs
RUST_SOURCES += src/loose.rs
diff --git a/src/csum_file.rs b/src/csum_file.rs
new file mode 100644
index 0000000000..7f2c6c4fcb
--- /dev/null
+++ b/src/csum_file.rs
@@ -0,0 +1,81 @@
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation: version 2 of the License, dated June 1991.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, see <https://www.gnu.org/licenses/>.
+
+use crate::hash::{HashAlgorithm, GIT_MAX_RAWSZ};
+use std::ffi::CStr;
+use std::io::{self, Write};
+use std::os::raw::c_void;
+
+/// A writer that can write files identified by their hash or containing a trailing hash.
+pub struct HashFile {
+ ptr: *mut c_void,
+ algo: HashAlgorithm,
+}
+
+impl HashFile {
+ /// Create a new HashFile.
+ ///
+ /// The hash used will be `algo`, its name should be in `name`, and an open file descriptor
+ /// pointing to that file should be in `fd`.
+ pub fn new(algo: HashAlgorithm, fd: i32, name: &CStr) -> HashFile {
+ HashFile {
+ ptr: unsafe { c::hashfd(algo.hash_algo_ptr(), fd, name.as_ptr()) },
+ algo,
+ }
+ }
+
+ /// Finalize this HashFile instance.
+ ///
+ /// Returns the hash computed over the data.
+ pub fn finalize(self, component: u32, flags: u32) -> Vec<u8> {
+ let mut result = vec![0u8; GIT_MAX_RAWSZ];
+ unsafe { c::finalize_hashfile(self.ptr, result.as_mut_ptr(), component, flags) };
+ result.truncate(self.algo.raw_len());
+ result
+ }
+}
+
+impl Write for HashFile {
+ fn write(&mut self, data: &[u8]) -> io::Result<usize> {
+ for chunk in data.chunks(u32::MAX as usize) {
+ unsafe {
+ c::hashwrite(
+ self.ptr,
+ chunk.as_ptr() as *const c_void,
+ chunk.len() as u32,
+ )
+ };
+ }
+ Ok(data.len())
+ }
+
+ fn flush(&mut self) -> io::Result<()> {
+ unsafe { c::hashflush(self.ptr) };
+ Ok(())
+ }
+}
+
+pub mod c {
+ use std::os::raw::{c_char, c_int, c_void};
+
+ extern "C" {
+ pub fn hashfd(algop: *const c_void, fd: i32, name: *const c_char) -> *mut c_void;
+ pub fn hashwrite(f: *mut c_void, data: *const c_void, len: u32);
+ pub fn hashflush(f: *mut c_void);
+ pub fn finalize_hashfile(
+ f: *mut c_void,
+ data: *mut u8,
+ component: u32,
+ flags: u32,
+ ) -> c_int;
+ }
+}
diff --git a/src/lib.rs b/src/lib.rs
index 442f9433dc..0c598298b1 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -1,3 +1,4 @@
+pub mod csum_file;
pub mod hash;
pub mod loose;
pub mod varint;
diff --git a/src/meson.build b/src/meson.build
index 1eea068519..45739957b4 100644
--- a/src/meson.build
+++ b/src/meson.build
@@ -1,4 +1,5 @@
libgit_rs_sources = [
+ 'csum_file.rs',
'hash.rs',
'lib.rs',
'loose.rs',
next prev parent reply other threads:[~2025-10-27 0:44 UTC|newest]
Thread overview: 118+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-10-27 0:43 [PATCH 00/14] SHA-1/SHA-256 interoperability, part 2 brian m. carlson
2025-10-27 0:43 ` [PATCH 01/14] repository: require Rust support for interoperability brian m. carlson
2025-10-28 9:16 ` Patrick Steinhardt
2025-10-27 0:43 ` [PATCH 02/14] conversion: don't crash when no destination algo brian m. carlson
2025-10-27 0:43 ` [PATCH 03/14] hash: use uint32_t for object_id algorithm brian m. carlson
2025-10-28 9:16 ` Patrick Steinhardt
2025-10-28 18:28 ` Ezekiel Newren
2025-10-28 19:33 ` Junio C Hamano
2025-10-28 19:58 ` Ezekiel Newren
2025-10-28 20:20 ` Junio C Hamano
2025-10-30 0:23 ` brian m. carlson
2025-10-30 1:58 ` Collin Funk
2025-11-03 1:30 ` brian m. carlson
2025-10-29 0:33 ` brian m. carlson
2025-10-29 9:07 ` Patrick Steinhardt
2025-10-27 0:43 ` [PATCH 04/14] rust: add a ObjectID struct brian m. carlson
2025-10-28 9:17 ` Patrick Steinhardt
2025-10-28 19:07 ` Ezekiel Newren
2025-10-29 0:42 ` brian m. carlson
2025-10-28 19:40 ` Junio C Hamano
2025-10-29 0:47 ` brian m. carlson
2025-10-29 0:36 ` brian m. carlson
2025-10-29 9:08 ` Patrick Steinhardt
2025-10-30 0:32 ` brian m. carlson
2025-10-27 0:43 ` [PATCH 05/14] rust: add a hash algorithm abstraction brian m. carlson
2025-10-28 9:18 ` Patrick Steinhardt
2025-10-28 17:09 ` Ezekiel Newren
2025-10-28 20:00 ` Junio C Hamano
2025-10-28 20:03 ` Ezekiel Newren
2025-10-29 13:27 ` Junio C Hamano
2025-10-29 14:32 ` Junio C Hamano
2025-10-27 0:43 ` [PATCH 06/14] hash: add a function to look up hash algo structs brian m. carlson
2025-10-28 9:18 ` Patrick Steinhardt
2025-10-28 20:12 ` Junio C Hamano
2025-11-04 1:48 ` brian m. carlson
2025-11-04 10:24 ` Junio C Hamano
2025-10-27 0:43 ` [PATCH 07/14] csum-file: define hashwrite's count as a uint32_t brian m. carlson
2025-10-28 17:22 ` Ezekiel Newren
2025-10-27 0:43 ` [PATCH 08/14] write-or-die: add an fsync component for the loose object map brian m. carlson
2025-10-27 0:43 ` [PATCH 09/14] hash: expose hash context functions to Rust brian m. carlson
2025-10-29 16:32 ` Junio C Hamano
2025-10-30 21:42 ` brian m. carlson
2025-10-30 21:52 ` Junio C Hamano
2025-10-27 0:44 ` [PATCH 10/14] rust: add a build.rs script for tests brian m. carlson
2025-10-28 9:18 ` Patrick Steinhardt
2025-10-28 17:42 ` Ezekiel Newren
2025-10-29 16:43 ` Junio C Hamano
2025-10-29 22:10 ` Ezekiel Newren
2025-10-29 23:12 ` Junio C Hamano
2025-10-30 6:26 ` Patrick Steinhardt
2025-10-30 13:54 ` Junio C Hamano
2025-10-31 22:43 ` Ezekiel Newren
2025-11-01 11:18 ` Junio C Hamano
2025-10-27 0:44 ` [PATCH 11/14] rust: add functionality to hash an object brian m. carlson
2025-10-28 9:18 ` Patrick Steinhardt
2025-10-29 0:53 ` brian m. carlson
2025-10-29 9:07 ` Patrick Steinhardt
2025-10-28 18:05 ` Ezekiel Newren
2025-10-29 1:05 ` brian m. carlson
2025-10-29 16:02 ` Ben Knoble
2025-10-27 0:44 ` [PATCH 12/14] rust: add a new binary loose object map format brian m. carlson
2025-10-28 9:18 ` Patrick Steinhardt
2025-10-29 1:37 ` brian m. carlson
2025-10-29 9:07 ` Patrick Steinhardt
2025-10-29 17:03 ` Junio C Hamano
2025-10-29 18:21 ` Junio C Hamano
2025-10-27 0:44 ` brian m. carlson [this message]
2025-10-28 18:19 ` [PATCH 13/14] rust: add a small wrapper around the hashfile code Ezekiel Newren
2025-10-29 1:39 ` brian m. carlson
2025-10-27 0:44 ` [PATCH 14/14] object-file-convert: always make sure object ID algo is valid brian m. carlson
2025-10-29 20:07 ` [PATCH 00/14] SHA-1/SHA-256 interoperability, part 2 Junio C Hamano
2025-10-29 20:15 ` Junio C Hamano
2025-11-11 0:12 ` Ezekiel Newren
2025-11-14 17:25 ` Junio C Hamano
2025-11-14 21:11 ` Junio C Hamano
2025-11-17 6:56 ` Junio C Hamano
2025-11-17 22:09 ` brian m. carlson
2025-11-18 0:13 ` Junio C Hamano
2025-11-19 23:04 ` brian m. carlson
2025-11-19 23:24 ` Junio C Hamano
2025-11-19 23:37 ` Ezekiel Newren
2025-11-20 19:52 ` Ezekiel Newren
2025-11-20 23:02 ` brian m. carlson
2025-11-20 23:11 ` Ezekiel Newren
2025-11-20 23:14 ` Junio C Hamano
2025-11-17 22:16 ` [PATCH v2 00/15] " brian m. carlson
2025-11-17 22:16 ` [PATCH v2 01/15] repository: require Rust support for interoperability brian m. carlson
2025-11-17 22:16 ` [PATCH v2 02/15] conversion: don't crash when no destination algo brian m. carlson
2025-11-17 22:16 ` [PATCH v2 03/15] hash: use uint32_t for object_id algorithm brian m. carlson
2025-11-17 22:16 ` [PATCH v2 04/15] rust: add a ObjectID struct brian m. carlson
2025-11-17 22:16 ` [PATCH v2 05/15] rust: add a hash algorithm abstraction brian m. carlson
2025-11-17 22:16 ` [PATCH v2 06/15] hash: add a function to look up hash algo structs brian m. carlson
2025-11-17 22:16 ` [PATCH v2 07/15] rust: add additional helpers for ObjectID brian m. carlson
2025-11-17 22:16 ` [PATCH v2 08/15] csum-file: define hashwrite's count as a uint32_t brian m. carlson
2025-11-17 22:16 ` [PATCH v2 09/15] write-or-die: add an fsync component for the object map brian m. carlson
2025-11-17 22:16 ` [PATCH v2 10/15] hash: expose hash context functions to Rust brian m. carlson
2025-11-17 22:16 ` [PATCH v2 11/15] rust: add a build.rs script for tests brian m. carlson
2025-11-17 22:16 ` [PATCH v2 12/15] rust: add functionality to hash an object brian m. carlson
2025-11-17 22:16 ` [PATCH v2 13/15] rust: add a new binary object map format brian m. carlson
2025-11-17 22:16 ` [PATCH v2 14/15] rust: add a small wrapper around the hashfile code brian m. carlson
2025-11-17 22:16 ` [PATCH v2 15/15] object-file-convert: always make sure object ID algo is valid brian m. carlson
2026-02-07 20:04 ` [PATCH v3 00/16] SHA-1/SHA-256 interoperability, part 2 brian m. carlson
2026-02-07 20:04 ` [PATCH v3 01/16] repository: require Rust support for interoperability brian m. carlson
2026-02-07 20:04 ` [PATCH v3 02/16] conversion: don't crash when no destination algo brian m. carlson
2026-02-07 20:04 ` [PATCH v3 03/16] hash: use uint32_t for object_id algorithm brian m. carlson
2026-02-07 20:04 ` [PATCH v3 04/16] rust: add a ObjectID struct brian m. carlson
2026-02-07 20:04 ` [PATCH v3 05/16] rust: add a hash algorithm abstraction brian m. carlson
2026-02-07 20:04 ` [PATCH v3 06/16] hash: add a function to look up hash algo structs brian m. carlson
2026-02-07 20:04 ` [PATCH v3 07/16] rust: add additional helpers for ObjectID brian m. carlson
2026-02-07 20:04 ` [PATCH v3 08/16] csum-file: define hashwrite's count as a uint32_t brian m. carlson
2026-02-07 20:04 ` [PATCH v3 09/16] write-or-die: add an fsync component for the object map brian m. carlson
2026-02-07 20:04 ` [PATCH v3 10/16] hash: expose hash context functions to Rust brian m. carlson
2026-02-07 20:04 ` [PATCH v3 11/16] rust: fix linking binaries with cargo brian m. carlson
2026-02-07 20:04 ` [PATCH v3 12/16] rust: add a build.rs script for tests brian m. carlson
2026-02-07 20:04 ` [PATCH v3 13/16] rust: add functionality to hash an object brian m. carlson
2026-02-07 20:04 ` [PATCH v3 14/16] rust: add a new binary object map format brian m. carlson
2026-02-07 20:04 ` [PATCH v3 15/16] rust: add a small wrapper around the hashfile code brian m. carlson
2026-02-07 20:04 ` [PATCH v3 16/16] object-file-convert: always make sure object ID algo is valid brian m. carlson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20251027004404.2152927-14-sandals@crustytoothpaste.net \
--to=sandals@crustytoothpaste$(echo .)net \
--cc=ezekielnewren@gmail$(echo .)com \
--cc=git@vger$(echo .)kernel.org \
--cc=gitster@pobox$(echo .)com \
--cc=ps@pks$(echo .)im \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox