From: "Shawn O. Pearce" <spearce@spearce•org>
To: Robin Rosenberg <robin.rosenberg@dewire•com>
Cc: git@vger•kernel.org
Subject: [JGIT PATCH v2 09/10] Add test cases for loading new (or replaced) pack files
Date: Mon, 20 Apr 2009 19:10:35 -0700 [thread overview]
Message-ID: <20090421021035.GB23604@spearce.org> (raw)
In-Reply-To: <1240276872-17893-10-git-send-email-spearce@spearce.org>
Originally-by: Robin Rosenberg <robin.rosenberg@dewire•com>
Signed-off-by: Shawn O. Pearce <spearce@spearce•org>
---
Matches with v2 08/10. Tests take a few seconds longer now as
we have to delay at least 1 second on most filesystems; longer if
their modification timestamp granularity is larger than 1 second.
.../org/spearce/jgit/lib/ConcurrentRepackTest.java | 222 ++++++++++++++++++++
1 files changed, 222 insertions(+), 0 deletions(-)
create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/lib/ConcurrentRepackTest.java
diff --git a/org.spearce.jgit.test/tst/org/spearce/jgit/lib/ConcurrentRepackTest.java b/org.spearce.jgit.test/tst/org/spearce/jgit/lib/ConcurrentRepackTest.java
new file mode 100644
index 0000000..825fbb8
--- /dev/null
+++ b/org.spearce.jgit.test/tst/org/spearce/jgit/lib/ConcurrentRepackTest.java
@@ -0,0 +1,222 @@
+/*
+ * Copyright (C) 2009, Robin Rosenberg <robin.rosenberg@dewire•com>
+ * Copyright (C) 2009, Google Inc.
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials provided
+ * with the distribution.
+ *
+ * - Neither the name of the Git Development Community nor the
+ * names of its contributors may be used to endorse or promote
+ * products derived from this software without specific prior
+ * written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+package org.spearce.jgit.lib;
+
+import java.io.File;
+import java.io.FileOutputStream;
+import java.io.IOException;
+
+import org.spearce.jgit.errors.IncorrectObjectTypeException;
+import org.spearce.jgit.errors.MissingObjectException;
+import org.spearce.jgit.revwalk.RevObject;
+import org.spearce.jgit.revwalk.RevWalk;
+
+public class ConcurrentRepackTest extends RepositoryTestCase {
+ public void setUp() throws Exception {
+ WindowCacheConfig windowCacheConfig = new WindowCacheConfig();
+ windowCacheConfig.setPackedGitOpenFiles(1);
+ WindowCache.reconfigure(windowCacheConfig);
+ super.setUp();
+ }
+
+ protected void tearDown() throws Exception {
+ super.tearDown();
+ WindowCacheConfig windowCacheConfig = new WindowCacheConfig();
+ WindowCache.reconfigure(windowCacheConfig);
+ }
+
+ public void testObjectInNewPack() throws IncorrectObjectTypeException,
+ IOException {
+ // Create a new object in a new pack, and test that it is present.
+ //
+ final Repository eden = createNewEmptyRepo();
+ final RevObject o1 = writeBlob(eden, "o1");
+ pack(eden, o1);
+ assertEquals(o1.name(), parse(o1).name());
+ }
+
+ public void testObjectMovedToNewPack1()
+ throws IncorrectObjectTypeException, IOException {
+ // Create an object and pack it. Then remove that pack and put the
+ // object into a different pack file, with some other object. We
+ // still should be able to access the objects.
+ //
+ final Repository eden = createNewEmptyRepo();
+ final RevObject o1 = writeBlob(eden, "o1");
+ final File[] out1 = pack(eden, o1);
+ assertEquals(o1.name(), parse(o1).name());
+
+ final RevObject o2 = writeBlob(eden, "o2");
+ pack(eden, o2, o1);
+
+ // Force close, and then delete, the old pack.
+ //
+ whackCache();
+ delete(out1);
+
+ // Now here is the interesting thing. Will git figure the new
+ // object exists in the new pack, and not the old one.
+ //
+ assertEquals(o2.name(), parse(o2).name());
+ assertEquals(o1.name(), parse(o1).name());
+ }
+
+ public void testObjectMovedWithinPack()
+ throws IncorrectObjectTypeException, IOException {
+ // Create an object and pack it.
+ //
+ final Repository eden = createNewEmptyRepo();
+ final RevObject o1 = writeBlob(eden, "o1");
+ final File[] out1 = pack(eden, o1);
+ assertEquals(o1.name(), parse(o1).name());
+
+ // Force close the old pack.
+ //
+ whackCache();
+
+ // Now overwrite the old pack in place. This method of creating a
+ // different pack under the same file name is partially broken. We
+ // should also have a different file name because the list of objects
+ // within the pack has been modified.
+ //
+ final RevObject o2 = writeBlob(eden, "o2");
+ final PackWriter pw = new PackWriter(eden, NullProgressMonitor.INSTANCE);
+ pw.addObject(o2);
+ pw.addObject(o1);
+ write(out1, pw);
+
+ // Try the old name, then the new name. The old name should cause the
+ // pack to reload when it opens and the index and pack mismatch.
+ //
+ assertEquals(o1.name(), parse(o1).name());
+ assertEquals(o2.name(), parse(o2).name());
+ }
+
+ private static void whackCache() {
+ final WindowCacheConfig config = new WindowCacheConfig();
+
+ config.setPackedGitOpenFiles(0);
+ WindowCache.reconfigure(config);
+
+ config.setPackedGitOpenFiles(1);
+ WindowCache.reconfigure(config);
+ }
+
+ private RevObject parse(final AnyObjectId id)
+ throws MissingObjectException, IOException {
+ return new RevWalk(db).parseAny(id);
+ }
+
+ private File[] pack(final Repository src, final RevObject... list)
+ throws IOException {
+ final PackWriter pw = new PackWriter(src, NullProgressMonitor.INSTANCE);
+ for (final RevObject o : list) {
+ pw.addObject(o);
+ }
+
+ final ObjectId name = pw.computeName();
+ final File packFile = fullPackFileName(name, ".pack");
+ final File idxFile = fullPackFileName(name, ".idx");
+ final File[] files = new File[] { packFile, idxFile };
+ write(files, pw);
+ return files;
+ }
+
+ private static void write(final File[] files, final PackWriter pw)
+ throws IOException {
+ final long begin = files[0].getParentFile().lastModified();
+ FileOutputStream out;
+
+ out = new FileOutputStream(files[0]);
+ try {
+ pw.writePack(out);
+ } finally {
+ out.close();
+ }
+
+ out = new FileOutputStream(files[1]);
+ try {
+ pw.writeIndex(out);
+ } finally {
+ out.close();
+ }
+
+ touch(begin, files[0].getParentFile());
+ }
+
+ private static void delete(final File[] list) {
+ final long begin = list[0].getParentFile().lastModified();
+ for (final File f : list) {
+ f.delete();
+ assertFalse(f + " was removed", f.exists());
+ }
+ touch(begin, list[0].getParentFile());
+ }
+
+ private static void touch(final long begin, final File dir) {
+ while (begin >= dir.lastModified()) {
+ try {
+ Thread.sleep(25);
+ } catch (InterruptedException ie) {
+ //
+ }
+ dir.setLastModified(System.currentTimeMillis());
+ }
+ }
+
+ private File fullPackFileName(final ObjectId name, final String suffix) {
+ final File packdir = new File(db.getObjectsDirectory(), "pack");
+ return new File(packdir, "pack-" + name.name() + suffix);
+ }
+
+ private RevObject writeBlob(final Repository repo, final String data)
+ throws IOException {
+ final RevWalk revWalk = new RevWalk(repo);
+ final byte[] bytes = data.getBytes(Constants.CHARACTER_ENCODING);
+ final ObjectWriter ow = new ObjectWriter(repo);
+ final ObjectId id = ow.writeBlob(bytes);
+ try {
+ parse(id);
+ fail("Object " + id.name() + " should not exist in test repository");
+ } catch (MissingObjectException e) {
+ // Ok
+ }
+ return revWalk.lookupBlob(id);
+ }
+}
--
1.6.3.rc1.188.ga02b
next prev parent reply other threads:[~2009-04-21 2:12 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-04-21 1:21 [JGIT PATCH 00/10] Object access improvements Shawn O. Pearce
2009-04-21 1:21 ` [JGIT PATCH 01/10] Safely handle closing an already closed WindowedFile Shawn O. Pearce
2009-04-21 1:21 ` [JGIT PATCH 02/10] Change empty tree test case to use a temporary repository Shawn O. Pearce
2009-04-21 1:21 ` [JGIT PATCH 03/10] Replace hand-coded read fully loop with NB.readFully Shawn O. Pearce
2009-04-21 1:21 ` [JGIT PATCH 04/10] Clear the reverse index when closing a PackFile Shawn O. Pearce
2009-04-21 1:21 ` [JGIT PATCH 05/10] Introduce a new exception type for an in-place pack modification Shawn O. Pearce
2009-04-21 1:21 ` [JGIT PATCH 06/10] Refactor object database access with new abstraction Shawn O. Pearce
2009-04-21 1:21 ` [JGIT PATCH 07/10] Rescan packs if a pack is modified in-place (part 1) Shawn O. Pearce
2009-04-21 1:21 ` [JGIT PATCH 08/10] Scan for new packs if GIT_DIR/objects/pack has been modified Shawn O. Pearce
2009-04-21 1:21 ` [JGIT PATCH 09/10] Add test cases for loading new (or replaced) pack files Shawn O. Pearce
2009-04-21 1:21 ` [JGIT PATCH 10/10] BROKEN TEST: ObjectLoader stays valid across repacks Shawn O. Pearce
2009-04-21 23:16 ` Robin Rosenberg
2009-04-22 16:33 ` Shawn O. Pearce
2009-04-22 23:02 ` Shawn O. Pearce
2009-04-21 2:10 ` Shawn O. Pearce [this message]
2009-04-21 1:39 ` [JGIT PATCH 08/10] Scan for new packs if GIT_DIR/objects/pack has been modified Shawn O. Pearce
2009-04-21 2:08 ` [JGIT PATCH v2 " Shawn O. Pearce
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20090421021035.GB23604@spearce.org \
--to=spearce@spearce$(echo .)org \
--cc=git@vger$(echo .)kernel.org \
--cc=robin.rosenberg@dewire$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox