Subject: Re: [PATCH 00/32] Split index mode for very large indexes

From: Junio C Hamano <[email-masked]>
Date: Mon, 28 Apr 2014 15:23:53 -0700

Nguyễn Thái Ngọc Duy  <[email-masked]> writes:

> The read penalty is not addressed here, so I still pay 14MB hashing
> cost.

Hmm, yeah, the cost for verify_hdr() would still matter, and
presumably you would be hashing the additional 200kB to validate the
smaller "changes since the base" file to give users the same level
of protection against corruption.

> Doing this in other implementations should be easy (at least the
> reading part) and with small code change. The whole index format is
> retained. All you need is to read a new extension that contains two
> ewah-bitmaps and apply the changes to create the final index.

Why bitmaps, though?  Naïvely I would have expected you to read from
two sorted streams and have the transaction log override the base.

Intrigued to find it out...