Subject: Re: Need some help on patching buildin-files // was: Looking for feedback and help with a git-mirror for local usage

From: Bernd Naumann <[email-masked]>
Date: Fri, 12 Jun 2015 12:52:44 +0200

Hello again,

After digging the code I may have got a clue where to start but I
would  still appreciate some help from a developer, cause I have never
learned to write C. (Some basics at school which happened over a
decade ago.)

Currently I have questions on:

* How to patch clone: would cmd_clone() a good place? Or are there
other calls which might be better. I think about to insert the check
if a mirror will be setup or just updated, right after dest_exists.

* Is it correct that a new config key just get specified via a config
file or by cmd_init_db()? So later, a check on that value is enough?
Would be the section 'user' a good place for this key or is it
something that would get a own/new section?

* Have I missed a relevant file?

git/git.c
git/builtin/clone.c
git/builtin/fetch.c
git/builtin/push.c
git/buildin/remote.c
along with the translation and Documentation, of course.

If you have some comments on that, please share these with me, and if
you are interested in helping me to got this implemented, I would
appreciate that :)

Sincere regards,
Bernd

On 06/11/2015 10:44 PM, Bernd Naumann wrote:
> Hello,
> 
> I have came up with an idea # Yep I know, exactly that kind of 
> e-mail everyone wants to read ;) and I'm working currently on a 
> shell-prototype to face the following situation and problem and 
> need some feedback/advise:
> 
> 
> I often build in example 'openwrt' with various build-scripts which
> depends heavily on a fresh or clean environment and they omit many
> sources via `git clone`, which results sometimes in over 100 MB of
> traffic just for one build. /* Later needed .tar.gz source archives
> are stored in a symlinked download directory which is supported by
> 'openwrt/.config' since a few months... to reduce network traffic.
> */
> 
> My connection to the internet is not the fastest in world and 
> sometimes unstable, so I wanted to have some kind of local bare 
> repository mirror, which is possible with `git clone --mirror`.
> 
> From these repositories I can later clone from, by calling `git 
> clone --reference /path/to.git <url>`, but I do not wish to edit 
> all the build-scripts and Makefiles.
> 
> 
> So I wrote a git wrapper script (`$HOME/bin/git`), which checks if
>  `git` was called with 'clone', and if so, then it will first 
> clones the repository as a mirror and then clones from that local 
> mirror. If the mirror already exists, then it will only be updated 
> (`git remote update`). This works for now.
> 
> /* To be able to have multiple identical named repositories, the 
> script builds paths like:
> 
> ~/var/cache/gitmirror $ find . -name "*.git"
> 
> ./github.com/openwrt-management/packages.git 
> ./github.com/openwrt/packages.git 
> ./github.com/openwrt-routing/packages.git ./nbd.name/packages.git 
> ./git.openwrt.org/packages.git ./git.openwrt.org/openwrt.git
> 
> It strips the schema from the url and replaces ":" with "/" in
> case a port is specified or a svn link is provided. The remaining
> should be a valid linux file and directory structure, if I guess 
> correctly!? */
> 
> Ok, so far, so good, but the implementation of the current 
> shell-prototype looks way too hacky [0] and I have found some edge
>  cases on which my script will fail: The script depends on the
> fact that the last, or at least the second last argument is a
> valid git-url, but the following is a valid call, too :
> 
> `git --no-pager \ clone [email-masked]:openwrt/packages.git 
> openwrt-packages --depth 1`
> 
> But this is not valid:
> 
> `git clone https://github.com/openwrt/packages.git --reference 
> packages.git packages-2` or `git clone --verbose 
> https://github.com/openwrt/packages.git packages-2 --reference 
> packages.git`
> 
> 
> I found out that git-clone actually also can only make a guess
> what is the url and what not.
> 
> 
> 
> However, now I'm looking for a way to write something like a 
> submodul for git which will check for a *new* git-config value like
> "user.mirror" (or something...) which points to a directory, and
> will be used to clone from, and in case of 'fetch', 'pull' or 
> 'remote update' update the mirror first, and then the update of
> the current working directory is gotten from that mirror. (And in
> case of 'push' the mirror would be updated from the working dir,
> of course.)
> 
> 
> I would like to hear some toughs on that, and how I could start to
>  build this submodul, or if someone more talented, then I am, is 
> willed to spent some time on that. If requested/wished I could
> send a link to the shell-prototype.
> 
> 
> [0] For a reason I have to do ugly things like `$( eval exec 
> /usr/bin/git clone --mirror $REPO_URL ) 2>&1 >/dev/null` cause 
> otherwise in case of just `eval exec` the script stops after 
> execution, and without `eval exec` arguments with spaces will 
> interpreted as seperated arguments, which is no good, because of 
> failing .
> 
> 
> Thanks for your time! Yours faithfully, Bernd -- To unsubscribe 
> from this list: send the line "unsubscribe git" in the body of a 
> message to [email-masked] More majordomo info at 
> http://vger.kernel.org/majordomo-info.html
> 

- -- 
Bernd Naumann <[email-masked]>

PGP:   0xA150A04F via pool.sks-keyservers.net
XMPP:  [email-masked]