=head1 NAME

public-inbox-clone - "git clone --mirror" wrapper

=head1 SYNOPSIS

public-inbox-clone [OPTIONS] INBOX_URL [INBOX_DIR]

public-inbox-clone [OPTIONS] ROOT_URL [DESTINATION] # public-inbox 2.0+

=head1 DESCRIPTION

public-inbox-clone is a wrapper around C<git clone --mirror> for
making the initial clone of a remote HTTP(S) public-inbox.  It
allows cloning multi-epoch v2 inboxes with a single command and
zero configuration.

In public-inbox 2.0+, public-inbox-clone can create and maintain
a mirror of multiple inboxes or code repositories using manifest.js.gz
files like L<grok-pull(1)> from grokmirror.  L<public-inbox-fetch(1)> is
NOT required when using this mode.

It does not run L<public-inbox-init(1)> nor
L<public-inbox-index(1)>.  Those commands must be run separately
if serving/searching the mirror is required.  As-is,
public-inbox-clone is suitable for creating a git-only backup
without Xapian and SQLite indices.

When cloning a single inbox, public-inbox-clone creates a Makefile
with handy targets to update the inbox once indexed.
This Makefile may be edited by the user; it will
not be rewritten by L<public-inbox-fetch(1)> unless it is removed
completely.

public-inbox-clone does not use nor require any extra
configuration files (not even C<~/.public-inbox/config>),
but it can download snippets suitable for adding to any
L<public-inbox-config(5)> file.

L<public-inbox-fetch(1)> may be used to keep a single C<INBOX_DIR>
up-to-date.

For v2 inboxes, it will create a C<$INBOX_DIR/manifest.js.gz>
file to speed up subsequent L<public-inbox-fetch(1)>.

=head1 OPTIONS

=over

=item --epoch=RANGE

Restrict clones of L<public-inbox-v2-format(5)> inboxes to the
given range of epochs.  The range may be a single non-negative
integer or a (possibly open-ended) C<LOW..HIGH> range of
non-negative integers.  C<~> may be prefixed to either (or both)
integer values to represent the offset from the maximum possible
value.

For example, C<--epoch=~0> alone clones only the latest epoch,
C<--epoch=~2..> clones the three latest epochs.

Default: C<0..~0> or C<0..> or C<..~0>
(all epochs, all three examples are equivalent)

=item -I PATTERN

=item --include=PATTERN

When cloning a top-level with multiple inboxes via manifest,
only clone inboxes and repositories matching a given wildcard pattern
(using C<*?> and C<[]> is supported).

This is a new option in public-inbox 2.0+

=item --exclude=PATTERN

When cloning a top-level with multiple inboxes via manifest,
ignore inboxes and repositories matching the given wildcard pattern.
Supports the same wildcards as L</--include>

This is a new option in public-inbox 2.0+

=item --inbox-config=always|v2|v1|never

Whether or not to retrieve the C<$INBOX/_/text/config/raw> HTTP(S)
endpoint when cloning.

Since we can't deduce v1 inboxes from code repositories, setting this
to C<v2> or C<never> can allow faster clones of code repositories if
no v1 inboxes are present.

Default: C<always>

This is a new option in public-inbox 2.0+

=item --inbox-version=NUM

Force a remote public-inbox version (must be C<1> or C<2>).
This is auto-detected by default, and this option exists mainly
for testing.

This is a new option in public-inbox 2.0+

=item --objstore=DIR

Enables space savings when the remote C<manifest.js.gz>
includes C<forkgroup> entries as generated by grokmirror 2.x.

If C<DIR> does not start with C</>, C<./>, or C<../>, it is treated
as relative to the C<DESTINATION> directory.  If only C<--objstore=>
is specified where C<DIR> is an empty string (C<"">), then C<objstore>
(C<$DESTINATION/objstore>) is the implied value of C<DIR>.

This is a new option in public-inbox 2.0+

=item --manifest=FILE

When incrementally updating an existing mirror, load the given
manifest (typically C<manifest.js.gz>) to speed up updates.

By default, public-inbox writes the retrieved manifest to
C<$DESTINATION/manifest.js.gz>, this directive also
changes the destination to the specified C<FILE>

If C<FILE> does not start with C</>, C<./>, or C<../>, it is treated
as relative to the C<DESTINATION> directory.  If only C<--manifest=>
is specified where C<FILE> is an empty string (C<"">), then C<manifest.js.gz>
(C<$DESTINATION/manifest.js.gz>) is the implied value of C<FILE>.

When updating manifests with many forks using the same objstore,
git 2.41+ is highly recommended for performance as we automatically
use the C<fetch.hideRefs> feature to speed up negotiation.

C<--manifest=> is a new option in public-inbox 2.0+

=item --remote-manifest=URL|RELATIVE_PATH

Use an alternate location for the remote manifest.js.gz file.
This may be specified as a full absolute URL (e.g
C<--remote-manifest=https://80x24.org/lore/pub/manifest.js.gz>),
or a pathname relative to the ROOT_URL (e.g
C<--remote-manifest=pub/manifest.js.gz> when ROOT_URL is
C<https://80x24.org/lore/>

By default, C<ROOT_URL/manifest.js.gz> is used.

This is a new option in public-inbox 2.0+

=item --project-list=FILE

When cloning code repos from a manifest, generate a cgit-compatible
project list.

If C<FILE> does not start with C</>, C<./>, or C<../>, it is treated
as relative to the C<DESTINATION> directory.  If only C<--project-list=>
is specified where C<FILE> is an empty string (C<"">), then C<projects.list>
(C<$DESTINATION/projects.list>) is the implied value of C<FILE>.

This is a new option in public-inbox 2.0+

=item --post-update-hook=COMMAND

Hooks to run after a repository is cloned or updated, C<COMMAND> will
have the bare git repository destination given as its first and only
argument.

For v2 inboxes, this operates on a per-epoch basis.

May be specified multiple times to run multiple commands in the
order specified on the command-line.

This is a new option in public-inbox 2.0+

=item -p

=item --prune

Pass the C<--prune> and C<--prune-tags> flags to L<git-fetch(1)>
calls on incremental clones.

This is a new option in public-inbox 2.0+

=item --purge

Deletes entire repos which no longer exist in the remote manifest,
or are filtered out by C<--include=> or C<--exclude=>.

This is only useful when using C<--manifest>

This is a new option in public-inbox 2.0+

=item --exit-code

Exit with C<127> if no updates are done when relying on a manifest.
Updates include fingerprint mismatches in the manifest, new symlinks,
new repositories, and removed repositories from the L<--project-list>

This is a new option in public-inbox 2.0+

=item -k

=item --keep-going

Continue as much as possible after an error.

This is a new option in public-inbox 2.0+

=item -n

=item --dry-run

Show what would be done, without making any changes.

This is a new option in public-inbox 2.0+

=item -q

=item --quiet

Quiets down progress messages, also passed to L<git-fetch(1)>.

=item -v

=item --verbose

Increases verbosity, also passed to L<git-fetch(1)>.

=item --torsocks=auto|no|yes

=item --no-torsocks

Whether to wrap L<git(1)> and L<curl(1)> commands with L<torsocks(1)>.

Default: C<auto>

=item -j JOBS

=item --jobs=JOBS

The number of parallel processes to spawn at once for various network
operations using L<git(1)> and/or L<curl(1)>.

=back

=head1 EXAMPLES

=for comment
Sticking to smaller projects in examples to minimize load on servers

=over

=item To mirror the most recent epochs of dwarves and LTTng inboxes:

  public-inbox-clone --epoch=~0 \
	--include='*lttng*' --include='*dwarves' \
	https://80x24.org/lore/ /path/to/inbox-mirror

C<https://lore.kernel.org/> may be used instead of C<https://80x24.org/lore/>

=item To mirror all code repos of the sparse project:

  public-inbox-clone --objstore= --project-list= --prune \
	--include='*sparse*' --inbox-config=never \
	--remote-manifest=https://80x24.org/lore/pub/manifest.js.gz \
	https://80x24.org/lore/ /path/to/code-mirror

C<https://git.kernel.org/> may be used instead of C<https://80x24.org/lore/>
and the C<--remote-manifest> option can be omitted.

=back

=head1 CONTACT

Feedback welcome via plain-text mail to L<mailto:meta@public-inbox.org>

The mail archives are hosted at L<https://public-inbox.org/meta/> and
L<http://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/meta/>

=head1 COPYRIGHT

Copyright all contributors L<mailto:meta@public-inbox.org>

License: AGPL-3.0+ L<https://www.gnu.org/licenses/agpl-3.0.txt>

=head1 SEE ALSO

L<public-inbox-fetch(1)>, L<public-inbox-init(1)>, L<public-inbox-index(1)>
