[PATCH 0/2] [RFC] scalar: prepare documentation for future work

["Victoria Dye via GitGitGadget" <gitgitgadget@gmail.com>]

What's this series?
===================

The questions of "what is Scalar?" and "why is Scalar part of Git?" have
been discussed at length already [1], but in preparation for the next few
series I plan to submit "finishing" the initial Scalar implementation
effort, I'd like to start by reframing its purpose.

Beginning with the "Philosophy of Scalar" [2], much of the existing
documentation refers to Scalar as "opinionated" - not unlike git itself.
However, with scalar now part of Git, continuing to describe it as
"opinionated" could unintentionally portray it as a divergence from "the
rest of Git" and its development community.

Rather than think of Scalar as an opinionated take on Git, I prefer to think
of it as a manifestation of Git's own "opinions" for a specific use case,
like many of the built-in commands. Over the years, developers have
contributed tons of features - often gated behind config settings - that
address performance issues they've encountered working in large
repositories: untracked cache, sparse index, merge-ort, etc. Some of these
features (like merge-ort) provide near-universal benefit to users and become
part of Git's default configuration. More often, though, these features are
large and difficult to prove are stable, or require some advanced manual
configuration to use properly, or simply make the "average" Git user's
experience worse (e.g., improving performance in in large repos, but
degrading it in smaller repos).

We've accumulated so many of these features that it's become difficult for
users to even find all of them, let alone use them. Git "knows" these
features exist, but also "knows" not to enable them by default because
they're either too experimental or not helpful to most users. Scalar is a
way for Git to take all of these features, bundle them together, and tell a
user, "Hey, do you need to make your large repo faster? Use this!" without
touching the default user experience.

All that to say, Scalar is essentially a collection of porcelain commands
[3] that fill a substantial UX niche in Git. Now, let's get into Scalar's
"future state".


A plan for Scalar
=================

Given the slightly tweaked "philosophy" above, my ultimate goal for Scalar
is to have it contain only what is too experimental or too large
repo-focused to be a default option or behavior in Git. Over time, some
features may be moved out of Scalar and into Git defaults as they are proven
stable and beneficial to the vast majority of users [4].

So what do we need to get there?

At a high level, the remaining work to "finalize" Scalar (past this RFC) can
be broken into three parts:

 1. Complete a few more features and subcommands of Scalar (integrate with
    the built-in FSMonitor & implement scalar help).
 2. Move stable, general-purpose parts of 'scalar.c' into other Git
    builtins/libraries (mainly scalar diagnose, either as part of git
    bug-report [5] or a new git built-in).
 3. Move Scalar out of contrib/ and into the "top-level" of Git. Includes
    expanded testing, especially performance testing.

The first makes scalar "feature complete" enough to be valuable to large
repo users (per my entirely subjective assessment, at least). The second
brings it in line with the goal of making Scalar only contain what can't
exist as a default feature of Git. Once those are finished, I think Scalar
will be out of its "work-in-progress" phase and ready to use as a built-in
component of Git (accompanied by sufficient testing, of course).


What's Scalar's future?
=======================

With the completion of the tasks listed in the previous section, there's
nothing left from the original Scalar CLI (in the Microsoft fork of Git) to
upstream. I don't consider that the "end" of supporting Scalar because, for
it to remain an effective tool, it'll need to stay up-to-date with the
latest performance features introduced to Git.

For example, one possible future extension might be to have Scalar enable
the sparse index by default, especially when more built-ins are updated with
sparse index compatibility. I'm interested in hearing what other features
might fit well there!


What I'm looking for
====================

The two patches in this series revise existing documentation to match the
description above. Please let me know if 1) those revisions match your
understanding of the above, 2) they convey the intent clearly, and (most
importantly) 3) they reflect a reasonable direction to take Scalar. And, of
course, if there's something I missed, I welcome any and all feedback &
ideas!

Thanks,

 * Victoria

[1]
https://lore.kernel.org/git/b67bbef4-e4c3-b6a7-1c7f-7d405902ef8b@gmail.com/

[2]
https://github.com/microsoft/git/blob/173a9b783a414/contrib/scalar/docs/philosophy.md

[3] If you're wondering "why not call it git-scalar then?", my short answer
is 1) backward compatibility with the original "scalar" CLI, and 2) its
scope is larger and more persistent than a single command; Scalar works
across multiple repos, and each Scalar subcommand has scope and
functionality on par with a built-in command. I'm happy to talk through this
more if people are still unsure, though!

[4] That being said, Scalar's existence should not make Git more risk averse
in accepting new general-purpose features.

[5] https://lore.kernel.org/git/220517.86y1z05gja.gmgdl@evledraar.gmail.com/

Victoria Dye (2):
  scalar: reword command documentation to clarify purpose
  scalar: convert README.md into a technical design doc

 contrib/scalar/README.md            |  82 ------------------
 contrib/scalar/scalar.txt           |   9 +-
 contrib/scalar/technical/scalar.txt | 127 ++++++++++++++++++++++++++++
 3 files changed, 131 insertions(+), 87 deletions(-)
 delete mode 100644 contrib/scalar/README.md
 create mode 100644 contrib/scalar/technical/scalar.txt


base-commit: e4a4b31577c7419497ac30cebe30d755b97752c5
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1275%2Fvdye%2Fscalar%2Fdoc-update-rfc-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1275/vdye/scalar/doc-update-rfc-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1275
-- 
gitgitgadget