* [PATCH 0/5] patchwork-bot: Use worktrees for repos
@ 2021-12-01 0:11 Kees Cook
2021-12-01 0:11 ` [PATCH 1/5] patchwork-bot: Note which actions are part of "housekeeping" Kees Cook
` (5 more replies)
0 siblings, 6 replies; 7+ messages in thread
From: Kees Cook @ 2021-12-01 0:11 UTC (permalink / raw)
To: Konstantin Ryabitsev; +Cc: Kees Cook, tools
Hi,
To use patchwork-bot locally, I wanted to use my existing worktrees for
various repos. This makes a few small improvements and then adds logic
to deal with specifying branchs and operating in worktrees.
Thanks!
-Kees
Kees Cook (5):
patchwork-bot: Note which actions are part of "housekeeping"
patchwork-bot: Create CACHEDIR if it is missing
patchwork-bot: Allow limiting heads to single branch
patchwork-bot: Handle repo being a worktree
patchwork-bot: Allow initial database to skip recent commits
git-patchwork-bot.example.yaml | 3 ++
git-patchwork-bot.py | 50 ++++++++++++++++++++++++++++------
2 files changed, 45 insertions(+), 8 deletions(-)
--
2.30.2
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/5] patchwork-bot: Note which actions are part of "housekeeping"
2021-12-01 0:11 [PATCH 0/5] patchwork-bot: Use worktrees for repos Kees Cook
@ 2021-12-01 0:11 ` Kees Cook
2021-12-01 0:11 ` [PATCH 2/5] patchwork-bot: Create CACHEDIR if it is missing Kees Cook
` (4 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Kees Cook @ 2021-12-01 0:11 UTC (permalink / raw)
To: Konstantin Ryabitsev; +Cc: Kees Cook, tools
Expand the --help text slightly to remind users what things the
"housekeeping" option does.
Signed-off-by: Kees Cook <keescook@chromium.org>
---
git-patchwork-bot.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/git-patchwork-bot.py b/git-patchwork-bot.py
index f87af6ef41d6..d1eac6c0c0df 100755
--- a/git-patchwork-bot.py
+++ b/git-patchwork-bot.py
@@ -1325,7 +1325,7 @@ if __name__ == '__main__':
parser.add_argument('-v', '--verbose', action='store_true', default=False,
help='Be more verbose in logging output')
parser.add_argument('-k', '--housekeeping', action='store_true', default=False,
- help='Perform a housekeeping run')
+ help='Perform a housekeeping run (supersede, archive)')
parser.add_argument('--cachedir', default=None,
help='Cache directory to use instead of ~/.cache/git-patchwork-bot')
parser.add_argument('--domain', default=None,
--
2.30.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 2/5] patchwork-bot: Create CACHEDIR if it is missing
2021-12-01 0:11 [PATCH 0/5] patchwork-bot: Use worktrees for repos Kees Cook
2021-12-01 0:11 ` [PATCH 1/5] patchwork-bot: Note which actions are part of "housekeeping" Kees Cook
@ 2021-12-01 0:11 ` Kees Cook
2021-12-01 0:11 ` [PATCH 3/5] patchwork-bot: Allow limiting heads to single branch Kees Cook
` (3 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Kees Cook @ 2021-12-01 0:11 UTC (permalink / raw)
To: Konstantin Ryabitsev; +Cc: Kees Cook, tools
Create the CACHEDIR if it is missing (e.g. the first time
git-patchwork-bot is executed).
Signed-off-by: Kees Cook <keescook@chromium.org>
---
git-patchwork-bot.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/git-patchwork-bot.py b/git-patchwork-bot.py
index d1eac6c0c0df..dc10fd0effa9 100755
--- a/git-patchwork-bot.py
+++ b/git-patchwork-bot.py
@@ -1375,6 +1375,9 @@ if __name__ == '__main__':
cfgyaml = fh.read()
CONFIG = ruamel.yaml.safe_load(cfgyaml)
+ if not os.path.isdir(CACHEDIR):
+ os.makedirs(CACHEDIR, exist_ok=True)
+
if cmdargs.housekeeping:
for _pserver, _sconfig in CONFIG['patchworks'].items():
for _pname in _sconfig['projects']:
--
2.30.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 3/5] patchwork-bot: Allow limiting heads to single branch
2021-12-01 0:11 [PATCH 0/5] patchwork-bot: Use worktrees for repos Kees Cook
2021-12-01 0:11 ` [PATCH 1/5] patchwork-bot: Note which actions are part of "housekeeping" Kees Cook
2021-12-01 0:11 ` [PATCH 2/5] patchwork-bot: Create CACHEDIR if it is missing Kees Cook
@ 2021-12-01 0:11 ` Kees Cook
2021-12-01 0:11 ` [PATCH 4/5] patchwork-bot: Handle repo being a worktree Kees Cook
` (2 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Kees Cook @ 2021-12-01 0:11 UTC (permalink / raw)
To: Konstantin Ryabitsev; +Cc: Kees Cook, tools
To sanely use a single git repository with multiple worktrees, it must
be possible to limit the commits by branch (otherwise all commits are
visible for all worktrees). Add an optional "branch" argument for this
limit.
Signed-off-by: Kees Cook <keescook@chromium.org>
---
git-patchwork-bot.example.yaml | 2 ++
git-patchwork-bot.py | 6 +++---
2 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/git-patchwork-bot.example.yaml b/git-patchwork-bot.example.yaml
index d6140ed68e53..c3f6c1ea3133 100644
--- a/git-patchwork-bot.example.yaml
+++ b/git-patchwork-bot.example.yaml
@@ -52,6 +52,8 @@ repos:
'pub/scm/linux/kernel/git/khilman/linux-amlogic.git':
# For the notification emails
treename: 'khilman/linux-amlogic.git'
+ # Optional: limit checking to a single branch (instead of --heads)
+ #branch: 'origin/master'
# For the notification emails, must include %s
commitlink: 'https://git.kernel.org/khilman/linux-amlogic/c/%.12s'
# Only notify for commits where committer email exactly matches one
diff --git a/git-patchwork-bot.py b/git-patchwork-bot.py
index dc10fd0effa9..fc4d9e384122 100755
--- a/git-patchwork-bot.py
+++ b/git-patchwork-bot.py
@@ -409,9 +409,9 @@ def git_run_command(gitdir, args, stdin=None):
return output
-def git_get_repo_heads(gitdir):
+def git_get_repo_heads(gitdir, branch):
refs = list()
- lines = git_get_command_lines(gitdir, ['show-ref', '--heads'])
+ lines = git_get_command_lines(gitdir, ['show-ref', branch])
if lines is not None:
for line in lines:
(commit_id, refname) = line.split()
@@ -1049,7 +1049,7 @@ def housekeeping(pname):
def pwrun(repo, rsettings):
- git_heads = git_get_repo_heads(repo)
+ git_heads = git_get_repo_heads(repo, branch=rsettings.get('branch', '--heads'))
if not git_heads:
logger.info('Could not get the latest ref in %s', repo)
sys.exit(1)
--
2.30.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 4/5] patchwork-bot: Handle repo being a worktree
2021-12-01 0:11 [PATCH 0/5] patchwork-bot: Use worktrees for repos Kees Cook
` (2 preceding siblings ...)
2021-12-01 0:11 ` [PATCH 3/5] patchwork-bot: Allow limiting heads to single branch Kees Cook
@ 2021-12-01 0:11 ` Kees Cook
2021-12-01 0:11 ` [PATCH 5/5] patchwork-bot: Allow initial database to skip recent commits Kees Cook
2021-12-13 22:30 ` [PATCH 0/5] patchwork-bot: Use worktrees for repos Konstantin Ryabitsev
5 siblings, 0 replies; 7+ messages in thread
From: Kees Cook @ 2021-12-01 0:11 UTC (permalink / raw)
To: Konstantin Ryabitsev; +Cc: Kees Cook, tools
Locate the real git location when operating on a worktree, allowing
multiple pw.db files to be stored in a single top-level git repo.
Signed-off-by: Kees Cook <keescook@chromium.org>
---
git-patchwork-bot.example.yaml | 1 +
git-patchwork-bot.py | 27 +++++++++++++++++++++++++--
2 files changed, 26 insertions(+), 2 deletions(-)
diff --git a/git-patchwork-bot.example.yaml b/git-patchwork-bot.example.yaml
index c3f6c1ea3133..94ab334f5043 100644
--- a/git-patchwork-bot.example.yaml
+++ b/git-patchwork-bot.example.yaml
@@ -53,6 +53,7 @@ repos:
# For the notification emails
treename: 'khilman/linux-amlogic.git'
# Optional: limit checking to a single branch (instead of --heads)
+ # This becomes a required setting if the repo path is a worktree.
#branch: 'origin/master'
# For the notification emails, must include %s
commitlink: 'https://git.kernel.org/khilman/linux-amlogic/c/%.12s'
diff --git a/git-patchwork-bot.py b/git-patchwork-bot.py
index fc4d9e384122..9be945d92a8a 100755
--- a/git-patchwork-bot.py
+++ b/git-patchwork-bot.py
@@ -1054,8 +1054,28 @@ def pwrun(repo, rsettings):
logger.info('Could not get the latest ref in %s', repo)
sys.exit(1)
+ dbpath = repo
+ # If we're aimed at a worktree, move up from the ".git" file to
+ # the worktree directory.
+ if not os.path.isdir(dbpath):
+ gitdir = open(dbpath).readline().strip()
+ if not gitdir.startswith('gitdir: '):
+ logger.info('Could not find git tree in %s', dbpath)
+ sys.exit(1)
+ gitdir = gitdir.split(' ', 1)[1]
+ gitdir, worktree = os.path.split(gitdir)
+ gitdir, category = os.path.split(gitdir)
+ if category != "worktrees":
+ logger.info('Could not find git worktree in %s', dbpath)
+ sys.exit(1)
+ # To store multiple pw.db files in a single .git directory,
+ # add a suffix based on the repo treename.
+ treename = rsettings.get('treename').replace('/', '_')
+ dbpath = os.path.join(gitdir, f'pw-{treename}.db')
+ else:
+ dbpath = os.path.join(dbpath, 'pw.db')
+
# Do we have a pw.db there yet?
- dbpath = os.path.join(repo, 'pw.db')
db_exists = os.path.isfile(dbpath)
dbconn = sqlite3.connect(dbpath, sqlite3.PARSE_DECLTYPES | sqlite3.PARSE_COLNAMES)
c = dbconn.cursor()
@@ -1295,10 +1315,13 @@ def check_repos():
for repo in CONFIG['repos']:
fullpath = os.path.join(cmdargs.reposdir.rstrip('/'), repo.lstrip('/'))
- if not os.path.isdir(fullpath):
+ if not os.path.exists(fullpath):
logger.info('Repository not found: %s', repo)
continue
settings = CONFIG['repos'][repo]
+ if not os.path.isdir(fullpath) and settings.get('branch', None) == None:
+ logger.info('Worktree must specify "branch" setting: %s', repo)
+ continue
logger.info('Processing: %s', repo)
pwrun(fullpath, settings)
--
2.30.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 5/5] patchwork-bot: Allow initial database to skip recent commits
2021-12-01 0:11 [PATCH 0/5] patchwork-bot: Use worktrees for repos Kees Cook
` (3 preceding siblings ...)
2021-12-01 0:11 ` [PATCH 4/5] patchwork-bot: Handle repo being a worktree Kees Cook
@ 2021-12-01 0:11 ` Kees Cook
2021-12-13 22:30 ` [PATCH 0/5] patchwork-bot: Use worktrees for repos Konstantin Ryabitsev
5 siblings, 0 replies; 7+ messages in thread
From: Kees Cook @ 2021-12-01 0:11 UTC (permalink / raw)
To: Konstantin Ryabitsev; +Cc: Kees Cook, tools
To test patchwork-bot, it is helpful to create the initial database
without a certain number of the latest commits, so that on the next
execution, the "missing" commits will appear new, and will be used for
processing state changes. Add the --ancestors argument (defaulting to
0, the existing behavior), to provide this ability.
Signed-off-by: Kees Cook <keescook@chromium.org>
---
git-patchwork-bot.py | 14 +++++++++++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/git-patchwork-bot.py b/git-patchwork-bot.py
index 9be945d92a8a..2ebe5b4e88e4 100755
--- a/git-patchwork-bot.py
+++ b/git-patchwork-bot.py
@@ -409,13 +409,17 @@ def git_run_command(gitdir, args, stdin=None):
return output
-def git_get_repo_heads(gitdir, branch):
+def git_get_repo_heads(gitdir, branch, ancestry=None):
refs = list()
lines = git_get_command_lines(gitdir, ['show-ref', branch])
+ if ancestry == None:
+ ancestry = ''
+ else:
+ ancestry = f'~{ancestry}'
if lines is not None:
for line in lines:
(commit_id, refname) = line.split()
- refs.append((refname, commit_id))
+ refs.append((refname, commit_id + ancestry))
return refs
@@ -1082,7 +1086,9 @@ def pwrun(repo, rsettings):
if not db_exists:
db_init_pw_sqlite_db(c)
- db_save_repo_heads(c, git_heads)
+ initial_git_heads = git_get_repo_heads(repo, branch=rsettings.get('branch', '--heads'),
+ ancestry=cmdargs.ancestors)
+ db_save_repo_heads(c, initial_git_heads)
# Exit early
dbconn.commit()
return
@@ -1353,6 +1359,8 @@ if __name__ == '__main__':
help='Cache directory to use instead of ~/.cache/git-patchwork-bot')
parser.add_argument('--domain', default=None,
help='Domain to use when creating message-ids')
+ parser.add_argument('--ancestors', default=None,
+ help='During initial database creation, consider this many ancestor commits as fresh')
cmdargs = parser.parse_args()
--
2.30.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 0/5] patchwork-bot: Use worktrees for repos
2021-12-01 0:11 [PATCH 0/5] patchwork-bot: Use worktrees for repos Kees Cook
` (4 preceding siblings ...)
2021-12-01 0:11 ` [PATCH 5/5] patchwork-bot: Allow initial database to skip recent commits Kees Cook
@ 2021-12-13 22:30 ` Konstantin Ryabitsev
5 siblings, 0 replies; 7+ messages in thread
From: Konstantin Ryabitsev @ 2021-12-13 22:30 UTC (permalink / raw)
To: Kees Cook; +Cc: tools
On Tue, 30 Nov 2021 16:11:21 -0800, Kees Cook wrote:
> To use patchwork-bot locally, I wanted to use my existing worktrees for
> various repos. This makes a few small improvements and then adds logic
> to deal with specifying branchs and operating in worktrees.
>
> Thanks!
>
> -Kees
>
> [...]
Applied, thanks!
[1/5] patchwork-bot: Note which actions are part of "housekeeping"
commit: bba5718890cbc3704528c028e104dde197d4c49c
[2/5] patchwork-bot: Create CACHEDIR if it is missing
commit: e2dc648b93b77e45fde736ecbff8d0080fca41ff
[3/5] patchwork-bot: Allow limiting heads to single branch
commit: 36971ee3ad62a56f40c3663a82b7475151df2a77
[4/5] patchwork-bot: Handle repo being a worktree
commit: 460990bdca13805f7e9b5d6c8037437f655bcfac
[5/5] patchwork-bot: Allow initial database to skip recent commits
commit: 6fc2e91a876b0a2771eedb0fe6106d7f47eb8aab
Best regards,
--
Konstantin Ryabitsev <konstantin@linuxfoundation.org>
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-12-13 22:30 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-12-01 0:11 [PATCH 0/5] patchwork-bot: Use worktrees for repos Kees Cook
2021-12-01 0:11 ` [PATCH 1/5] patchwork-bot: Note which actions are part of "housekeeping" Kees Cook
2021-12-01 0:11 ` [PATCH 2/5] patchwork-bot: Create CACHEDIR if it is missing Kees Cook
2021-12-01 0:11 ` [PATCH 3/5] patchwork-bot: Allow limiting heads to single branch Kees Cook
2021-12-01 0:11 ` [PATCH 4/5] patchwork-bot: Handle repo being a worktree Kees Cook
2021-12-01 0:11 ` [PATCH 5/5] patchwork-bot: Allow initial database to skip recent commits Kees Cook
2021-12-13 22:30 ` [PATCH 0/5] patchwork-bot: Use worktrees for repos Konstantin Ryabitsev
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.