Changelog

Note

Below is the rendered version of the project’s CHANGELOG.md file, found at the root of the code repository.

The format is based on Keep a Changelog. The project uses semantic versioning (see semver).

v0.20.0 - 2025-02-02

Added

• New “deleter” PatchDriver, deletes file(s) specified if found.

Fixed

• Uncaught exceptions of the CLI are no longer shamefully hidden.

Removed

• The under-cooked “precommit” PatchDriver is removed, useless and left broken for a long time. If you need it, use a ShellDriver with pre-commit autoupdate command instead

v0.19.0 - 2025-01-22

Added

• Add GlobFileEditor.before_run() to run arbitrary code before processing the first file

• New function replace_many in patchdrivers.brick, to replace many patterns in a string at once. Perfect for the kind of file sed replacement

• PatchDriver Pydantic base class now allows extra fields (Pydantic Extra fields)

Fixed

• ShellDriver crash caused by incorrect construction of PatchOutcome object.

v0.18.0 - 2023-11-21

Added

• New SingleFileEditor, derived from PatchDriver, for editing single files.

  • Parameter target_file, will be fed its text content to process_file().

  • Use via from mass_driver.drivers.bricks import SingleFileEditor.

• New GlobFileEditor, similarly for editing multiple files from glob pattern.

  • Use via from mass_driver.drivers.bricks import GlobFileEditor.

v0.17.1 - 2023-11-13

Added

• Update pyGithub to 2.1.1, now throttles Github API to avoid ratelimits

Fixed

• Catch and log import errors during activity loading, previously silent crashes

v0.17.0 - 2023-11-12

Added

• Completing an activity (source, migration, forge) now summarizes results

  • Breakdown of repo count per outcome type, then sorted list of repos per type

• FASTER: Optional, experimental multi-threaded per-repo processsing! Handling of clone, scan, patching is done as individual thread per repo, with N=8 pooled threads.

  • Early data shows a x6 improvement in performance, as cloning one repo doesn’t block others anymore.

  • Enable via new experimental flag run --parallel, defaulting to False

Changed

• BREAKING: RenamedRepo to SourcedRepo in mass_driver.models.repository

  • This exposes better the idea of “a Repo as it was Source-d”, in contrast to ClonedRepo “a Repo after it was cloned”.

  • Also avoids clashes with git.Repo object from gitpython dependency.

• Replaced all print() calls to logging module

• Loggers used are mostly nested:

  • from root (default)

  • to run (or other file-activity-based)

  • to run.repo.<repo-id> for logs for a specific repo’s processing

  • to subloggers like run.repo.<repo-id>.driver.<driver-plugin-name>

• PatchDriver now has a logger obj for such customized logging: Repo-processing sets Patchdriver.logger named run.repo.<repo-id>.driver.<driver-plugin-name>

  • ACTION: Please replace any print with self.logger.info!

v0.16.4 - 2023-11-04

Fixed

• Attempting to run a Forge activity with git_push_first=True, without migration or scan activity, no longer causes exit without processing any repos. Clone step invoked properly mean remote-clone URLs are now supported, converted to local filepaths internally.

• make docs now works again with sphinx-autodoc2: Pinned astroid dep to 2.15.8 (< 3.0.0) to avoid the regression caused by unpinned astroid.

v0.16.3 - 2023-09-11

Fixed

• ShellDriver no longer crashes due to irrelevant dataclass import

• Updated pyGithub, fixes “missing cryptography” error

• Error messages for bad config file for Sources no longer insist wrongly about “Forge config error”. Now detecting the validation error’s model properly.

• Failing to load a scanner that was selected in config now throws ImportError on the first plugin-load failure, instead of silently skipping the scanner.

Changed

• Updated to python-template v1.3.0 (from 1.1.0)

v0.16.2 - 2023-07-25

Fixed

• github-app forge no longer crashes due to type confusion on a param.

v0.16.1 - 2023-07-15

Fixed

• Formatting of the % of PR per status back to 2 digits precision not 11.

v0.16.0 - 2023-07-13

Breaking interface of Forge to facilitate new view-pr feature.

Added

• New view-pr feature for bulk reviewing the status of PRs that already exist.

Changed

• BREAKING: New Forge.get_pr_status(), required from derived classes, returning a string status, used as key to group PRs together for summary purposes.

• BREAKING: New Forge.pr_statuses property, required from derived classes, returning a list of all possible string statuses of Forge.get_pr_status(), sorted from most complete (e.g. merged) to least complete (e.g. not merged, has merge-conflicts).

Removed

• BREAKING: Removed Forge.get_pr(), which had unclear return type anyway.

v0.15.0 - 2023-07-05

Changed

Major break of interface: Rework of the cloning system, merges migration/scan codepaths, enabling use of Source-discovered information in PatchDriver.run.

• BREAKING: PatchDriver.run() passes ClonedRepo obj, not pathlib.Path.

  • Any use of repo in your PatchDriver.run() should use repo.cloned_path.

  • See ClonedRepo docs, contains information derived from Source, such as patch_data field, arbitrary source-issued information dict.

• BREAKING: tests.fixtures.massdrive() now returns 3-item-tuple, not 2.

  • Returned tuple: PatchResult, ForgeResult, ScanResult (or None)

  • Any tests using fixtures.massdrive should now set mig, forge, scan =…

  • Swap fixtures.massdrive_scan with fixtures.massdrive accepting 2 junk arg

• BREAKING: mass-driver scan CLI removed, now part of mass-driver run. Activity flow for run command is now:

  • Source discovery phase (if any, or from CLI), generating Repo list

  • Main phase, iterating over each Repo, first to clone them =ClonedRepo list

  • Inside main phase, scan (if any), generating ScanResult

  • Inside main phase, migrate (if any), generating MigrationResult

  • After main phase, interactively pause for review if requested

  • Forge activity, iterating over each repo again, creating ForgeResult

• BREAKING: models.source module renamed to models.repository.

Added

• Scan+Migration+Forge can now ALL happen in one run command:

  • Clones one repo, then scanning it, then migrating it, then next repo

  • Can thus do all of Source -> [Clone] -> Scan -> Migrate -> Forge

• New csv-filelist Source for importing repos in CSV file format

• New tests.fixture.massdrive_runlocal() func to enable source testing

Fixed

• Secret tokens for Github plugins no longer leak on config dump (--json-outfile flag), by replacing str with pydantic.SecretStr.

  • Docs updated to warn downstream devs about this risk.

• Pin pydantic to 1.*, as breaking version 2.0 was just released.

v0.14.0 - 2023-06-08

Added

• New source feature for discovering what repos to patch/scan.

  • Sources are plugins with discover() method, returning Repos by ID.

  • Alternative sources subcommand to list and detail them

  • New TOML file entry [mass-driver.source], with subkey source_name used to select which source plugin to enable.

  • Simple sources provided:

    • repo-list for in-activity-file repository list

    • repo-filelist to point to a separate file listing repos

    • template-filelist to expand a template against a file listing repos

  • github-search and github-app-search Sources for Github Repository search

  • CLI args --repo-path and --repo-filelist still available, overriding any source, so that massdriver.source is only required if lacking CLI args

• CI (pytest, pre-commit) set up via Github Actions: PR #1

v0.13.2 - 2023-06-03

Added

• New file_ownership parameter for stamper, defaulting to 0664.

Fixed

• Exit codes harmonized:

  • 0 for success

  • 1 for failures during the main function

  • 2 for argument parsing errors

• stamper driver now creates any missing parent folder to the target

• Remove test depending on git clone from Github: Faster, offline tests now

v0.13.1 - 2023-04-17

Fixed

• scan command now uses --json-outfile as expected

v0.13.0 - 2023-04-17

Added

• New scan feature for scanning repos with arbitrary python functions. See new “Scanning” docs:

  • Scanners are plugins declared under mass-driver.scanners, linking to functions like my_scanner(repo: Path) -> dict[str, Any]

  • Alternative scanners command to list out detected, available scanners

  • New TOML file entry [mass-driver.scan], with subkey scanner_names used to select which scanner plugins to enable.

  • Simple scanners root-files and dockerfile-from provided for reference

  • New fixture massdriver_scan and massdriver_scan_check for testing scanners

• New optional CLI parameter --json-outfile for run and scan, to save the activity outcome to JSON files for analysis

Changed

• Test fixture massdrive_check_file now returns unchecked result and reference blobs for equality assertion (assert result == reference) to be done by the caller. This enables plugins like pytest-clarity to show colorful diff. Users of massdrive_check_file need to change (on pain of lack of test assertion):

- massdrive_check_file(workdir)
+ result, reference = massdrive_check_file(workdir)
+ assert result == reference, "Massdriver result should match reference"

v0.12.0 - 2023-04-05

Added

• Auto-detect repo’s base branch for Forge: parameter base_branch now optional, defaulting to repo’s default branch

v0.11.0 - 2023-04-02

Added

• New github-app forge plugin for creating PRs on Github when running mass-driver as a Github App

• New Forge params:

  • forge_config dict, for Forge-specific non-sensitive config to keep in config file, complementing envvars. Similar to driver_config for Migration.

  • interactive_pause_every int, for blocking the Forge, pausing for confirmation interactively every few PRs generated. Disabled by default, set to 1 to block every PR, or 5 every 5…

v0.10.0 - 2023-04-02

Removed

• Unused migration_name field of Migration now removed

Added

• Forge subclasses can now grab config via envvars prefixed FORGE_. Observe that Forge now derives from pydantic.BaseSettings, see BaseSettings docs.

• New, simpler testing fixture massdrive_check_file for PatchDriver that affect single files

v0.9.0 - 2023-03-12

Added

• New optional Migration params: commit_author_name + commit_author_email, used to override the git commit author.

v0.8.0 - 2023-02-24

Added

• New file type Activity combines Migration and Forge

• New Forge named dummy for testing purposes

• New git_push_first boolean param in Forge to disable git pushing.

• New ActivityOutcome to capture the full result of a migration/forge sequence

Changed

• Replace commands run-migration + run-forge by new run, using the Activity file type with optionals.

• Internals refactored: all Pydantic objects now under mass_driver.models (PatchDriver, Forge, Activity)

Removed

• Options --really-commit-changes and --dry-run

v0.7.0 - 2023-02-22

Added

• New ROADMAP.md to clarify The Plan.

Changed

• Upgrade minimum Python version to 3.11 to use tomllib

• Replace tomlkit with stdlib tomllib

Fixed

• Migrations, once loaded from TOML, are now proper dict again

• PatchDriver instance now independent across repos

v0.6.1 - 2023-02-06

Added

• Upgrade pre-commit dependencies + poetry in Dockerfile

• Update pyproject.toml for release to Pypi

v0.6.0 - 2023-02-04

Added

• Forges now discovered via setuptools

• New “run-forge” subcommand for creating PRs for on-disk branches

• Implemented GithubForge for PR creation on Github

• New “Stamper” PatchDriver for “stamping” new files on repos

Fixed

• Incorrect Repo creation causing silent cloning errors on some devices

v0.5.0 - 2023-01-24

Added

• New plugin system for including third-party PatchDrivers, based on the setuptools “entrypoints” system.

• CLI Subcommands for inspecting drivers, running migrations…

• PatchDriver test harness, for ease of plugin development.

• Project objectives (gherkin features in features/) now part of docs

Changed

• PatchDriver now inherits Pydantic BaseModel (allows serialization)

• PatchDriver func prototype: run(repo: Path) -> PatchResult

Removed

• Complex PatchDrivers (the ones requiring package dependencies) removed, moved to separate “plugins” repo.

v0.4.0 - 2022-11-16

Added

• New Jsonpatch Driver, for applying an RFC6902 JSON Patch

• New PreCommit Driver, for running pre-commit autoupdate

• New ShellDriver for running arbitrary shell commands

• New mass-driver(1) man page output to make docs

v0.3.1 - 2022-11-11

Changed

• PatchDriver API simplified, now using single func for detect + patch: called via PatchDriver.run(repo: Path, dry_run: bool).

• CLI simplified when exposing PatchDriver.run(), now uses either --dry-run (the default setup) or --really-commit-changes for committing (not pushing)

v0.3.0 - 2022-11-11

Added

• Poetry Driver stub for major version of packages in pyproject.toml. Showcases JSON Pointers (RFC6901) for structured fields modification.

v0.2.0 - 2022-11-11

Added

• Basic clone/detect/patch/commit workflow, with basic Github support

v0.1.0 - 2022-11-10

Added

• New python module mass_driver, exposed as shell command mass-driver