QtWebEngine/Rebase on New Chromium: Difference between revisions
Allan Jensen (talk | contribs) |
The Compiler (talk | contribs) (Suggest better way of getting patches) |
||
| (7 intermediate revisions by 2 users not shown) | |||
| Line 21: | Line 21: | ||
* Switch to latest xx-based branch | * Switch to latest xx-based branch | ||
* Call "git merge upstream-master" and push the merge change to gerrit. | * Call "git merge upstream-master" and push the merge change to gerrit. | ||
* Create commit in the main qtwebengine module where you include the | * Create a commit in the main qtwebengine module where you include the updated version_resolver.py and the src/3rdparty SHA1 at the same time. | ||
These steps can also be used if you need to use a new Chromium component that was previously trimmed away, or other files we trimmed away. | These steps can also be used if you need to use a new Chromium component that was previously trimmed away, or other files we trimmed away. | ||
| Line 29: | Line 29: | ||
Google does not maintain older versions of Chrome. So if we need security patches or important bug fixes in a branch that is no longer Google's stable version, we need to cherry-pick patches. | Google does not maintain older versions of Chrome. So if we need security patches or important bug fixes in a branch that is no longer Google's stable version, we need to cherry-pick patches. | ||
The easiest way is usually to find the commit in Chromium git, and store it using 'git | The easiest way is usually to find the commit in Chromium git, and store it using 'git format-patch -1 --src-prefix="a/chromium/" --dst-prefix="b/chromium/" COMMIT' or similar, and then apply and commit in our submodule. It is important to maintain most of the original commit message when doing so, especially the link to the Chromium review. Ensure though that the commit subject clearly indicates that this is a cherry-pick or backported patch so we will know to skip it when rebasing on a new version. Currently we usually do that by either starting with 'Cherry-pick of' or '[Backport]'. | ||
After cherry-picking a patch, again remember to update the src/3rdparty submodule in the main module using a separate commit. | After cherry-picking a patch, again remember to update the src/3rdparty submodule in the main module using a separate commit. | ||
== Rebase on new major version == | == Rebase on new major version == | ||
In theory switching to a new major version could work like a minor version update, or using simple git rebase commands, but in practice it is not that simple. And most of the scripts we tried to develop to automate the process doesn't really help. | |||
The first steps of a updating to a new major version are the same as for a minor version, but instead of a doing a merge to an existing branch, we create a new branch from the head of upstream-master. This you usually need to ask for sysadmin help for, either using JIRA or simply walking over and ask Ossi. | |||
Once you have a new submodule branch you can rebase all the patch from the last xx-based branch. In theory you can do this with a clever three way git rebase command, but it will often fail and require you to start over. In practice I have found it the simpliest to cherry-pick each patch one at a time. This is also sufficiently tedious that you are self-encouraged to squash changes so we have fewer changes next time. To help with squashing changes, the subject of changes can be prefixed with 'FIXUP: ' to indicate it should be squashed with another change during rebase. You will also often find changes that will need to dropped and some that will need to be rewritten during this process. | |||
Note that due to Chromium changing directory structures and implementing new features, you will almost always also need to update take_snapshot.py to include new modules, or strip out new large source or resource additions that we don't need. | |||
Once the submodule is done the core module needs to be updated. The API always changes and there are always new methods added, methods removed and methods that changes the number of arguments. Trivial adaptations to a new API can be grouped together in a single commit, but if something major needs to be reworked it is often better as a separate change that can reviewed separately, even if it is not functional separately. | |||
Note there are circular dependencies in fixing patches against the chromium submodule and in the webengine core module. In practice I often do a trial run merge to resolve these issues before doing a real rebase. A trial run can be done using a merge like with a chromium patch update, and brutally disable or hack around any big changes, until you can get the most basic version compiling. This helps establish exactly what needs to be included in take_snapshot.py when doing the real commit to upstream-master, and having run into all the compile issues also means you will have discovered if any of our patches can't or shouldn't be merged the trivial way. | |||
If you are doing merges two versions up like from 65 to 67, you can do the merge run 66 and the full rebase on 67, that way you deal with fewer changes to resolve and adapt to at a time. | |||
For Chromium merges it is beneficial to use the conflictstyle diff3, for instance "git config merge.conflictstyle diff3". This will give you both the common ancestor of the conflicting changes and each change making it much easier to see what is conflicting if you don't know the original change to begin with. | |||
Latest revision as of 14:12, 29 November 2021
Qt WebEngine keeps its branch of Chromium in the src/3rdparty/chromium submodule.
The submodule is a flattened and trimmed version of chromium where all the submodules of Chromium have been imported into a single module, and many large parts we don't need have been removed to save overhead.
The qtwebengine-chromium submodule has multiple branches.
- upstream-master is the pure snapshotted Chromum upstream
- xx-based are patched branches of Chromium xx with our patches applied to it.
In tools/scripts are a few scripts meant to help with the rebasing.
- init-repository.py can check out new version of Chromium matching requested versions
- take-snapshot.py can take a snapshot of an upstream Chromium version and apply our trimming logic to it, while copying it into src/3rdparty.
Rebase on new patch release
When updating to a new patch release of Chromium.
- Update tools/scripts/version_resolver.py with the new version
- Run "tools/scripts/init-repository.py --baseline-upstream" this will create src/3rdparty_upstream directory with Chromium from the given version
- Make sure src/3rdparty is checked out to 'upstream-master', and you have no files or changes there you want to preserve.
- Run "tools/script/take-snapshot.py" this will copy a trimmed version of src/3rdparty_upstream to src/3rdparty
- Stage all changes (new files and deleted files "git add -A") in src/3rdparty
- Commit and push to upstream-master
- Switch to latest xx-based branch
- Call "git merge upstream-master" and push the merge change to gerrit.
- Create a commit in the main qtwebengine module where you include the updated version_resolver.py and the src/3rdparty SHA1 at the same time.
These steps can also be used if you need to use a new Chromium component that was previously trimmed away, or other files we trimmed away. Instead of changing version_resolver.py, you update take_snapshot.py so it includes the wanted files, but perform the same steps.
Cherry-picking patches
Google does not maintain older versions of Chrome. So if we need security patches or important bug fixes in a branch that is no longer Google's stable version, we need to cherry-pick patches.
The easiest way is usually to find the commit in Chromium git, and store it using 'git format-patch -1 --src-prefix="a/chromium/" --dst-prefix="b/chromium/" COMMIT' or similar, and then apply and commit in our submodule. It is important to maintain most of the original commit message when doing so, especially the link to the Chromium review. Ensure though that the commit subject clearly indicates that this is a cherry-pick or backported patch so we will know to skip it when rebasing on a new version. Currently we usually do that by either starting with 'Cherry-pick of' or '[Backport]'.
After cherry-picking a patch, again remember to update the src/3rdparty submodule in the main module using a separate commit.
Rebase on new major version
In theory switching to a new major version could work like a minor version update, or using simple git rebase commands, but in practice it is not that simple. And most of the scripts we tried to develop to automate the process doesn't really help.
The first steps of a updating to a new major version are the same as for a minor version, but instead of a doing a merge to an existing branch, we create a new branch from the head of upstream-master. This you usually need to ask for sysadmin help for, either using JIRA or simply walking over and ask Ossi.
Once you have a new submodule branch you can rebase all the patch from the last xx-based branch. In theory you can do this with a clever three way git rebase command, but it will often fail and require you to start over. In practice I have found it the simpliest to cherry-pick each patch one at a time. This is also sufficiently tedious that you are self-encouraged to squash changes so we have fewer changes next time. To help with squashing changes, the subject of changes can be prefixed with 'FIXUP: ' to indicate it should be squashed with another change during rebase. You will also often find changes that will need to dropped and some that will need to be rewritten during this process.
Note that due to Chromium changing directory structures and implementing new features, you will almost always also need to update take_snapshot.py to include new modules, or strip out new large source or resource additions that we don't need.
Once the submodule is done the core module needs to be updated. The API always changes and there are always new methods added, methods removed and methods that changes the number of arguments. Trivial adaptations to a new API can be grouped together in a single commit, but if something major needs to be reworked it is often better as a separate change that can reviewed separately, even if it is not functional separately.
Note there are circular dependencies in fixing patches against the chromium submodule and in the webengine core module. In practice I often do a trial run merge to resolve these issues before doing a real rebase. A trial run can be done using a merge like with a chromium patch update, and brutally disable or hack around any big changes, until you can get the most basic version compiling. This helps establish exactly what needs to be included in take_snapshot.py when doing the real commit to upstream-master, and having run into all the compile issues also means you will have discovered if any of our patches can't or shouldn't be merged the trivial way.
If you are doing merges two versions up like from 65 to 67, you can do the merge run 66 and the full rebase on 67, that way you deal with fewer changes to resolve and adapt to at a time.
For Chromium merges it is beneficial to use the conflictstyle diff3, for instance "git config merge.conflictstyle diff3". This will give you both the common ancestor of the conflicting changes and each change making it much easier to see what is conflicting if you don't know the original change to begin with.